Glossary

Hyperparameter Optimisation

What is Hyperparameter Optimisation?

Hyperparameters are settings fixed before a model is trained, such as the learning rate, the number of layers or the strength of regularisation. Hyperparameter optimisation systematically searches for the combination that performs best on validation data, for instance through grid search, random search or Bayesian optimisation. Well-chosen values improve accuracy and reduce over- or underfitting, but the search costs computing time.

How KI.ordo helps

KI.ordo does not train models; a related control is the agents' 'temperature' slider, a run-time setting that governs how varied the answers are.

Learn more →

Topics

Training
Model quality
Tuning

Try KI.ordo

Agents, flows and templates in one workspace – free 24-hour demo.

Start demo

← All terms

Related terms

Overfitting

Overfitting occurs when a model memorises the training data, including random quirks and noise, instead of capturing generally valid patterns. It then appears very accurate on familiar data but performs poorly on new data. A large gap between training and test error is a typical sign. Countermeasures include more or more varied data, simpler models, regularisation, dropout and cross-validation.

model quality
training
+1

Cross-Validation

Cross-validation is a method for checking how well a model generalises to new, unseen data. In k-fold cross-validation, the data is split into k parts; the model is trained k times on k−1 parts and tested on the remaining part each time, after which the results are averaged. This helps detect overfitting and compare models or settings fairly, especially when little data is available.

Model evaluation
Validation
+1

Machine learning

Machine learning is a branch of artificial intelligence in which computers derive patterns from data instead of having every rule explicitly programmed. Depending on the task, models learn from labelled examples (supervised), from unlabelled data (unsupervised) or through rewards (reinforcement). Applications range from spam filters and demand forecasts to product recommendations. Quality depends heavily on the quantity and quality of the training data.

fundamentals
AI
+1

Regularisation

Regularisation covers techniques that stop a model from fitting its training data too closely during training (overfitting). Examples include penalising large weights (L1 or L2 regularisation), randomly switching off neurons (dropout) or stopping training at the right moment (early stopping). The aim is a simpler model that performs more reliably on new, unseen data, even if it scores slightly lower on the training data.

training
overfitting
+1