Glossary

Convergence

What is Convergence?

In machine learning, convergence means that a training procedure approaches a stable state: the loss barely decreases any more and the model parameters change only minimally. Whether and how quickly a model converges depends on factors such as learning rate, data quality and model architecture. If the learning rate is too high, the loss jumps around; if it is too low, training takes unnecessarily long. Convergence does not guarantee a good model, as it may still be overfitted.

Topics

Training
Optimisation
Machine learning

Try KI.ordo

Agents, flows and templates in one workspace – free 24-hour demo.

Start demo

← All terms

Related terms

Gradient descent

Gradient descent is the optimisation method used to train most AI models. You measure how large the model’s error is, calculate in which direction the parameters must change to reduce it, and then take a small step in that direction – again and again. It is like a hiker in fog who keeps walking downhill to reach the valley. The step size (learning rate) determines speed and stability.

Optimisation
Model training
+1

Learning Curve

In machine learning, a learning curve is a chart showing a model's performance or error over training time or the amount of training data, usually separately for training and validation data. Its shape reveals whether a model is underfitting or overfitting and whether more data is likely to help. If the validation error stays well above the training error, for example, this points to overfitting. In everyday language, the term also describes how quickly people master a new technology.

Training
Model evaluation
+1

Training Error

Training error indicates how far a model's predictions deviate from the correct values on exactly the data it was trained on. On its own it says little: what matters is the comparison with the generalisation error on new data. A high training error points to underfitting; a very low training error combined with a high error on test data usually signals overfitting.

Machine learning
Evaluation
+1

Hyperparameter Optimisation

Hyperparameters are settings fixed before a model is trained, such as the learning rate, the number of layers or the strength of regularisation. Hyperparameter optimisation systematically searches for the combination that performs best on validation data, for instance through grid search, random search or Bayesian optimisation. Well-chosen values improve accuracy and reduce over- or underfitting, but the search costs computing time.

Training
Model quality
+1