Glossary

Learning Curve

What is Learning Curve?

In machine learning, a learning curve is a chart showing a model's performance or error over training time or the amount of training data, usually separately for training and validation data. Its shape reveals whether a model is underfitting or overfitting and whether more data is likely to help. If the validation error stays well above the training error, for example, this points to overfitting. In everyday language, the term also describes how quickly people master a new technology.

Topics

Training
Model evaluation
Diagnostics

Try KI.ordo

Agents, flows and templates in one workspace – free 24-hour demo.

Start demo

← All terms

Related terms

Convergence

In machine learning, convergence means that a training procedure approaches a stable state: the loss barely decreases any more and the model parameters change only minimally. Whether and how quickly a model converges depends on factors such as learning rate, data quality and model architecture. If the learning rate is too high, the loss jumps around; if it is too low, training takes unnecessarily long. Convergence does not guarantee a good model, as it may still be overfitted.

Training
Optimisation
+1

Overfitting

Overfitting occurs when a model memorises the training data, including random quirks and noise, instead of capturing generally valid patterns. It then appears very accurate on familiar data but performs poorly on new data. A large gap between training and test error is a typical sign. Countermeasures include more or more varied data, simpler models, regularisation, dropout and cross-validation.

model quality
training
+1

Underfitting

Underfitting occurs when a model is too simple to capture the relationships in the data. It then makes many errors on the training data as well as on new data, for example when a straight line is used to represent a clearly curved pattern. Remedies include a more powerful model, more informative features, longer training or less regularisation; the opposite problem is overfitting.

Machine learning
Model quality
+1

Training Error

Training error indicates how far a model's predictions deviate from the correct values on exactly the data it was trained on. On its own it says little: what matters is the comparison with the generalisation error on new data. A high training error points to underfitting; a very low training error combined with a high error on test data usually signals overfitting.

Machine learning
Evaluation
+1