Underfitting occurs when a model is too simple to capture the relationships in the data. It then makes many errors on the training data as well as on new data, for example when a straight line is used to represent a clearly curved pattern. Remedies include a more powerful model, more informative features, longer training or less regularisation; the opposite problem is overfitting.
Overfitting occurs when a model memorises the training data, including random quirks and noise, instead of capturing generally valid patterns. It then appears very accurate on familiar data but performs poorly on new data. A large gap between training and test error is a typical sign. Countermeasures include more or more varied data, simpler models, regularisation, dropout and cross-validation.
The generalisation error measures how far off an AI model is on new, previously unseen data – as opposed to its error on the training data. A large gap between the two points to overfitting: the model has memorised examples instead of learning general patterns. It is estimated using held-out test data or cross-validation. It is the figure that really counts, because in practice a model almost only meets new cases.
Training error indicates how far a model's predictions deviate from the correct values on exactly the data it was trained on. On its own it says little: what matters is the comparison with the generalisation error on new data. A high training error points to underfitting; a very low training error combined with a high error on test data usually signals overfitting.
Regularisation covers techniques that stop a model from fitting its training data too closely during training (overfitting). Examples include penalising large weights (L1 or L2 regularisation), randomly switching off neurons (dropout) or stopping training at the right moment (early stopping). The aim is a simpler model that performs more reliably on new, unseen data, even if it scores slightly lower on the training data.
