Cross-validation is a method for checking how well a model generalises to new, unseen data. In k-fold cross-validation, the data is split into k parts; the model is trained k times on k−1 parts and tested on the remaining part each time, after which the results are averaged. This helps detect overfitting and compare models or settings fairly, especially when little data is available.
Overfitting occurs when a model memorises the training data, including random quirks and noise, instead of capturing generally valid patterns. It then appears very accurate on familiar data but performs poorly on new data. A large gap between training and test error is a typical sign. Countermeasures include more or more varied data, simpler models, regularisation, dropout and cross-validation.
The generalisation error measures how far off an AI model is on new, previously unseen data – as opposed to its error on the training data. A large gap between the two points to overfitting: the model has memorised examples instead of learning general patterns. It is estimated using held-out test data or cross-validation. It is the figure that really counts, because in practice a model almost only meets new cases.
Hyperparameters are settings fixed before a model is trained, such as the learning rate, the number of layers or the strength of regularisation. Hyperparameter optimisation systematically searches for the combination that performs best on validation data, for instance through grid search, random search or Bayesian optimisation. Well-chosen values improve accuracy and reduce over- or underfitting, but the search costs computing time.
Training data are the examples from which an AI model learns patterns: large text collections for language models, labelled images for image models. Their quality, diversity and recency largely determine how reliable and fair a model is and how up to date its knowledge will be. For organisations it also matters whether a provider uses the customer data they enter for further training.
