Cross-entropy is a measure from information theory that indicates how far a predicted probability distribution deviates from the actual one. In machine learning, it is the most common loss function for classification tasks and also for language models that predict the next token. It penalises confident wrong predictions especially heavily: assigning 99% to the wrong class yields a much higher loss than assigning 60%.
Log loss (logarithmic loss) measures how well the predicted probabilities of a classification model match the actual outcomes. Confident but wrong predictions are penalised particularly heavily; a value close to zero indicates well-calibrated forecasts. Example: if a model gives a 95% churn probability for a customer who stays, the log loss rises sharply. Mathematically it is the same as binary cross-entropy.
Gradient descent is the optimisation method used to train most AI models. You measure how large the model’s error is, calculate in which direction the parameters must change to reduce it, and then take a small step in that direction – again and again. It is like a hiker in fog who keeps walking downhill to reach the valley. The step size (learning rate) determines speed and stability.
A classifier is a model or method that assigns inputs to one of several predefined categories. Classic variants such as decision trees or naive Bayes are trained on labelled examples; language models can also sort texts based on a description of the categories. Typical uses include spam detection, sorting customer enquiries by topic and categorising receipts. Quality is measured with metrics such as accuracy, precision and recall.
In machine learning, convergence means that a training procedure approaches a stable state: the loss barely decreases any more and the model parameters change only minimally. Whether and how quickly a model converges depends on factors such as learning rate, data quality and model architecture. If the learning rate is too high, the loss jumps around; if it is too low, training takes unnecessarily long. Convergence does not guarantee a good model, as it may still be overfitted.
