XGBoost (Extreme Gradient Boosting) is an open-source library that implements gradient boosting with decision trees particularly efficiently. Many small trees are trained one after another, each correcting the errors of the previous ones, while regularisation helps prevent overfitting. XGBoost is regarded as a proven choice for tabular data, for example in credit risk scoring, churn prediction or sales forecasting.
Gradient boosting is a machine learning technique that combines many simple models – usually small decision trees – one after another. Each new tree is trained specifically to reduce the remaining errors of the previous ones. The result is highly accurate models for tabular data, for example for credit risk, sales forecasts or churn probability. Well-known implementations are XGBoost, LightGBM and CatBoost; careful tuning is needed to avoid overfitting.
LightGBM is an open-source library for gradient boosting with decision trees, originally developed by Microsoft. It is designed for speed and low memory use, among other things through histogram-based training and trees that grow leaf-wise at the most promising leaf. LightGBM is particularly suited to large tabular datasets, for example sales forecasts, churn prediction or fraud detection, and is often compared with XGBoost.
Boosting is an ensemble method in machine learning in which many simple models are trained one after another, each new model focusing on the cases its predecessors got wrong. Together they form a much more accurate overall model. Well-known variants are AdaBoost and gradient boosting, with libraries such as XGBoost or LightGBM. Boosting is particularly strong on tabular data, for example credit risk, sales or churn forecasts.
Random forest regression predicts numerical values by combining many decision trees. Each tree is trained on a random sample of the data and a random selection of features; the result is the average of all the trees' predictions. This makes the method more robust and less prone to overfitting than a single tree. It is used, for example, for sales forecasts, price estimates or energy demand – often as a solid starting point for tabular data.
