Random forest regression predicts numerical values by combining many decision trees. Each tree is trained on a random sample of the data and a random selection of features; the result is the average of all the trees' predictions. This makes the method more robust and less prone to overfitting than a single tree. It is used, for example, for sales forecasts, price estimates or energy demand – often as a solid starting point for tabular data.
A decision tree is a model that represents a decision as a sequence of simple yes/no questions: each branch tests one feature and each leaf gives an outcome. In machine learning, such trees are learnt automatically from data and used for classification or forecasting. Their main strength is transparency – for instance, showing why a loan application was flagged based on income, term and payment history.
Bagging (short for bootstrap aggregating) is an ensemble method in machine learning: many models are trained on randomly drawn samples of the training data, and their predictions are averaged or combined by majority vote. This makes results more stable and reduces overfitting. The best-known example is the random forest, which combines many decision trees – for instance to predict customer churn or prices.
Gradient boosting is a machine learning technique that combines many simple models – usually small decision trees – one after another. Each new tree is trained specifically to reduce the remaining errors of the previous ones. The result is highly accurate models for tabular data, for example for credit risk, sales forecasts or churn probability. Well-known implementations are XGBoost, LightGBM and CatBoost; careful tuning is needed to avoid overfitting.
Linear regression is a statistical method that describes the relationship between a target variable and one or more predictors using a straight line or plane. The coefficients are usually chosen to minimise the sum of squared deviations. The method is easy to understand and interpret: for example, it can estimate how much revenue rises when the advertising budget increases by €1,000. It reaches its limits with non-linear relationships.
