Activation functions determine how strongly a neuron in an artificial neural network passes on its signal. They introduce non-linearity – only then can the network represent complex relationships such as language or images. Common examples are ReLU, which sets negative values to zero, sigmoid for probabilities between 0 and 1, and GELU, used in many transformer models. The choice affects training speed and stability.
Artificial neural networks (ANNs) are computational models made up of many connected units (“neurons”) arranged in layers that process inputs via weighted connections. During training, the weights are adjusted step by step using examples, usually by backpropagation. ANNs are the foundation of deep learning and power image recognition, speech recognition and large language models. Despite their biological inspiration, they work far more simply than the human brain.
Backpropagation is the standard method by which neural networks learn. After each prediction, the error is measured and propagated backwards through the network's layers, showing how much each weight contributed to it. An optimisation method such as gradient descent then adjusts the weights step by step. Without backpropagation, training today's language and image models would be practically impossible.
Deep learning is a branch of machine learning that uses artificial neural networks with many layers. Such networks learn from large amounts of data which features matter – from edges in images to relationships of meaning in texts. Deep learning underpins today's language models, image and speech recognition and machine translation, and usually requires considerable computing power and training data.
