Q-learning is a fundamental reinforcement learning method. An agent learns by trial and error what long-term value (the “Q-value”) a particular action has in a particular situation, and over time chooses the most promising actions. It does not need a model of its environment to do this. Combined with neural networks (deep Q-learning), a DeepMind system used this approach to learn to play many Atari video games from screen pixels alone.
Reinforcement learning is a branch of machine learning in which an agent learns by trial and error. It acts within an environment, receives rewards for good outcomes and penalties for bad ones, and in this way develops a strategy that maximises its overall reward. Well-known examples include programs that master board games such as Go, robot control and the fine-tuning of language models based on human ratings.
RLHF is a training method that makes language models more helpful and safer. People compare and rate several of the model's answers; these judgements are used to train a reward model, which then further trains the language model itself through reinforcement learning. This teaches it, for instance, to answer more clearly or to decline problematic requests. RLHF takes place at the model provider before a model is released – not with individual users.
Monte Carlo methods are computational techniques that use a very large number of random samples to approximate results that are hard to calculate analytically. A system is simulated thousands of times with randomly varied assumptions, and the distribution of outcomes is analysed. Organisations use this for risk analysis, project scheduling or portfolio valuation. In AI, the methods are used in reinforcement learning and in tree search for games, among other areas.
Artificial neural networks (ANNs) are computational models made up of many connected units (“neurons”) arranged in layers that process inputs via weighted connections. During training, the weights are adjusted step by step using examples, usually by backpropagation. ANNs are the foundation of deep learning and power image recognition, speech recognition and large language models. Despite their biological inspiration, they work far more simply than the human brain.
