Reinforcement learning is a branch of machine learning in which an agent learns by trial and error. It acts within an environment, receives rewards for good outcomes and penalties for bad ones, and in this way develops a strategy that maximises its overall reward. Well-known examples include programs that master board games such as Go, robot control and the fine-tuning of language models based on human ratings.
Q-learning is a fundamental reinforcement learning method. An agent learns by trial and error what long-term value (the “Q-value”) a particular action has in a particular situation, and over time chooses the most promising actions. It does not need a model of its environment to do this. Combined with neural networks (deep Q-learning), a DeepMind system used this approach to learn to play many Atari video games from screen pixels alone.
RLHF is a training method that makes language models more helpful and safer. People compare and rate several of the model's answers; these judgements are used to train a reward model, which then further trains the language model itself through reinforcement learning. This teaches it, for instance, to answer more clearly or to decline problematic requests. RLHF takes place at the model provider before a model is released – not with individual users.
In supervised learning, a model learns from examples for which the correct answer is already known, such as emails labelled “spam” or “not spam”. From these it derives rules to classify new, unseen cases or to predict values. Typical tasks are classification and regression; most of the effort goes into creating cleanly labelled data.
In unsupervised learning, a model receives data without predefined correct answers and searches for structures on its own. Typical methods are clustering, which groups similar records, and dimensionality reduction, which condenses many features into a few essential ones. Organisations use it to discover customer segments or spot unusual transactions; the groups found must then be interpreted by domain experts.
