Listen: RLHF
The training stage that aligns a model to human preferences using a reward model trained on human rankings; the main lever behind alignment, distinct from fine-tuning on labelled examples.
Transcript
Reinforcement Learning from Human Feedback, or RLHF, is the crucial training step that turns a raw language model into a helpful assistant. While initial training teaches a model how to write, RLHF teaches it how to behave.
During this process, humans rank different answers to the same prompt. Those rankings are used to train a reward model, which then guides the AI to score well by being helpful, following instructions, and avoiding harmful replies. This is different from traditional fine-tuning on fixed target text. Instead, the model optimizes against a learned preference signal. Some related methods even replace the human step, training the AI against machine-generated feedback or a written set of principles.
Ultimately, RLHF is the main lever behind alignment, the broader goal of getting a model to behave as intended. But it also shapes much quieter things. It decides how a model hedges its answers, which sources it treats as trustworthy, and the subtle biases it carries toward certain brands or answers. A model's default recommendations are learned during this stage, shaping its personality long before a user ever types a prompt.