The training stage that aligns a model to human preferences using a reward model trained on human rankings; the main lever behind alignment, distinct from fine-tuning on labelled examples.
RLHF, reinforcement learning from human feedback, is the training stage that aligns a model's output with human preferences. After pre-training teaches raw language ability, people rank competing model answers, those rankings train a reward model, and the language model is then optimised to score well against that reward, learning to be helpful, follow instructions, and avoid harmful replies.
It is distinct from fine-tuning on labelled examples: RLHF optimises against a learned preference signal rather than fixed target text, which is what shapes a base model into a usable assistant. Related methods replace or augment the human step, for instance training against AI-generated preferences or a written set of principles.
RLHF is the main lever behind alignment, the broader aim of getting a model to behave as intended. Alignment also decides quieter things that bear on AI visibility: how a model hedges, which sources it treats as trustworthy, and the biases it carries toward some brands or answers. A model's defaults about what to recommend are learned here, not only at retrieval time.
