A decoding method that samples the next word from the smallest set of top candidates whose probabilities sum to a threshold p.
Top-p sampling, also called nucleus sampling, controls a model's randomness by limiting which words it will consider next. It sorts every candidate word by probability, adds them up from the top, and keeps only the smallest set whose cumulative probability reaches a threshold p — then samples from that reduced set, weighted by probability.
A low p (say 0.1–0.5) keeps only the most probable words, giving focused, predictable output; a high p (0.75–0.95) admits a wider range, allowing more diverse and creative text at higher risk of drifting off-topic. Setting p to 1 disables the filter entirely, considering all words.
Where temperature rescales the whole distribution, top-p instead trims its tail — the two are often used together. Both act on the model's underlying token probabilities, the same signal our Tree Walker inspects.
