Listen: Temperature Parameter for Controlling AI Randomness

When a generative AI model predicts the next word in a sentence, it calculates a probability score for every word in its vocabulary. To control how creative or predictable the final choice is, we use a setting called temperature.

Temperature works by mathematically adjusting these probability scores. A low temperature, between zero and zero-point-seven, makes the most likely words even more dominant. The result is highly focused, conservative, and predictable text. For example, given the phrase, "The cat sat on the," a low-temperature model will almost always choose common words like "mat" or "couch."

If you raise the temperature, say to one-point-five, you flatten the probabilities. This gives less common words a fighting chance, boosting randomness and creativity. In this high-temperature setting, the cat might end up sitting on a "keyboard" or a "moonbeam." However, if you push the temperature too high, the text can quickly become nonsensical. Setting the temperature to exactly one simply uses the model's default, unaltered settings.

While temperature reshapes the probability of every word, a related setting called top-p sampling works differently. Top-p doesn't change the probabilities themselves; instead, it dynamically cuts off the least likely options entirely. By using temperature and top-p together, you can encourage the model to be highly creative while still keeping its output coherent.