Temperature

Temperature is a parameter applied during the text generation process that controls how random or deterministic the model's output will be. Technically, it scales the logits (raw prediction scores) before they are converted to probabilities for selecting the next token. A temperature of 0 makes the model always choose the highest-probability token (greedy decoding), while higher temperatures flatten the probability distribution, giving lower-probability tokens a better chance of being selected.

In practical terms, lower temperatures (0.0-0.3) produce more predictable, consistent, and focused outputs ideal for tasks requiring accuracy like data extraction, code generation, and factual question answering. Medium temperatures (0.4-0.7) balance creativity with coherence, suitable for general writing and conversation. Higher temperatures (0.8-1.5) produce more creative, diverse, and sometimes surprising outputs, useful for brainstorming, creative writing, and generating varied suggestions.

Understanding temperature is important for building reliable AI applications. For customer-facing chatbots where consistency matters, you want low temperature. For creative tools where variety is valued, higher temperature works better. Many developers mistakenly use high temperatures for all tasks, not realizing that it can increase hallucinations and reduce the quality of factual responses. Temperature is often used in conjunction with top-p (nucleus sampling) for fine-grained control over output diversity.