Explain temperature and top-p. How would you set them for different tasks?

What the interviewer wants to hear

• Both control randomness in token sampling
• Temperature reshapes the probability distribution
• Top-p limits sampling to the most likely tokens up to a cumulative probability
• Low values for classification and extraction, higher for creative work
• Usually tune one at a time and test the effect

Model answer

Both settings control how random the model's word choices are. At each step the model has a probability for every possible next token. Temperature reshapes those probabilities: a low value makes the top choice dominate, so output is focused and consistent, while a high value flattens them, so output is more varied. Top-p, or nucleus sampling, keeps only the smallest group of likely tokens whose probabilities add up to p and samples from that group, cutting off the unlikely tail. For classification, extraction or anything parsed by code, I use a low temperature for consistency. For brainstorming or marketing ideas, I raise it to get variety. I usually change one setting, not both, and I check the effect on a test set, because the best value depends on the model and task.

More easy Prompt Engineer questions

• What makes a good prompt? Walk me through how you would write one for a new task.
• When would you use zero-shot prompting and when would you add few-shot examples?
• How can you use prompting to reduce hallucinations?
• Your product team needs model output that software can read reliably. How do you design the prompt?
• Describe your process for improving a prompt that is giving mixed results.
• Explain retrieval-augmented generation (RAG) to a non-technical colleague. Why would we use it?
• What are tokens and context windows, and why do they matter when you design prompts?
• What is chain-of-thought prompting, and when would you use it or avoid it?
• A prompt works on your demo examples but fails for some users. How do you find out why?