Obligatory "The Conspiracy Against High Temperature Sampling":
https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...
Obligatory "The Conspiracy Against High Temperature Sampling":
https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...
where can one learn what top_k and top_p mean?
https://www.longformexplainers.com/llm-inference/10-sampling
ironically, any frontier LLM will easily generate a tutorial at any detail you like explaining what these are.
if don't have time for that, just know that these are technical parameters that affect how likely it is an llm will produce the same result after being asked the same question.
https://www.ibm.com/docs/en/watsonx/saas?topic=prompts-model...