I don't understand why changing effort levels busts the cache. Couldn't effort levels be a decoding-only thing where they just change the probability of the <end of thought> token? Are they literally adding a hidden system prompt that says "effort level: $level" ?
> Are they literally adding a hidden system prompt that says "effort level: $level" ?
Yes. https://magazine.sebastianraschka.com/p/controlling-reasonin...
Obviously it‘s something being put in the context that can not be taken out of it anymore.