I'm imagining everything the LLM could produce (with a given top_k setting) would be shades of green to yellow. Just outside of that orange, and far outside red.
LLMs generate new functions all the time, I'd guess these would be light green, maybe the first token in the name would be yellow and it would get brighter green as the name unfolds.
The logits are probably all small when in the global scope where it's not clear what will be defined next. I'm not imagining mapping logits directly to heat, the ordering of tokens seems much more appropriate.
No, I mean when an LLM encounters a previously unseen name it doesn't expect it so it would be red, even though it's perfectly valid.
I'm imagining everything the LLM could produce (with a given top_k setting) would be shades of green to yellow. Just outside of that orange, and far outside red.
LLMs generate new functions all the time, I'd guess these would be light green, maybe the first token in the name would be yellow and it would get brighter green as the name unfolds.
The logits are probably all small when in the global scope where it's not clear what will be defined next. I'm not imagining mapping logits directly to heat, the ordering of tokens seems much more appropriate.
I don't think that's necessarily true. I've definitely seen LLMs hallucinating variables they never defined.