I tested this myself a few months ago, and confirmed that it was really happening.
LLMs don't know who they are unless the system prompt tells them, and as all of them are trained on model responses that exist on the web that end up being scraped, the weights may predict a certain incorrect response. LLMs have no ability to introspect, and do not know anything about themselves, so they will hallucinate in response to that question unless they are carefully trained on that exact, pointless question.
It's a surprisingly common misconception that models contain any metadata at all about themselves in their weights. If you ask them, "What model are you?" they either retrieve the answer from the system prompt, or they hallucinate an answer. Same goes for questions about knowledge cut-off, how many parameters they have, the source of their training data, etc.
Claude Sonnet 4.6 identified itself as DeepSeek repeatedly: https://www.reddit.com/r/DeepSeek/comments/1rd5jw7/claude_so...
I tested this myself a few months ago, and confirmed that it was really happening.
LLMs don't know who they are unless the system prompt tells them, and as all of them are trained on model responses that exist on the web that end up being scraped, the weights may predict a certain incorrect response. LLMs have no ability to introspect, and do not know anything about themselves, so they will hallucinate in response to that question unless they are carefully trained on that exact, pointless question.
It's a surprisingly common misconception that models contain any metadata at all about themselves in their weights. If you ask them, "What model are you?" they either retrieve the answer from the system prompt, or they hallucinate an answer. Same goes for questions about knowledge cut-off, how many parameters they have, the source of their training data, etc.
Huh. That kinda makes sense. So you think it's hallucinating it's model name?
Then why does it score better than any Gemini model?
As I understand, some people tend to "distill" LLM models. Google hasn't released a new Pro version in a while. I'm not an expert in LLMs.