This isn't how intelligence works. The LLM may not be able to play chess directly through inference, but it can write a program to do it and execute that program. Same as how human intelligence works. We can't fly, but we can build planes.
This isn't how intelligence works. The LLM may not be able to play chess directly through inference, but it can write a program to do it and execute that program. Same as how human intelligence works. We can't fly, but we can build planes.
Human beings can play chess directly without coding up a tool.
Asking an LLM to play chess by writing algebraic notation is like asking a human to play chess blindfolded.
Yes some people can do it but most people can't even if they're unusually intelligent.
You really need to be giving the LLM a board representation.
EDIT: I see that they actually were giving the LLMs a board representation and they still played badly. Fair enough then.
If they wanted to train an LLM to play chess they could easily do so.
But nobody wants that.
Very poorly compared to the tools we have built. Similar to the LLM.
Comparing to raw LLMs? Much much better.
Poorly in what sense? I think human chess leagues are way more popular and fun than just playing a computer by yourself. Human oriented communities are always a vastly better experience than their digital counterparts.
There's more to games than simply winning you know.
Thanks for saying this, feels like everyone has gone insane over this stuff.
Humans don’t code a $game engine to play $game, they can just play it. It seems like you are the one that has gone insane.
And how many years of direct play and study does it take for a human to get good at chess or any other game? Absolutely no human ever could be good at chess just by reading a few books, or even every book on chess. That's just not how the brain works. If LLMs could do that they would truly be superintelligence.
> Absolutely no human ever could be good at chess just by reading a few books, or even every book on chess.
Maybe not, but you'd be surprised how little it takes.
A six year old child can learn the rules of chess well enough to be able to play legal moves only in a single day. And they can improve their game at a pace which is almost frightening to behold. I have taught children, and I've witnessed significant improvement materialise in a single game. LLMs have probably thousands of chess books, games, videos, etc in their training data, yet they are unable to even follow the rules.
This is, at the very least, interesting. It illustrates many of the things brains can do, which current ML systems in general, and LLMs in particular, can't.
> And how many years of direct play and study does it take for a human to get good at chess or any other game?
Time is irrelevant to training; the more relevant comparison is "how many games does a human need to play to get diminishing returns".
No, learning is definitely not a sign of super intelligence. I know words don’t mean anything anymore, but that is simply general intelligence, despite the claims we have reached this milestone.
No, but superhuman capabilities derived from ordinary learning is, which is what the parent comment described. Why is that not obvious?
A week. My brother learned and was above 1100 online within 12 hours, after a few hundred games.