The state machine with a random number generator is soundly beating some people in cognition already. That is, if the test for intelligence is set high enough that chatgpt doesn't pass it, nor do quite a lot of the human population.
If you can prove this can't happen, your axioms are wrong or your deduction in error.
I’m beginning to feel like the tests are part of the problem. Our intelligence tests are all tests of specialisation. We’ve established LLMs are part of the problem. Plenty of people who would fail a bar exam yet still know how many Rs there are in strawberry, could learn a new game just by reading the rules, know how to put up a set of shelves.
I think the problem is that, as far as we can tell, AIs are just more generally intelligent than humans and people are trying to figure out how to assert that they are not. A specialist human in their area of competence can still outperform an AI, but there don't seem to be any fields now where a human novice can reliably out-think a computer.
We're seeing a lot more papers like this one where we have to define humans as non-general-intelligences.
If you rarely got to see letters and just saw fragments of words as something like Chinese characters (tokens), could you count the R's in arbitrary words well?
The bigger issue is LLMs still need way way more data than humans get tons what they do. But they also have many less parameters than the human brain.
> If you rarely got to see letters and just saw fragments of words as something like Chinese characters (tokens), could you count the R's in arbitrary words well?
While this seems correct, I'm sure I tried this when it was novel and observed that it could split the word into separate letters and then still count them wrong, which suggested something weird is happening internally.
I just now tried to repeat this, and it now counts the "r"'s in "strawberry" correctly (presumably enough examples of this specifically on the internet now?), but I did find it making the equivalent mistake with a German word (https://chatgpt.com/share/6859289d-f56c-8011-b253-eccd3cecee...):
But even then, having it spell the word out first, fixed it: https://chatgpt.com/share/685928bc-be58-8011-9a15-44886bb522...Would you consider those who fail intelligent?