The ability to solve general problems previously insoluble using only experiential data and modification to internal world models.
The ability to solve general problems previously insoluble using only experiential data and modification to internal world models.
So, like ARC-AGI-3? I'm sympathetic to the notion that the things we're able to measure are necessarily going to miss important aspects of capability and intelligence. But people are attempting to measure this kind of thing, and models keep getting better at it.
And yet, curiously, they fall flat on their silicone asses until they have a copy of the test to "train on".
Yes, except actually novel.
They've read a few thousand years of experiential data in text form.
That's not true and its exactly the point - real life data is noisy, difficult to parse, and ultimately often ambiguous. Text is the distilled information. It's significantly easier to distilled text into a likely response than it is to distill into lessons a continuous data stream whose contents are partially controlled by your own actions.
Yeah but we're talking about something that was trained on petabytes of noisy, difficult to parse and ambiguous information.