Yes, I failed to anticipate environment/playground-based RL-training and how much difference that would make, although I think the broader point about limitations of lack of continual learning still stands, and I can't see these simulated environments ever become complex and/or varied enough to be any substitute for the need to learn on the job.
However, gotcha acknowledged, but my point was about the jaggedness of AI, and how huge the divide in capability is between the domains where RLVR works, primarily coding, hacking and math, and everything else. The success and scale of RLVR in areas such as coding only make this gap more noticeable.
This is what the frontier can do with a computer now.
https://stillwet.art/
Where are the verifiable rewards? It's simply not true that RLVR amenable domains are the only things LLMs are improving massively on. Even Vending Bench is improving.
Intelligence is a jagged beast. That's just the nature of things. We evolved in certain environments and situations and sometimes there's a mismatch and it causes all sorts of wonky things. We just call them funny names like optical illusions and cognitive biases. Most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters (Watson Test).