That's the thing about machine learning over time, and the big hole in the whole world model idea. Time and time again we make simulations and train models and we think that loop will work. It never does.
And why not? Because the first model that can actually learn to handle the simulation not only can do the simulation internally too, but it can do it better than out simulation.
It's where (ex-)"AI luminaries" go to die. It's a way to make glacial, slow progress in the correct direction while never actually getting anywhere.
So we get to a point and people keep making complicated simulations of one kind or another ... and then someone (some model) who's never used the simulation, and doesn't need it for model training gets by far the highest score on every benchmark and everyone agrees the simulation was kind of beside the point.
It depends if the goal is a more accurate simulation, or if the goal is a tool for creatives to build games or content.
If it's the latter, I don't see a 'hole'.
My point was that there is also the possibility of AI intelligence improving to the point that there isn't a difference anymore. That whatever beats the next set of benchmarks (which of course won't be the simulator or world model) is so smart it is both a better solution to the problem the world simulator poses and a better world simulator all in one and a better tool for creatives to build games or content.
This has already happened to other ML fields. For example, there's not much point anymore to ASR models (just feed audio into LLMs) and the space that TTS models occupy is fast disappearing. Oh and it's not just that Qwen 3.6 is a good ASR in addition to being good at coding mario. Doing things this way, not using an ASR model is fundamentally more expressive. A whole bunch of companies started up on ASR + TTS "towers" (lots of models, Pipecat style) as they're called, and they're dead companies walking. Their tech has been surpassed by just bog-standard generic LLMs in a way that they will never get the upper hand again.
A chain of models approach, a separate ASR, will never do this: "you said you agree but I can hear you're sarcastic, so I'll explain again". Also getting a conversational tower to switch languages when the user does that (or when it makes sense in a language course) is not quite impossible but very, very hard. And yet feeding audio directly into Qwen will almost magically solve these problems better than I will ever do it (lightly finetuned). It's fundamentally better in every way (except perhaps speed, of course. Although arguably such a tower needs an LLM too, it'll just have way less tokens)
This point has long been passed for image recognition. Feeding a photo or screenshot into a full LLM along with a request on what to do beats the crap out of any OCR, image recognition, ... model every time these days.
And now there's also hints of world models just being bypassed. Just giving a finetuned model the last 50 video frames (surprisingly, that fits in context, especially at reduced resolution), movement statistics, motor power, status, ... of a robot and asking what to do actually works (with heavy finetuning, but I think it'll start working without). Well ... AND then distilling that knowledge into a much smaller model because B200 racks are more than a bit to heavy for a logistics robot to carry around. But we'll get over that.
Its easy to see what works in hindsight, but in reality with this sort of research you have to find an approach that’s working and keep pushing it and see where it goes.
The results so far from genie have been stunning, and its not obvious to me that another approach is guaranteed to end up better for all use cases.
> Its easy to see what works in hindsight
Yes! Hence my comment about world models being where (ex-)"AI luminaries" go to die. They had a great idea and pushed it, advancing their career, but as you point out this does not provide even them with a better perspective on where things are going. When it comes to the next big thing they have no better chance than a fresh grad student has.
But simulation (in this round, world models) is how you get the current thing to perform 1% better. So they work on that, and because of their reputation loads of money and resources pours into it because they chose it ... and then the solution comes from a different place entirely.
Except things are changing too, and world models like Google and Meta and Mistral and ... are pursuing appear to be getting bypassed by the current tech. It's starting to look a lot like the "internal world model" of a transformer that you've shown 1 million youtube movies is better than a separate world model, probably because the resources they pour into their general chatbot models are still an order of magnitude above what they can pour into their world models.