The challenge with estimating abilities, is that we don’t know what the models can achieve if we just burn enough money. The navier-stokes shows us what mathematical problem can be solved when $10m worth of compute is thrown at something.
It also makes one wonder: What could AI solve if we managed to orchestrate billions worth of agents to take on a specific problem?
IMO the very best case scenario / potential for these are likely better than we think, but right now hidden due to logistical and financial reasons.
But if we assume that the model costs will continue to drop by a factor of 5-10 annually, there will always be a latency of a couple of years between what is completely out of reach, and what is financially viable.
Basically: If you knew AI could be affordable enough in 3-5 years so that even the most underfunded researchers could use it to solve cancer, how much would you value it now?
The whole point of this post was that it's questionable what can be achieved without huge investments into oversight and steering, because navier-stokes was a topic with an unusual level of specification. The problem itself was a specification. Such situations are rare in real-world scenarios.
AI agents are good at solving well-specified tasks, not at solving problems. They do well in fields where the cost/effort of specification is already part of the business.
> a topic with an unusual level of specification
Solving cancer also has an unusual level of specification. Many real world problems have that characteristic.
Wasn't Navier Stokes solved by ripping off a researcher's private chat log?
> The navier-stokes shows us what mathematical problem can be solved when $10m worth of compute is thrown at something.
That's the thing, it very much does NOT show us that. What happened was mathematicians at openAI learned of an imminent development on this problem, and the insight that it entailed, then they were able to prompt a system in the correct direction and spend 20 million dollars to write down the final steps.
Which is rather precisely the point that the article is making!
> If you knew AI could be affordable enough in 3-5 years so that even the most underfunded researchers could use it to solve cancer
As the saying goes, if my grandmother had wheels she would have been a truck.