It is mind blowing that these long-standing conjectures and open questions are just falling by the dozen every other week. What a time to be alive.
It is mind blowing that these long-standing conjectures and open questions are just falling by the dozen every other week. What a time to be alive.
How long until we see similar announcements for (wet lab) chemistry, medicine or material science? Months? Years? Never?
(Never would mean AI will be used in future labs like a fancy calculator, but not much more changes)
I'll give you an example from my field (nuclear fusion). I have a small team working on an idea which I think would really advance the field. Our initial concept paper was published in 2019. Progress on it has been slow because we don't have enough manufacturing capabilities, budget, or time on the facility. Will AI solve that? Don't get me wrong, I'm pretty bullish on AI, and it's been phenomenally useful so far (I definitely feel like a 10x physicist in the last 6 months), but I don't see a swarm of agents putting fusion energy on the grid tomorrow.
Also, fwiw I have a personal test problem I give to every frontier model on release, which is to ask it to final a realatively obvious solution (in hindsight) to a minorly important open problem in my field. I know the answer, but havent gotten around to writing it up yet (it's really not that important). Every frontier model (so far) just regurgitates existing solutions.
How much effort do you put into prompting it? Are you asking it one shot with a simple prompt? Are you giving it to multiple agents and giving them a workspace to collaborate in? You might give it to a trusted colleague who does not know the solution along with an agreement to jointly publish, ask them to go through the process of solving it with agentic support, and then compare your solutions.
Edit: to be clear, I completely believe your experience - just pointing out that it might be interesting for you to actually “try” to solve it thoroughly with the tools, though it is obviously hard to do that in a fair way when you know the solution.
Enough! I think it's a reasonably fair test of the models. Let's put it this way, I use the same process to resolve problems outside my peak expertise, and I find it works well.
I think the solution is easy enough that any competent colleague would come up with it. Some variant on it has probably even been published (hence why I'm not chomping at the bit to publish). I think it comes down to the models being extremely spiky, and the problems they're trying to solve being easily verifiable in code.
Have you ever told the AI your answer? I bet if you did it would train on it and know it going forward.
Nah. And I have an enterprise plan, so I'm /relatively/ confident that they aren't going to do that.
With my personal little test problem, that never made a difference.
However current frontier models got good enough at websearch and thinking through the problem, that they can finally piece together the answer from my own public writing (eg on stackoverflow). And recently someone wrote up an answer to the problem and put it on Arxiv.
Iirc some of the prior math discoveries involved very very very heavy handed guidance to force the AI to discard prior assumptions, and to try many avenues with MAX effort. So ya models don’t behave like ground breaking theoreticians by default.
Do a breakthrough!
Ha, I had a similar little test problem, but the frontier models are now smart enough to read enough online of what I posted here and there to piece together the answer. And someone else has written the paper in the meantime and got it accepted for publishing.
Definitely. It will let some smooth-talker with a vibe-coded website raise billions for "Nuclear Fusion AI". Then they can use AI to squeeze your team with lawsuits until you agree to sell cheap, and maybe keep you on as an employee to build it.
Hah. Fusion scammers are a dime a dozen. Unlike SaaS apps or finance, though, you can't sweet talk a neutron into existence, so they fizzle out pretty fast. I'm more worried about fusion dying because nobody comes up with a good enough idea than someone stealing my ideas.
Going on a tangent: I think fusion is pretty much useless this century. We already have energy generation technology that has high capital costs and low running costs: it's called nuclear fission and works fairly well.
Fission hasn't exactly taken the world by storm.
There are a few nice advantages fusion has, but they are more important once we want to reach the stars as far as I can tell; not this century on this planet.
But I'm just an armchair analyst. You probably have a much better understanding.
Nah I agree with you. Not this century, not this planet. Fusion rockets are really efficient, though ;)
I just find it to be an interesting problem! My hope is it happens before I die. My rule of thumb is anyone who says energy on the grid in < 10 years is a blatant liar.
How about swarm of agents controling a fleet of humanoids and automated construction vehicles?
I mean, you never know. If we could have simulation models with excellent fidelity, then AI could do all the research and iterations and come up with something very plausible. Cutting down physical world experimental iterations/prototyping.
So effectively, you would spend the first 1000 experiments inside a computer simulation and move to the physical world only when you had enough confidence.
I think you should realize that we already do that. In fact, that's one of the reasons computers were invented in the first place! It turns out that science is about unknown unknowns, and by definition simulations can't answer that.
But agents are really useful for doing some of the really boring stuff, like convergence checks or setting up meshes or whatever.
If you're talking about things at the equivalent level of say, finding out if positrons maintain symmetry with electrons for their magnetic moment, then yeah - I hear you. Or if you wanted to research some unknown behaviour of hot plasma at 15 million C, you can't do that in a simulation.
..or can we?
Humour me for a moment. Maybe simulating to the degree that was necessary for cutting edge work was not possible earlier, when humans had to program each and every thing in the system that could affect the result. But with current gen petaflop-on-your-desk level compute + swarms of agents that can work tirelessly to account for every known thing to humanity, maybe the threshold has been pushed far enough to produce some more meaningful discovery work in simulation?
I've got frontier models doing pretty sophisticated nuclear reactor design and analysis using open-source nuclear fission reactor tools using open-source reactor designs as baselines. It is WILD how good they do.
Nothing could go wrong
Doing AI assisted enggineering to explore a space and then using classical methods to validate a point design for licensing is a good approach. In licensing the vendor and the regulator check things and then also verify many times during startup. The overall safety remains good.
Up front: I totally believe you are doing your level best so please don't take this as trying to take you down, I just want to understand. In the hypothetical case that in 10 years time it turns out that that design was flawed who will be on the hook? You, the AI vendor, the regulator, the device vendor?
Nuclear liability is always 100% held by the owner/operator by law in most countries. All manufacturers of equipment and components demand and are given indemnity.
Well you can’t verify a prediction about human biology without a phase 3 clinical trial so…
The thing that I've not fully groked here (might be relevant) is that OAI is generating "paper" proofs first, and only then formalizes them in Lean as a separate step. They've as far as I'm aware not said anywhere how many write ups failed formalization due to a bug in the proof. If we assume that number is low, models would be quite good at one-shotting math proofs. Then it depends if you can replicate the math RL training for chemistry/biology/medicine. Is there enough data out there for that? No idea.
I'm seeing estimates of a %95-91 failure rate on attempts in generally (or, conversely, a max of ~10% success, but likely lower, since we can only go based on how many attempts they are willing to mention in their statements). E.g., from analysis in https://medium.com/@mehmet.ozel2701/openai-tried-about-4-000...
I wonder how many of those founder at the "paper" to formalization step, or at earlier or later steps (e.g., proof doesn't turn out to prove what the paper said, hits a bug in the kernel, etc.).
Wouldn't a bug in the (Lean) kernel just result in a PR to Lean?
based on how bad these models are at one-shotting complex real-world software engineering problems -- it always has to be a conversation -- there is absolutely no way they're one-shotting complex math proofs.
Also, goes without saying, but put Sam Altman in jail, obviously.
I would trade (as if I have any say lol) every single breakthrough they have made in math/CS for _any_ good progress towards a universal cancer cure (very theoretical, I know).
I'm kinda hoping they are investing just as much into that type of research. Its their money but man... I'd say 999/1000 people are unaffected by all these breakthroughs. They probably have no idea tons of legendary math/CS conjectures were proven true/false. If a breakthrough in cancer research were to be made, it would very quickly start effecting everyone on earth.
The time required to verify is much higher, so some combination of vastly speeding that up or vastly increasing the quality of individual "thought steps".
It’s accelerating. We can automate scientific discovery. Imagine the amount of knowledge we may have over the next several years. It could be staggering.
It seems that we have yet to see any new conjectures.
They have an advantages of a coordination layer above the attempts of their users plus the breadth of attention
How are people getting these results? Are they correct?
I use these latest available public models daily in max effort modes, and they barely seem to work and need extreme guidance and as much context and constraints as possible.
Exactly my thoughts.
I find that unless the outcome I'm looking for from the model is almost binary and easily verifiable, it is basically faster to do the work myself.
Assumptions made by the models multiply and by the non-deterministic nature (dbf, similar to work by humans) I'll have something end up light years away from where it should be. During a normal workflow, I have to contend with every line I write forcing a very tight feedback loop... I suppose I could also sit there and spend 9 hours writing out a "perfect spec" but we've already tried that with waterfall development!
So, not sure if its the models, rather the class of problems I'm applying them to... but it is a bit ironic they are allegedly able to solve these conjectures but not stop to ask about assumptions.
1. This has not been my experience
2. We dont have the actual frontier models
internal unpublished models + large swarm of agents (>2000)
Chasing open questions, it seems to me, is worthwhile for the things learned along the way. Do you expect these results will be helpful, or is it just a sign that we need a new way to navigate uncharted spaces now that longstanding problems don't stand long enough for us to organize around?
It feels a bit like having "visited" a place because you had a layover there.
I hope it's more like discovering islands of information which we can use as footholds to connect the dots between them.
If knowing the answer is not valuable then asking the question wasn’t valuable to begin with.
We should remember that math exists primarily to serve a purpose and that purpose is not being a fun puzzle hobby for a select few mathematicians.
Imagine a programmer telling their boss it’s not about working software it’s about the journey of writing the code. Good luck getting hired.
I think if pro mathematicians don’t change their tune quickly they will soon see their funding approach zero.
I don't understand this dialogue, which seems to repeat with every result of this sort. Of course this is interesting! And of course the answer tells us something! And of course we also want to understand where it came from! None of this is in contradiction!
Knowing the answer to Fermat’s Last Theorem wasn’t valuable. But the pursuit of it led to many developments in other fields of mathematics that were valuable (where value is measured in terms of $).
I disagree. We climb mountains for the sake of it, and the rest of the world benefits because the gear improves and makes adjacent, non-for-the-sake-of-it activities safer.
We chase open problems for the sake of it, and others sometimes support us in doing so because the paths we pave along the way end up being useful for them for adjacent purposes. If you get the answer for free, you might have scratched the itch, but nothing was built for others to use, no fodder was found for the next great question.
Nobody pays mathematicians for solving famous problems. Famous problems are how mathematicians orient themselves while they do the other things that they're paid for (maintaining a useful community, mostly). All this does is deprive them of a navigational tool.
> Imagine a programmer telling their boss it’s not about working software it’s about the journey of writing the code. Good luck getting hired.
Your formulation is a little bit exaggerated, but I (and many, though far from all, programmers) would agree with the following more "tame" formulation:
Getting software to solve a business problem in an elegant and maintainable way means that we got the abstractions right. Similarly, if a software can easily and elegantly be modified to cope with new business problems, it is also a strong sign that we got the abstractions right. In this sense, the business problems that the software solves are indeed just waypoints on the journey of finding the proper elegant abstractions for putting the business problems into code.
Turning these novel insights about "good" abstractions into code and programming languages enables the team to iterate faster and generate business value faster. So, appreciating that it indeed all about the journey means a commitment towards an approach to accelerate the rate with which the team can turn business insights into code and code into business value.
I know that many managers do hate such a formulation, but I stand behind it.
This is such a short sighted take. Number theory was considered beautifully useless for centuries, and now it’s the backbone of the cryptography that secures every bank transaction. We don't know where the pursuit of knowledge will take us.
Thinking in such a utilitarian way feels like a flavor of Meditations on Moloch.[1] Just look at how short-term, quarter-to-quarter thinking is affecting the economy and the lived experience of every human on earth. It’s horrible.
[1] https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
> This is such a short sighted take. Number theory was considered beautifully useless for centuries, and now it’s the backbone of the cryptography that secures every bank transaction
If you refer to RSA or Diffie-Hellman: the number theory that it is based on is from 17th to at most 19th century, i.e. it precedes modern number theory.
I do consider modern number theory to be fascinating and do claim that many results from it could have huge applications in society, but the argument that parts of modern asymmetric cryptography is based on number theory is rather an argument that we don't need any number theory research anymore because these insanely old results that are used in cryptography have already been known for centuries - so no new research is necessary. :-(
We didn't know what investigations were going to lead to RSA or Diffie-Hellman and any particular breakthrough is standing on a mountain of prior work that didn't have an obvious application but nonetheless was necessary to get to a breakthrough.
Honestly I can't believe I even need to state this. Its literally how science works..
Completely wrong.
Often a problem is important because we have no idea how to solve it, which means we need to develop new machinery to solve it. That new machinery can then be used to do things we couldn't do before. Stimulating the development of that machinery is the most important part of the problem.
You should remember that mathematicians and programmers working for a company have different jobs with very different goals.
"Knowing the answer" means more than knowing the value of a single yes/no bit.
Knowing the answer is valuable, but it's not everything. We also want to understand why. The four-color theorem has a computer proof, so we know it's true, but we learned much less from that than we do from a human-understandable proof.
Many of the OpenAI results can be made human-understandable, so it's better than the four-color theorem example.
If you were a math grad student, would you rather your advisor be telling you to LLM maxx your ideas or working with you and giving you ideas on your thesis personally? I know my answer.
To say that we must have people that know how to X, because otherwise no one will be able to teach how to X, is kind of circular reasoning. Why is X important to begin with? What’s its inherent value to society?
I could address your situation in several ways and it would be a long discussion that a quick soundbite wouldn’t really satisfy. But at the very least, your argument is one of the arguments I would use to question the privatization of pure math by LLM companies. Pure math has little value to society so AI companies should consider not getting involved when they aren’t really making life better for anyone and are possibly harming young academics.
But to give a brief answer to your question directly: In my view, and I think many if they really thought about it, the point of life is to overcome. If there is an AI that spits out the answer to questions that don’t directly impact human wellbeing instead of letting humans explore the answer at our own pace, we should question whether that AI is worth keeping around or at least relying on too much.
Obviously the question is more nuanced when it comes to math, because even pure math can have applications to understanding reality and improving human wellbeing.
So the question then becomes what do we do with the AI when it can both improve our wellbeing and remove meaning from our lives? Is a life without struggle worth a life without meaning? Personally I don’t think so, so I think we have to optimize all of the benefits of AI while essentially taking a harsh stance against all the harms. Currently the main harm is to education, and that’s why I think we should question and probably even disallow its use by students still working to demonstrate their skills.
This is very well motivated. It's a bit like letting a combine for knowledge loose on that which made life satisfying. Having the answer isn't necessarily going to move anything forward. I also think that OpenAI is making a strategic mistake here: they are competing with their customers. That's something businesses usually only do when they are ready to ditch those customers.
the assumption is that training llms is greater than training individuals. llms may lead to their own "breakthroughs" but so have many people without the use of llms. is the thinking that we'd be dependent on llms and so we are now forced to be only able to discover what llms find discoverable? sounds like a loss
what is your "inherent value" to society?
> If knowing the answer is not valuable then asking the question wasn’t valuable to begin with.
Let's say it's possible to know an alien civilization exists 100 million light years away from their waste heat. But that's all we know and it's unlikely we'll ever interact with them.
Was the question of whether we're alone still "valuable"? Wouldn't a lot of philosophy and even some physics fail to meet your definition of valuable?
> Imagine a programmer telling their boss it’s not about working software it’s about the journey of writing the code. Good luck getting hired.
Imagine the next version of Claude Code or Codex no longer hands you source code and instead turns a natural language prompt directly into a compiled binary that passes the test suite. Would this programmer's boss ship the binary straight to production and dispense with the company's git repo altogether? After all, the goal at the end of the day is shipped software that produces value, not human-readable source code.
Absolutely. Managing source code and builds is tremendous overhead that most companies would get rid of in a heartbeat.
I have a strong sense that there's many more CS problems falling than will be released quickly. CS starts to get tangible / actionable pretty quickly.
Seeing a physics result in there has got me hyped.
Turn this loose on LHC data ASAP.
[dead]
I know, it's awful. How do we get OpenAI to stop before they ruin math forever? /s
[flagged]