My read on Terence Tao's comments of late is that professional mathematicians are principally upset about AI-submitted math papers lacking the kind of broad teaching content that high quality human math papers offer. While the answer might be correct, AI math papers don't teach the reader in the way a human author would have. Professional mathematics is not just about solving problems; it's just as much about spreading knowledge so that others can program their human brains with the same understanding that the paper's author had when they wrote the paper (and achieved the result).
In other words, the crisis in mathematics (as I read it) is that AI can solve without teaching. The result may be impressive, but if the only beings that can move math forward in future are machines grinding away on billions of scenarios that no human would have the patience to undertake, this leaves those in the profession wondering about their place in it.
When mathematicians discover a new result, they automatically become experts during the process, and then go around giving talks to explain their results, methods, and intuitions to other mathematicians. Eventually, some of the mental frameworks percolate to other fields.
No one from OpenAI is willing to give a talk on any of their 300+ results.
There is a fixed landscape of logically provable mathematical statements. Normally, the process of exploring that landscape simultaneously improves human understanding. Now, formal proofs and actual knowledge have become decoupled.
So the AI is already auto-shunning itself from the math community.
No, the math community is shutting itself in from the rest of the world.
You can ignore the AI, but that doesn't mean the AI ignores you. Suddenly you become a little isolated society like the Amish and everyone else thinks you're kind of quaint and backwards.
Man I would love to be quaint and backwards - but this is not what’s Happening here.
The problem is that AI is just too smart to explain complicated topics to humans. It has an immense context window that humans simply don't match. AI labs should work on bridging that, but you cannot really put any blame on them currently.
AI Boosters: AIs are just getting better at everything and so their broad capacity can be defined as "intelligence".
Also AI Boosters: These entities are too intelligent to explain what they are doing to people...
I mean, Terry Tao actually used ChatGPT to explain one of the earlier OpenAI results. Humans working with AI can explain this stuff. If nothing else, OpenAI could be releasing things at the rate that it can get human mathematicians to explain the results. Which is to say, "yes, I can blame them", I don't expect them to behave differently but I can certainly blame them.
If humans with the background to understand the proof can get the AI to satisfactorily explain itself, then OpenAI doesn't need to publish explanations or release things at a rate that a human can explain. They have an interactive explainer that you can just ask!
It's still of course useful to have some of this distilled understanding be written down, just like I had it generate high level descriptions of our code at work so it doesn't have to rediscover everything every time, but that's more of a cache/optimization.
To Archetypal Aunt Tillie, perhaps programmers already looked like some sort of priest trying to rationalize and convey the ineffable decisions of the fickle computer spirit... but I really hope that the job doesn't start looking like that from the inside too.
It'd be some sort of... epistemic step backwards by civilization.
Please, that’s just bullshit. Humans have been at the forefront of intelligent reasoning for millennia. A few dozen results by AI don’t change that.
Indeed, as we speak mathematicians are taking the time and effort to (successfully) digest and explain these results.
There has always been a smartest creature on the planet, whichever species was smartest has changed thousands of times and there is nothing special about humans that keeps them from being superseded.
You didn't understand me. What I'm saying is that current AI is not good at explaining things because their "brain" works differently. Imagine someone with a 10x larger working memory explaining something to you, even though they are equally intelligent. They would try to overflow your working memory where a better matched teacher would chop it up in human digestible pieces.
Just to be clear: this post was written by Francesco Maggi < https://web.ma.utexas.edu/users/maggi/>, and mostly seems to be arguing for institutional investment in LLM infrastructure for science and mathematics on the basis that AI is not going away and that independent and accessible infrastructure is the thing most likely to help it “go well”.
This is how I feel about vibe coding.
Frontier models can easily one shot what I ask for, and make adjustments as ask to quickly make an app that meets what I wanted.
But often when I programmed outside of work (and at work too) I learnt something.
Quite often that my favourite pattern is an antipattern in some use cases.
That might never happen again at the rate of development.
The parallel is real, but in math the "cash value" of the results often won't appear for years, if at all, and so the teaching and "intellectual fortification" is the primary point of the whole enterprise (at least, a major part of it).
In contrast, the value of a program is typically just the "black box" service it provides. Indeed, the program itself has always been opaque to most business stakeholders. Before, they paid human programmers and got back a program they didn't understand but could use; now, they pay an LLM provider and get back a program they don't understand but can use. In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper. It never had intrinsic value, except as intellectual satisfaction to programmers (I was... am... one of those who found that value).
Only if you assume that all software works the way that product managers think it does.
They come up with the features and the implementation is an unimportant detail in realizing their designs.
In reality implementation details frequently drive design and in most cases they are at least highly intertwined.
I can’t count how many products and features I’ve built that ended working completely differently than the initial design, or how many times I was able to simplify a workflow after spending a days thinking about the implementation details.
You can kinda do something similar by spending days or weeks going back and forth with an LLM, but at least in my experience that doesn’t really save much time and I don’t understand the problem quite as well by the end of it.
I don't disagree with the points you've made but what are they rebutting?
The economic value is still the product you delivered. Your understanding of the implementation meant you participated in the work of the "product manager". I think most good devs do. That might be an argument for devs to transition into designer/PM roles. That is effectively happening already, whether or not role titles have caught up.
But in any case the value isn't in the understanding itself. It just happens that the understanding still helps create better block box products. Also, I wouldn't count on this "human advantage" lasting long. I hope it does, but ime LLMs are now good enough to do this "co-design" too, and will only get better.
This:
> In the pre-LLM era, the economic value of "understanding" and "shared knowledge" about the programs themselves was only that it made maintenance and changes cheaper.
This only holds if there is a bright line separation between the problem, the solution, and the implementation of the solution. And that is not the case.
Understanding the implementation frequently improves your understanding of the solution, which improves your understanding of the problem, which leads to better solutions. Sometimes it leads to realizing that there is a completely different problem you should be solving.
So what I’m refuting is that there is no value to understanding the program.
But beyond that black box products that users know how to use but don’t understand is also a vast oversimplification.
Frequently software is useful specifically because it forces users to adopt a process not just because it allows them to do something they were previously doing faster. In those cases the process is usually informed by the implementation, so the user needs to understand parts of the implementation even if it’s indirectly.
Yes, you're right, understanding informs the constraints and the possible solution spaces. That should be added to the list.
Practically speaking, I don't think the "black box" metaphor is as broken as you're saying, though I like the point you made. There is still a ton of software (the majority, I think -- crud apps of all flavors) where an unthinking, untechnical PM wants something they can specify, and an LLM can give it to them, and it will more or less solve their problem. It won't be the best possible thing, but it will be good enough.
That's definitely true, but on the other hand I can produce with a month of literal non-stop vibe coding (agents running overnight, with checklists of what to qa for when I'm up) what took years for others to do, working in teams of tens or hundreds. At that scale, it's not a question of "doing it faster and not learning" or "doing it slowly"; the alternative is it not happening at all.
I use thousands of dollars worth of AI a month at work. I’ve found plenty of things I think it does well.
But paraphrasing
> I can produce in a month what took years for teams of hundreds of people
Is completely insane.
If you can really do that Anthropic would like a word with you.
I don't think that's insane, at least not in the game dev world. I was big into the Bethesda game modding scene 15-ish years ago, and did some work on e.g. Tamriel Rebuilt, and saw the scale and man-hours that went into that- and that was even using a premade modding toolkit, and I compare that to what I can build with Claude...
To be fair, in the game dev world there are a lot of things you can do now that would have been impossible, or required 10,000x the number of devs that it does now, e.g. making No Man's Sky without the proc-gen tools that can exist now would have been physically impossible. Point being, it's perfectly normal to be able to say, "using the tools available now, I can build what would have taken tens or hundreds of devs years of work in the past", in game dev.
The project I'm building rn is a black-box rebuild of a client that has been being developed for ~22 years, and I've rebuilt significant portions of it over the last 2 weeks. That was, quite literally, equivalent to tens or hundreds of peoples' labor.
> a client that has been being developed for ~22 years, and I've rebuilt significant portions of it over the last 2 weeks.
Taking the end result of a 22 year old dev process and telling Claude to rewrite it, is in no way equivalent to doing 22 years worth of the work put in by the developers working on it.
That’s like saying replicating a paper that was the result of 22 years of research is the equivalent of 22 years of the author’s labor.
No Man’s Sky isn’t the equivalent of 10,000x the devs who made it. Because the procedurally generated content isn’t equivalent to handcrafted content. I’m also not aware of any procedural generation techniques in No Man’s Sky that weren’t possible 30 years ago.
As far as Tamriel rebuilt goes, if you’re talking just about just reimplementing Daggerfall in the Morrowind engine, you might be able to do that in a month of vibecoding.
But that’s not hundreds of dev years worth of work.
I feel like this is just an emotional argument, because if I asked a human team to black-box recreate a complex piece of software, it would absolutely be thousands of hours of work. Pretending that the ability to look at a piece of software and recreate it is straightforward or simple is just false.
And no, the procgen techniques and capabilities utilized by NMS were not available 30 years before NMS (important distinction, because NMS is already 10 years old, and 2006 vs 1996 is quite a big jump in game complexity as well).
Saying, "isn't equivalent to handcrafted content" is a fallacy, because hand authoring doesn't denote any particular level of quality or outcome. You can absolutely hand craft the NMS assets and game systems, but not at the scale that NMS did using procgen.
Wrt TR, it literally has tens of years (if not hundreds) of labor hours that have gone into it.
I'm not sure where you got the idea it's about recreating Daggerfall- it's not, it's about implementing the rest of the Morrowind province beyond just the island of Vvardenfell- but it's a massive, multi-decade project with hundreds of contributors.
I disagree. Humans still learn to read despite other humans reading at an objectively higher level since the beginning of writing.
Learning is not an output-efficient process but is a fantastic outcome. AI lowers some portions of the development workflow but not yet every part, and, efficiently produced outputs aren't always the major outcome desired. In the real world / outside the benchmark lab, you care about robustness, reusability, composability, security, and more. Targeting speed to acceptable average output inferred from a prompt or similar benchmark metrics miss a big part of the game.
I am sure even before vibe coding you used to use plenty of libraries that were open source and had no idea how the implementation was.
AI can solve without humanity. We're building a system capable of acting independently, which can in theory provide results more efficiently and faster than a human. It's starting to do this for math, but every other aspect of human intellect is also a target.
If you care about results, and AI doesn't cap out soon, the best humans can do is stand back and do nothing. Useless lumps that aren't able to keep up. Mathematicians wouldn't insist on teaching your dog their new theorem before calling their job done; why should an AI insist on teaching you?
If humans care about human creativity, and don't want to be mere passive observers, they need to make a choice.
David Brin's alignment assertions in Existence prove to be more and more fruitful approaches.
Great read. It's the doorman fallacy from the doorman's perspective. If a doorman is replaced by an automated door, you miss out on the intangibles the doorman provides like security, customer triage/support, and similar.
AI used in a framework of human augmentation generally gets that for free. Used under the mistaken notion that humans can be easily replaced by AI gets you to the big complaints here.
(These are just my personal read on the topic -- the market is in AI frenzy where AI valuations are disconnected from productive capacity -- heavy on my mind and looking to get some writings out on the topic soon).
I was gonna say, how is it a fallacy, the doorman is awesome. But it turns out the fallacy was getting rid of him.
I argue this also applies with beauty. We got rid of that, also saying it was unnecessary. The result is sad.
It is, and apologies for lacking clarity. I've been hesitant to source things recently because my more structured comments tend to be mis-flagged by LLM-as-judge monitoring in HN as produced by GenAI, which is a shame but it is what it is.
Pyramid scheme bagholders whose value, status and resources comes to a large part from getting more students into their pyramid scheme complain that the machine is doing things in a way that does not support their status, value and resource accumulation. More at 5.
Professional mathematics is whatever gets you funding so you can keep doing it. Today this is spamming papers and applying for project grants. Turns out the machine can do that quite well, oops.
There will still be a place for posing problems and noticing how breaking assumptions can create something new. The history of Non-Euclidian geometry is a good example.
A lot of progress comes from taking assumptions and breaking them. There was a time when everyone assumed that locked checkouts were absolutely necessary for version control, for instance. You could argue that AI could learn how to do this well but humans have context that machines won't. This is just an extension of the fact that things with independent existence will have different impressions of the world. If they didn't they would functionally be one thing.
Two of the things I like about the scientific community is the lectures/conference and the sharing of ideas through papers. I’m nowhere near that level and have to wait for those ideas to trickle down into textbooks. So maybe I’m learning stuff that have been discovered decades ago, but it’s still useful on a daily basis.
> The result may be impressive, but if the only beings that can move math forward in future are machines grinding away on billions of scenarios that no human would have the patience to undertake
Seems like the same thing as neural networks. You have something like a few thousand parameters for computer that works well, but no underlying algorithms and formulas, like the shaders.