One thing about human mathematicians is that they only publish positive results. Professors etc might have file drawers full of "negative results", but the incentives and bandwidth of human mathematicians makes publishing these useful results impossible.
But AI agents have no such limitations and can publish and re-use negative traces easily. There have been some recent projects (https://www.theoremdb.org) aimed at exploiting this fact. https://news.ycombinator.com/item?id=49227505
In general though, LLMs do not have the same limitations and incentives as human mathematicians, and the next year's tsunami of change will make this abundantly. clear.
A great scientist once said, that the scientific method also includes stating, why what one "found out" could be wrong, and that one might need to retry experiments, whose results one relies upon. I think it was in "Cargo Cult Science" by Feynman.
In a way what they are doing, only publishing positive results is no longer good science.
The bandwidth is absolutely there ("we tried this and it didn't work" is totally the stuff of conference discussions).
The incentives are not.
The incentives are skewed towards "a magician never reveals her secrets". The results are presented as if a rabbit got pulled out of a hat, with a maximum ta-da! effect, and little backstory of how the hell did we get there.
Don't get me wrong, these things are discussed, often over beers (you better drink it you want to make a career in the field).
But not published.
The younger mathematicians are trying to change that with the blogging culture. But the professional incentives aren't there. (In corp-speak: can't put blogging on perf). They burn out.
That's why math blogs usually come from either the top dogs in the field, like Terrence Tao, who don't need to care about perf, or people outside academia.
That's one thing that I hope the disruptive/destructive effects of LLMs will force mathematicians to face.
As one of my fellow mathematicians sarcastically wrote¹, we've reached a point where we should become a cult because we're acting like one anyway.
The other possibility is, of course, that the shake-up will take us precisely into that direction.
My point here is that the real problem here is not mathematical; it's a social one: incentives and politics, organizational structures, policies, allocation of jobs and funding.
All of this directly impacts how we do mathematics, who we do it with and teach it to, how we teach and communicate, and, of course, what math we even do and look at.
Given that, I'm neither too worried about humans vs. AI standoff, nor hyped about the Glorious New Future full of AI-assisted discoveries.
AI or not, the organizational issues in the field are still there, as are the incentive structures (including the infamous publish-or-perish).
We are doomed, yes, but by our own hands and committees. And it's up to us, not the AI, to get us out of there.
The little shove from the AI might be just the thing we need.
____
¹ https://www.mcsweeneys.net/articles/an-open-letter-to-the-ma...
This is not specific to math. To non-academics, you basically have to rewrite your commit history to make it seem more impressive. You often write the motivation section last. At least math publication culture allows saying "We consider the problem of" and then solve it. In AI/ML academic papers you need much more "story" around why, what the applications are, why aren't you doing something else, defend against lack-of-novelty attacks, defend against "this is just A + B known techniques used together" etc.
Yeah, publishing in ML can be annoying because of that. What's wrong with "I thought this problem was interesting"?
Because your institution's PR department and the journal you're publishing in want the publicity of "scientist cures cancer". And you probably wrote "I'm curing cancer" in your grant application, team charter, or whatnot.
I think its a total travesty of our society that people are desired to identify both the problem and the solution
people without solutions are told not to point out the problem
and people with the skillset to bring solutions dont have the skillset to search for problems or apply their seemingly unrelated disciplines to it
so you get a lot of capital and energy thrown at people who purport to be both
If AI can address this too, that would be profoundly impactful. The cross disciplinary work that a single college course is supposed to reveal, finally realized
Yeah, LLMs are great at generating negative results for math-related prompts "We scanned values {a,b,c} from 0-100 and no results" Great..too bad journals will not publish this. But good job, I guess. A negative result is only truly useful if it can be bounded, requiring an actual proof.
> A negative result is only truly useful if it can be bounded, requiring an actual proof.
That's at least true for current journals, since they're supposed to be read by actual humans. I suppose one could imagine a sort of "AI" pure data journal that just "publishes" (in actuality aggregates) any sort of partial result. This body of knowledge would be entirely useless to humans, but could serve as a sort of "computation cache" for these stochastic systems.
> would be entirely useless to humans
why useless? humans can navigate these unsuccessful chains of thoughts to build on top of them or reject completely
What is next years tsunami of change? Asics?
They are always out of stock, and you can never find your size..