> It's not obvious to me that's an unethical thing to do
In terms of work in mathematics, something I personally would not do based on ethical grounds would be to hear a rumor that some researchers are taking a certain approach and may be nearing a solution, use a model that was possibly contaminated with intimate knowledge about that approach (though later they investigated and think it wasn't), and then commit millions to tens of millions of dollars and untold amounts of hardware to try to beat them to it. If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take.
Even if you don't think it was unethical, it was never going to be received well in the community that was especially going to care about this work, and who are very much peers to many of the people working on this solution, so it was at the least an enormous (and well-deserved) own-goal that their unveiling of their solution to NS went like this.
But by OpenAI's telling they heard a rumor that the problem had already been solved. So they reached out to the other researchers as an attempt to share the credit, and in fact have at least one of them be the lead author (which is when they found out the AI had solved a broader problem than the researchers.) Seems pretty ethically palatable.
I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.
Hold on, you've just ignored the point of the post you're responding to. What isn't being received well is hearing that others are close to publishing on a solution to a problem, so quickly using your power imbalance (millions of USD and access to way better models) to front run this. Even if their model wasn't trained on the conversations, this is just a dick thing to do.
That's it, that's why it isn't being received well.
Even if you judge OpenAI solely on their public communications it still sounds really bad.
That they heard a rumour that a major open problem had been solved, so they decided to try and scoop the other mathematicians while they were writing up their preprint is extremely unsporting.
Then they decided to exclude an author because of his employer, even though he had used their own products to write the proof!
They haven't necessarily breached any formal ethical rules but their behaviour will lead to them and their products being shut out from the mathematical community.
> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.
Because the training data is millions of hours human efforts being distilled into a cascading hierarchy of enrichment by interested parties without providing attribution or compensation?
I'm sure that's part of the reason for many, yes.
I don't see the cascading hierarchy of enrichment.
I mean, they are certainly trying, but so far there's too much competition so the surplus mostly goes to customers.
Well, the investment dollars are spent on the customers for the most part, though also on salaries and equipment. But the lions share of the value is going to the shareholders (eg employees and investors)... and they have liquidated and will continue to liquidate a disproportionate value to what they have spent on us. By some estimations at least. It's very possible $1 into this machine to feed your queries is worth $10+ to a shareholder based on whatever new valuation they get. So I'd say there is a hierarchy of enrichment.
> without providing attribution or compensation?
many teachers also taught many students over the course of history, and very few would eventually pay any compensation or even attribute their financial (or career) outcomes to the teachers.
What made model training different?
Because the model is owned by a for profit corporation, ran and owned by total psychos and the (presumably) competent teacher is a friendly uncle?
Huh? In your example these many teachers were paid for teaching these students and were able to make a living off of teaching without the students compensating or attributing their financial (or career) outcomes to the teachers while now we have a system where we are expected to pay a monthly amount to a corporation that has inhaled all human knowledge without any financial compensation to the people who created, managed or maintained this knowledge. The effective difference being that our knowledge, which used to be a means of income, has now become a subscription cost.
as a developer that had a brief career in academia, i don't think your last comment is right at all. 99.9% of what i work on as a webdev, even if it's challenging and unique at the margins, is not really novel. concerns about job security aside, i don't really think of an agent as stealing my ideas because it's good at writing CRUD APIs.
collaborating with ChatGPT on a novel solution to an unsolved problem, getting 90% of the way there, and then being "scooped" by your AI collaborator (or rather by the company behind it) is a totally different situation. were i in the same situation as these researchers, it would be extremely hard to take OpenAPI's explanation + denial of plagiarism seriously
> concerns about job security aside
But that is exactly what I'm implying is the core reason, whether people realize it or not.
I totally agree that the vast majority of software dev is not novel. I have even made several comments to that effect. The same can be said for a lot of creative work as well. Yet many, many devs and creators are very unhappy with AI, and a lot of their complaints are variations on accusations of plagiarism.
And note, I am not saying it is wrong, it is completely understandable, but we need to be clear about where this turmoil is coming from.
If I were in the same situation as these researchers, I would publish all pertinent research work and chats so that the rest of the world can see how close the model's work is to my own. It's been scooped anyway, so there is no reason to keep it private.
Not everything is about money.
Maybe not money directly, but pretty sure it's about economic disruption. These models directly undercut the value of one's skills and labor, regardless of whether this value is measured in hard cash or abstract self-worth.
They can also greatly assist you.
I am working on two applications using ChatGPT and Claude. I have no illusions these people won't steal/copy whatever you want to call it, "train their models". Yes, I keep unticking the boxes that allow it, that they so kindly tick for me.
But what happened to these math researchers is something else and I am not sure it's about the money for them. You don't do math research to get rich, but to get acknowledged by your peers. Yes, we live in a capitalist world so obviously you need money to feed yourself. but for some people, that is secondary.
OpenAI stole their thunder, and that's just fucked up. It's not equivalent to cranking out a CRUD app for profit.
That's just damage control lol. That's the equivalent of a NDA. Get your name as lead author, get paid, and stay silent forever.
If you’re making decisions of ethical and material importance based on rumors, I’d be surprised if anything ethically palatable did happen.
> I suspect the main reason the community is not receiving it well is largely the same reason many developers are not receiving coding agents well.
No need to be mysterious. State what reasons you think these are in plain English?
Being displaced from a vocation that they either have dedicated their professional lives getting good at, or was their livelihood, or likely, both.
I think all other complaints from all other people in all their myriad variations stem from this core reason. Even if people don't realize it themselves.
Like, if these models had trained on the entirety of human knowledge and art, and then turned out to be absolutely useless, I would bet nobody would waste a second's thought on them.
I'm not sure that's true. Like if they produced nothing but the worst of the slop they're currently producing, a lot of people would still be bothered by that just because of the sheer volume of such slop that can now be produced.
To me it looks like an asshole on quest to take something from you while trying to frame themselves as generous. It is always infuriating.
> So they reached out to the other researchers as an attempt to share the credit
This isn’t at all what happened? What are you talking about?
That was in response to this part of OP's post:
> If I had done this, I also wouldn't have pestered the researchers on a Sunday night to meet immediately so we could negotiate a nice way of presenting the actions I had decided to take.
From what I can tell, both OpenAI and the researchers agree on this meeting happening, except both sides clearly have very different interpretations of what happened and why.
Hearing that something is solvable is already a hint. I don’t think leveraging this knowledge is ethical. They could go after a different problem but didn’t.
I heard they also tried to strong-arm them into removing the name of their collaborator who happened to work at a different company (Anthropic)...
I haven't looked into it myself, but if true, that seems incredibly scummy.
That's also incorrect.
My understanding is that they asked the independent researcher to improve OpenAI's AI generated proof and be the lead author of the paper to publish OpenAI's result.
This is the paper where they did not want the Anthropic employee collaborating. Not their work.
I think both Seb and Sam have said that it would’ve been simpler if the coauthor hadn’t worked at Anthropic so they’ve largely admitted they didn’t invite the collaborator as a coauthor because it would’ve look bad to have an Anthropic employee on the paper.
Looks like a lot of insecurity from OpenAI. At the top level researchers move around at their own will, you don’t decide who they work for
Yes. The key point being that this concerns a new paper about OpenAI's result rather than the paper Buckmaster and Alpöge were working on.
seems pretty short-sighted - "our models are so good that even our competitors use them for the most advanced tasks" is pretty powerful marketing
The flip side is that the Anthropic researcher is clearly pushing the case against Open AI and one might have reason to suspect their motivation and version of events for the same reasons.
What a mess.
> the Anthropic researcher is clearly pushing the case against Open AI
Things like the nytimes interview are with Buckmaster, who works at NYU, not Alpöge. I saw a couple of tweets from him over the last week. Any chance of clarifying what makes you think he's "clearly pushing the case"?
> The flip side is that the Anthropic researcher is clearly pushing the case against Open AI and one might have reason to suspect their motivation and version of events for the same reasons.
I haven't seen any evidence of this. Much of the anger is coming from the unaffiliated researcher. levent (the anthropic employee) has mostly constrained his comments to basically "I would have been happy to collaborate w/ folks from OAI"
I think people are too reserved in their unwillingness to operationalize ambiguity. Ambiguity is constantly being thrown in our face, with internal audits and other laughable attestations of virtue that amount to a pantomime of transparency / good faith.
Why should I care if a company claims they find no evidence of wrongdoing? Is that the threshold for privacy/trust? “We don’t care if it appears that we’ve been dishonest unless there’s hard proof.” They can simply design proof keeping to terminate at the places their dishonesty is implemented.
For me, when there is a clear motive to be dishonest, a corporation should be assumed to be dishonest unless there are robust transparency measures and a regulatory environment shown to be providing a cost to dishonesty. Without it, all you do is burden yourself while the powerful entity moves ahead with its selective dishonesty and the rewards there reaped.
What's that saying about wrestling with pigs?