They always all gather around the same spot then that spot moves every 6-9 months. I think the clustering is more likely evidence of distillation. I don't personally think distillation is a bad thing. If the LLM providers can distill all of human output into their models for 'free'. I don't think distilling a model from the output of those models is morally wrong.
I think it's more likely to be the effect of synchronization of launches, and the fact that models that do not challenge SOTA in some way do not get launched (think Gemini Pro delays), launched quietly or do not get any attention.
If you talk to the Chinese models, even super smart Qwen 3.8, you can tell they are distilled just from the verbal ticks they have. Gemini, ChatGPT and Claude do not sound alike. The Chinese models 100% sound like one of the 3, usually Claude. American models are load bearing for this LLM generation seam.
- load bearing -
seams, boundaries, envelopes, etc
That’s definitely my impression of deepseek 0731 after a fair bit of use via ds4, it sounds like Claude.
I gathered that the most recent advances haven’t been in capabilities of the model but more the way that it’s able to be employed (most recently agents).
> LLM providers can distill all of human output into their models for 'free'
Not sure what part of being charged guilty and paying a fine you see as "free".
What an interesting take. One question, do you think stealing from a thief is morally okay? I'm just asking no judgement on my side.
I'd say it's more "Downloading LimeWire Pro from LimeWire" than actual theft.
Why isn’t it more like building a hardware store using lumber you purchased from a competing hardware store? Or founding a school using an education you obtained at a different school?
Because that doesn't satisfy the narrative of American exceptionalism. It's easier to point at something and say it was stolen or copied than it is to compete, especially with the political climate in the US.
This isn't an anti-American sentiment. It is an anti-corporate/regulatory capture/embrace and extinguish sentiment (which probably reads the same to many people these days).
If the legal system declares the first thief’s theft not theft then all bets are off.
> If the legal system declares the first thief’s theft not theft
But they didn't find it. The Big LLM provider accepted guilt and paid a fine.
You can argue whether it was a fair amount they paid, but there is no legal precedent that was set. It's still considered theft.
As i understand it, they accepted guilt for downloading stuff illegally. They didn’t accept guilt for incorporating all of human output into their model without consent.
Copyright law only considers illegal ownership of a work, so the crime - or tort - was making/acquiring copies without permission or payment.
Training from copies has been ruled fair use because it's "transformative" and not simply "derivative."
This is obviously debatable, but that's where the debate is at the moment.
So basically because they just browsed and used the information that was mostly public on the internet and they didnt copy it, they just learned from it and thats fine. Which makes sense. None of the llms let u copy someones work exactly anyways... makes total sense honestly. So in this case what happens to distilling? Is that also learning or ur trying to get to their actual weights by kind of reverse engineering it? Where would the argument fall there?
> but that's where the debate is at the moment.
Because of the rulings of a couple of judges. Is that actually what the majority of people think?
> Copyright law only considers illegal ownership of a work
That's definitely not true. File sharing, for example, is illegal even if you legally own the original copy you're sharing.
Similarly, copyright has something to say if I read a legal copy of harry potter and then create a new work in that world.
> Because of the rulings of a couple of judges. Is that actually what the majority of people think?
There's a good reason for the law not to be based on what the majority thinks.
> They didn’t accept guilt for incorporating all of human output into their model without consent.
Because that use case is actually permitted by law.
I mean... that's one interpretation of the law, sure.
The law was written before the idea of an LLM existed, and some judges in some specific cases decided the previous law covered this usage.
So, it comes down to if you believe a couple judges ruling on a couple cases is the right way to determine a world-altering new legal framework.
> So, it comes down to if you believe a couple judges ruling on a couple cases is the right way to determine a world-altering new legal framework.
It's not an either-or.
> But they didn't find it. The Big LLM provider accepted guilt and paid a fine.
That's not how it works. You have to give it back.
Otherwise, the distiller can just pay a fine (no larger than the original did) and be okay then, right ?
[dead]
True. It's the courts that failed humanity. Or perhaps the shits that invented copyright to start
Is it theft if another thief steal's the first thief's theft?
question's phrasing made your judgement obvious