It doesn't do anything?
Au contraire! Now the creations of the LLMs stand on legal ground. This was an excellent deal for Anthropic
It doesn't do anything?
Au contraire! Now the creations of the LLMs stand on legal ground. This was an excellent deal for Anthropic
It is legal to train LLMs on books but illegal to train on output of LLMs.
Perfect - an absolute steal for 1.5B.
> but illegal to train on output of LLMs.
Since when?
They're probably confused with anthropic seething about "distillation attacks" coming from "fraud accounts". But that is not the law, that is just Anthropic being upset.
Typically the big LLM providers write in the their ToS that it is prohibited to use their output to train another LLM.
Whereas for a book it is fair use.
ToS are usually not worth the toilet paper they're printed on. They're not legally binding.
The legal ground is: If you're rich enough you can do it.
Now only big tech companies can train models
AFAIK this does not set a legal precedent as it has been settled and last summer finding is that Anthropic was wrong for "acquiring books illegally" not for training which is fair use.
With model distillation being so effective now nobody actually needs to pirate books to train their models. You can get an open-weight Chinese model and get all that. Or you can just buy the books or buy a library - there are many creative solutions here that aren't piracy and not going to cost you billions of dollars.
The moat right now seems to be the compute resources which might actually be worse for us common folk than a legal moat as we need compute for many more things that aren't LLMs too.
Settlements do not in any way establish legal precedent or any legal standing.
This is simply an agreement between two parties.
Yeah, it's always interesting the two-sides of a situation like this. Add regulation/enforcement to the big companies and you often shut out the smaller ones following.
Meta also has copyright lawsuits for the open models they released, so open models are not immune.
... unless the line we want to draw is "american orgs pay, others don't", as currently seems to be happening.
> Add regulation/enforcement to the big companies and you often shut out the smaller ones following.
That is the case, any regulation increases the cost to enter a market.
But in this case, its irrelevant because the moat of cost to enter is already unfathomable and secondly, they are not adding regulation but fining them for committing a crime.
So yeah, adding that every food compnay needs 3 health inspectors that they pay for would benefit coca cola over you mom and pop bakery. But telling someone they cannot start a Space agency with money laundered from ransom and drug sales payments would not affect much the competition markets
>the moat of cost to enter is already unfathomable
At the moment.
There are multiple ways to respond to that and I will try and summarise them.
Current believe is that its a "winner takes all market", so companies are acting rationally and using Brute Force compute to get there first. Training costs scale linearly, which means the moat is directly related to compute cost
There are theories that they are wasting 90% of training costs and there are more efficient ways to do it than throw compute at the problem. But if thats the case then chances are the market is not "winner takes all". Which then means the valuation of the ENTIRE market is overvalued.
Basically the only way for the assertion "at the moment" to be true is if the market is a bubble, else if the current theory of winner takes all market means a monopoly will make it so that cost isnt even the worst of the moats to enter.