I feel like there's an implication here that distillation is a problem but I don't understand what you mean. I thought distillation was generating text from a model and then training another model on it. Is the something unethical in that? You're paying the API costs to generate the tokens, right?
Or I guess more to the point: is this something frontier labs have said is (or tried to paint at any rate) problematic? This feels like an "out of the loop" situation because I've only ever heard "distillation" with a positive connotation before.
Whether it's a 'problem' or not is viewpoint-dependent but it's against the OpenAI ToU:
> You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not:
> [...]
> * Use Output to develop models that compete with OpenAI.
Source: https://openai.com/policies/row-terms-of-use/
(I'm also curious whether they consider developing a competing model to be illegal, or harmful, or abusive...?)
> it's against the OpenAI ToU
Given that OpenAI doesn't care about training on copyrighted data, why is suddenly their ToU something anyone should care about?
That OpenAI was in the wrong when they ignored everyone copyright, does not make it right to ignore their ToU. If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient.
On a more risk-strategy level there is the size of their legal team, general endowment, and supplier and political connections to consider.
Everyone is free to ignore their ToU, but I can understand why a company would avoid it...
> If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient.
Yes that's what should be said to OpenAI. Now they should not cry about their T&Cs not being respected when they never cared about others' copyrights.
Feels like this should be some kind of anti-competitive violation even if it's not actually. Probably moot under this admin but still.
It's like saying you can't use windows to develop an OS, or drive a Ford on the way to your job at Hyundai.