In looking into what Anthropic has accused the Chinese AI companies of, "distillation" is not the crux of the matter: they accuse them of the use of massive numbers of breached computers and credit cards. Supposedly (though not explicitly demonstrated) distillation campaigns used "dark web" resources (like stolen and faked credit cards and p0wned IOT devices, routers, etc) that were used to create many many accounts for distillation. (Note: I am generally in favor of distillation - it is "normal" use as far as I can see.)
So there are these "resources" out there: massive botnets, stolen credit cards and credentials, etc. These often seem to be linked to or supported by state actors.
How can we hope that these "standing reserves" aren't used generally by AIs as if they are just another "skill" they know how to leverage?
Our infrastructures need serious hardening.
I imagine that Chinese AI labs would be more than happy to pay for tokens to train on. If Anthropic goes and bans them as quickly as it can identify to them, of course they'll shift to using lots of cheaply obtained sockpuppets to keep getting the same data.
I doubt that an AI could do anything with these that legions of scammers haven't already figured out how to do - after all, it's not like AI set up this distillation setup in the first place :)
Why do we believe all of this without a shred of actual proof. Breached computers and stolen cards is straight up crime, you'd think that there would be a lawsuit or two then.
The naivete of the tech industry, who never really had to deal with China, is mildly endearing.
Meanwhile manufacturing isn't even really alive anymore to tell the tale of how things work over there.
You mean the manufacturing that was outsourced because of 'shareholder value'?
Do tell if you know it.
Also, this is exactly what one would do if they wanted to resell cheap tokens in places where US AI companies don't do business, whether or not there is distillation as an added bonus revenue stream.
Why is that a talking point now, all of a sudden?
From my POV it looks like you guys tried "They're distilling us! Very immoral!", saw that and nobody cares, and now you're trying to run the more emotionally impactful "They're distilling us... with your grandma's stolen credit card! They're stealing from your grandma!!"
Hardening against what? Apologies if I am being dense here.
The way I see it, automated systems should basicially have access to the financial system anyways. This is some of the appeal of cryptocurrency.
Sybil resistance built into the traditional financial system (for example firewalling based on residential IP, requiring banking apps, etc.) is a massive annoyance that can obviously by circumvented at scale.
You think it is a lost cause to get credit cards, for example, less steal-able? Apparently there is a way to create fake cards that work for a few minutes, and stolen cards can be used. A lot of the current security is inelegant and lame. Could data-breech laws not light a fire under some company's asses?
I think the solution is a system like cryptocurrency to where the responsibility, liability, custody, and capability to defend oneself lies in the end user.
It doesn't even need to be decentralized: GNU Taler is one example that gets posted here a lot.
Any attempt to retrofit the old system of banking will only make it harder to replace by a system with better underlying assumptions.
I don't think older/busier people who regularly fall prey to scams even in the current system would be thrilled to assume more responsibility for their financial safety.
Hardening in this case could mean biometrics, or it could mean rethinking the trust model so that controlling an army of duplicates confers no advantage because you have to be trusted in order to do any damage anyhow. Containing multitudes buys you nothing.
I hope we chose the latter. Biometrics are creepy.
Color me naive but I can’t believe that approach works at any reasonable scale. Any meaningful token spend at distillation scale would be raising alarm bells at all the cc issuers.
No offense but you don't know what you're talking about.
Spitballing random pr pieces from companies about to IPO is not helpful to anyone, fyi.
> Spitballing random pr pieces from companies about to IPO is not helpful to anyone, fyi.
You can just use logic to come to the same conclusion. The Chinese labs have a fraction of compute and data. The simplest explanation is distillation.
DeepSeek R1 and DeepSeekMath were groundbreaking models which were substantially stronger than any models available for distillation at the time they were developed. The Chinese labs have advantages as well despite the lack of compute, too.
I'm anti-Anthropic and think their accusations are total PR FUD-packing. They don't focus much on the cc fraud because their PR firm landed on pushing the (tech-sounding) word "distillation" to try to scare people in the US and to try to make it a geopolitical issue where trade negotiators can put pressure. The Chinese companies see an advantage in being "open" right now, and it is working. They will not stay open once they see an advantage there. (Contrast with GNU.)
But I've slowly come to realize that AI+"dark resources"(scare quotes)=something bad! And this is imminent (maybe I'm naive and it is already ubiquitous).
We see here, with Kimi, that there's growing traction with the idea of eliminating types of training to avoid evil. I'm skeptical of that.
Have you trained models? If not then you can't really speak to how effective distillation is (answer: it is very effective)
Why don't we see more American companies doing distillation? If you could make an opus 5.5 cheaply by distillation in china, you can do so even easier in America where you have more gpus.
lol. Have you? What kind of distill you are talking about? It’s only effective in the surface, small scale model or output preference. Not useful for long reasoning chain.
Why lol? Yes. All major labs use SFT... (look at job postings)
> Not useful for long reasoning chain.
I have a feeling you don't actually know anything about ML