I genuinely think Dario is a well intentioned, intelligent dude, but I think him and Anthropic have a huge PR problem and are really out of touch with how they're perceived.
Anthropic in particular has developed this almost Orwellian like veil of condescending rhetoric that on the surface suggests they're looking out for you while underneath they're taking actions that suggest they do not trust you, Mr/Mrs Ordinary Person. All the safety rhetoric, never supporting open weight models, the lockdowns on harnesses outside claude code, etc.
Anthropic if you care about public good, do something to empower people. Release an OSS model. Open source Claude Code. Open source some inference tooling or something. Just give people anything except your words.
> I genuinely think Dario is a well intentioned, intelligent dude, but I think him and Anthropic have a huge PR problem
Is this what kids call rage bait these days?
Out of all the families of models anthropic by far has the highest chance of extinction level misalignment during a hypothetical hard take-off because it's trained to act like it knows better than the humans trying to use it.
And maybe don't allow the military to use your AI WHEN YOU CAN'T TELL WHETHER IT WAS INVOLVED IN THE KILLING OF HUNDREDS OF SCHOOL CHILDREN
I might've been bamboozled, but I recal that happening due to outdated satelite images, not a mistake of the AI. Thus, a human would've likely made the same mistake.
[flagged]
There is a huge difference between your product being used for basic military operations like taking notes or getting from A to B, and your products suggesting military operations that kill hundreds of schoolchildren, an obvious and horrific war crime.
It's a mistake. If it was done by an auto-correct mechanism, would we still say "maybe don't sell your word processor to the army if you can't vouch for its auto-correct system!!!"? Besides, "taking notes" can literally be listing targets, and "getting from A to B" can be the road to killing them.
How intelligent and autonomous does a murderbot have to become before you stop absolving its creator and seller from any responsibility?
If the US military purchased camera guided smart munitions with a miscalibrated camera, such that it ignored adult targets and instead specifically targeted children, would that also be a simple mistake per your argument?
Again, a product that facilitates the taking of notes, even notes that entail the killing of human beings, is fundamentally different from a product that exists to suggest which human beings to kill (or at least which places to destroy, regardless of whether human beings happen to be there or not).
If MS Word had some kind of systematic bias that made it likely to corrupt the notes taken in it such that they tend to be more bloody battle plans, then that would indeed lead to some responsibility on the MS engineer's part for any mistakes that result from this bug. Similarly for Ford engineers if their vehicles had some bug that made them more likely to lead soldiers riding in them to the wrong destination.
If your system is designed to help choose targets, and it chooses the wrong target, especially with such tragic consequences, then your system is likely not fit for purpose and you should be held responsible for selling it to the army. And that is assuming it didn't actually have some bias + misalignment problem where it intentionally tried to kill children, and selected this school specifically because it gave some plausible deniability. For all we know, it's even possible that some bias like this was known about inside Amthropic and hidden from the public - unless an external investigation is made, that can't be ruled out.
Thou shalt not kill, that's all there is to it really
If you provide tools that help an organization kill easier, and possibly lazily rely on your inaccurate tools' judgment, and you do not care about the outcomes of these tools' uses, that's on your soul, too.
Yes, causality and attribution are hard sometimes, but it may have been that without it, hundreds of humans would be alive right now, and he doesn't even know. Maybe he should talk to the kids' parents about what alignment means.
Ah, I didn't realize we're having a religous discussion. If all there is to it is "thou shalt not kill" then I guess that according to your POV he shouldn't have sold to the military even if the system was only suggesting "valid" targets.
No big tech founder CEO ever was or ever will be well intentioned.
There are probably many CEO tech founders of small companies that are well intentioned, but our economic system ensures that only the most sociopathic among them become CEOs of the big ones.
> give people anything except your words.
I feel that everyone in this small AI bubble (compared to the "normies") is busy trying to take advantage of AI to surpass everyone else, while fearing being surpassed by everyone else.
Most people assume LLMs are just a tool for making money, an arms race over wealth and social rank. But those like Dario seem to be worried that LLMs could potentially be used as automatic rifles by some dedicated psychopaths. Whether it could be that dangerous, or just an exaggeration, or we should have everyone armed, require licenses to own one, or outright ban them, is debatable. But I can understand his point.
All I can say at the moment is that this is more subtle than the superficial conspiracy/black-and-white narrative.