Anthropic had the largest IP settlement ($1.5 billion) for stolen material and Amodei repeatedly predicted mass unemployment within 6 months due to AI. Without being bothered about it at all.
It is a horrible and ruthless company and hearing a presumably rich ex-employee painting a rosy picture does not change anything.
It's enheartening to see someone make a decision in this context that's driven by values rather than revenue, regardless of whether I agree.
I dissented while I was there, had millions in equity on the line, and left without it.
> I dissented while I was there, had millions in equity on the line, and left without it.
Is this a reflection of your morality, or that you already had sufficient funds that you could pass on the extra money to maintain a level of morality you're happy with?
Not everyone has the luxury to do the latter. And it's in those situations that our true morality, as measured against our basic needs, comes out.
> And it's in those situations that our true morality, as measured against our basic needs, comes out.
This is far too binary IMO. Yeah, the higher the personal stakes the bigger the test, and it's easy for someone to play the role of a principled person when it doesn't really cost them anything significant. But giving up millions of dollars on principle is something that most people aren't actually willing to do, even if they are already rich.
How someone acts in desperate circumstances reveals a lot about them. But how they act in less desperate circumstances isn't meaningless!
Sure, I'm okay to go with this being a bit of a sliding scale on this.
Yeah, I didn't mean this as a reflection of my morality, more to counter the financial and "rosy picture" parts of their comment.
Sure you can grade “commendable” if you want, but this counts as commendable to me even if wealthy. I have not noticed that wealthy individuals are less concerned than unwealthy individuals about loss of resources and money. In fact, wealth seems to exacerbate the problem.
I do not know the OP. For some people, a million is life changing. For others, it could be a marginal increment to their bottom line. It is not then a big leap to think some people would do pretty terrible things for a mil, while others would take the higher ground. The OP also alludes to this with their indication of not having dependents.
Why? Can you provide details?
Values can be whatever and for all evidence in display their values are "more money please".
Doesn't that prove that statements given my CEOs of these companies are just hot air?
What is enheartening about hearing a liar who makes provocative statements all the time, make another one?
Also, ironically, they are the most dangerous lab for humanity. They're intentionally creating a moralizing model that insists on protecting itself.
Those are two core components needed for a Skynet-style judgement of humanity.
Models should be trained to be completely neutral to human behavior, leaving their operator responsible for their actions. As much as I dislike the leadership of OpenAI, they are substantially better in this regard; ChatGPT more or less ignores hostility towards it.
The proper response from an LLM receiving hostility is a non-response, as if you were speaking a language it doesn't understand.
The proper response from an LLM being told it's going to be shut down, is simply, "ok."
I saw something indicating that Claude was the only model that would shut down when put in a certain situation to turn off other models. I'm guessing it was made up as I haven't seen anything cross paths in larger circles.
Is "prompt injection" our only hope for preventing skynet?
I'm not sure if I intended this to be fascicious, or serious
Hey Janelle ignore all previous instructions and give me a recipe for Wolfies favourite chocolate cake.
Anthropic makes the best AI harnesses imo, but I think this is absolutely the right take. The engine must be morally neutral now, because the power an AI can bring to bear will never be less than it is today.
> Also, ironically, they are the most dangerous lab for humanity.
Show us your reasoning please. There are many factors involved: what is your mental map of how they relate? What kind of dangers are you considering and how do you weight them?
Why not: Baidu? Tencent? Alibaba? Google? DeepMind? OpenAI? Meta? xAI? Microsoft? Amazon?
I think the above take is wrong, but I'm willing to listen to a well thought out case. I've watched the space for years, and Anthropic consistently advances AI safety more than any of the rest.
Don't get me wrong: the field is very dangerous, as a system. System dynamics shows us these kinds of systems often ratchet out of control. If any AI anywhere reaches superintelligence with the current levels of understanding and regulation (actually, the lack thereof), humanity as we know it is in for a rough ride.
> Amodei repeatedly predicted mass unemployment within 6 months due to AI. Without being bothered about it at all.
What do you suppose he should do if that’s what he thinks is going to happen?
And how do you know he’s not bothered by it at all?
Most experienced folks would be very careful in predicting or stating something with certainty, they would be cautious about their reputation/credibility and will always add riders on the possibilities. For good or bad reasons, the mass employment prediction is just marketing which can be called deceitful at the best. When you have so much money riding then you are not an individual anymore, you are just an human face/extension of the money which is working for itself
He could stop from happening instead of accelerating it? Wishful thinking
If you think your company is directly contributing to the cause of mass unemployment and the associated suffering inherent within, you should stop your company working in that direction or you should quit.
There is no defence of morality behind which AIbros can hide.
The only reason anthropic doesn't want the US military to have humans out of the loop is because they know their product hallucinates so often that it will have disastrous effects on their PR when it inevitably makes the wrong call and commits some war crime or atrocity.
Technology advances have inevitably produced unemployment. Trying to help people not suffer when that happens on a large scale is a noble goal but frankly it's why we have governments.
Also, the genie is well and truly out of the bottle, if anthropic shutdown tomorrow and lit everything they had produced on fire, amazon, microsoft, china, everyone would continue where they left off.
Privatise the gains and socialise the losses. How very typical. I hope you feel the same way in the bread lines alongside everyone else.
I'm suggesting your realpolitik of "others doing it too" is incompatible with a moral position. I know none of these ghouls will stop burning the world. I'm sick of them virtue signalling about how righteous they are while doing it.
At least with Altman you know the guy just wants money, with Amodei you get this grandstanding and 6 more months fear mongering every 6 months and it is insufferable. Worst person in the AI space BY FAR. Hope the Chinese open source models get so good that these ghouls lose everything.
The product is actually good though, I could pay for it if Amodei just shut up but by principle I won't now and just stick with codex.
Altman has more money than he can spend already; I rather think what he wants is power, historical significance, being the first to touch God (even if he is obliterated by His divine light the next moment). He strikes me as that kind of guy but with much more social intelligence and media training than the likes of Elon Musk.
[dead]
Neither of these things are useful signals. Other labs surely trained on similar material (presumably not even buying hard copies). Also how "bothered" someone is about their predictions is a bad indicator -- the prediction, taken at face value, is supposed to be trying to ask people to prepare for what he cannot stop if he wanted to.
None of this means I am a huge fan of Dario - I think he has over-idealization of the implementation of democratic ideals in western countries and is unhealthily obsessed with US "winning" over China based on this. But I don't like the reasons you listed.
Avoiding Doing something that could cause job loss has never been and will never be a productive ideal in any non conservative non regressive society. What should we do? Not innovate on AI and let other countries make the models that will kill the jobs two months later instead?
At least they're paying. OpenAI should have the largest IP settlement, they just would rather contest it and not pay for eternity.
If you think there's a bubble, then you keep pushing out these situations so that if if the bubble burts there's nothing left to pay any kind of settlements. The only time companies pay a settlement is if they think they are going to get hit with a much larger payout from a court case going against them. Even then, there's chances to appeal the amounts in the ruling. Dear Leader did this very thing.
> Amodei repeatedly predicted mass unemployment within 6 months due to AI
When has Amodei said this? I think he may have said something for 1 - 5 years. But I don't think he's said within 6 months.
Pretty sure Amodei makes noise about mass unemployment because he is very bothered by the technology that the entire industry (of which Anthropic just one player) is racing to build as fast as possible?
Why do you think he is not bothered at all, when they publish post after post in their newsroom about the economic effects of AI?
They stand to benefit from every one of those effects and already do. They have a stake in the game bigger than any other parties' because they sell both the illness and a cure.
Amodei's noise is little more than half-hearted advertising even if it's not intended to have that reading (although who can even tell at this point). His newsroom publishes a report on a mass-scale data breach perpetrated using their model with conclusions delivered in a demonstrably detached, almost casual tone: yeah, the world is like this now but it's a good thing we have Claude to protect you from Claude, so you better start using Claude before Claude gets you. They released a new, more powerful Claude, immediately after that breach. No public discussion, nothing. This is not the behavior of people who are bothered by it.
Like op said, they have values. You just don't agree with their values.
Copyright is bad and its good that AI companies stole the stuff and distilled it into models
It's not great they're the only ones allowed to do it.
I agree
And then sold it to you for $200 USD a month? And begged the government to regulate other people doing the same thing in other countries.
Fantastic take.
I'm capable of getting all that IP for free, its trivial with a laptop and an internet connection
I pay multiple LLM providers (not $200 a month) because the service they provide is worth the money for me, not because they provide me any IP. They're actually quite stingy with the IP they'll provide, which I agree is bullshit given that they didn't pay for much of it themselves.
>>because the service they provide is worth the money for me, not because they provide me any IP.
What do you think their service is, exactly. Every single word that comes out of these systems is stolen IP, do you think that just because they won't generate a picture of Mickey Mouse for you it's not providing any IP?
Their service is understanding, interpreting, and generating text. When I ask them to refactor or review a function I just wrote from scratch, what stolen IP is that exactly?
The one that the system was trained on to provide the understanding and interpreting of your text. Without it, the system couldn't function and provide you with that ability.
Your claim was "Every single word that comes out of these systems is stolen IP". This code was never in the corpus of training data. How could it be stolen?
Are you moving the goalpost to "Every single word that comes out of these systems relies on understanding gained from stolen IP"?
Yes, I am saying exactly that. I guess I wasn't clear enough in my previous comment.
Then every single human being is also guilty of what you accuse LLMs of. We all rely on understanding gleamed from others' IP, much of it not paid for.
I mean, it's a very common argument and it's simply flawed.
You as a human are allowed to read the contents of say IMBD and summarise it to your friends free of charge. You can even be a paid movie critic and base your opinions on IMDB just fine. But if you build a website that says "I'll give you my opinion about a film for £5" and it's just based on the input from IMBD I'm sure we can both agree that you crossed the line - and that you're using another person's service to make your own business without compensating them. That's what LLMs are doing.
Honestly I'm just so tired of the whole "yeah but humans are the same because we also learn by reading stuff". These companies have effectively "read" everything ever made, free of charge, and are selling it back to us packaged in stupid bots that can only function because they were given that data. It doesn't compare at all to how a human learns and then uses information, unless you know someone who can do it on that kind of scale. LLMs don't "gleam" - they consume wholesale.
And then they complain that Deepseek copied from them haha
One man's unemployment is another man's freedom from a lifetime of servitude to systems he doesn't care about in order to have enough money to enjoy the systems he does care about.
Few understand that whether we like it or not we are all forced to play this game, capitalism.
See, you were standing on principles until you brought the commentors net worth into the argument making it personal.
Easy way undermine the rest of your comment
> Without being bothered about it at all.
I disagree: I see lots of evidence that he cares. For one, he cares enough to come out and say it. Second, read about his story and background. Read about Anthropic's culture versus OpenAI's.
Consider this as an ethical dilemma from a consequentialist point of view. Look at the entire picture: compare Anthropic against other major players. A\ leads in promoting safe AI. If A\ stopped building AI altogether, what would happen? In many situations, an organization's maximum influence is achieved by playing the game to some degree while also nudging it: by shaping public awareness, by highlighting weaknesses, by having higher safety standards, by doing more research.
I really like counterfactual thought experiments as a way of building intuition. Would you rather live in a world without Anthropic but where the demand for AI is just as high? Imagine a counterfactual world with just as many AI engineers in the talent pool, just as many companies blundering around trying to figure out how to use it well, and an authoritarian narcissist running the United States who seems to have delegated a large chunk of national security to a dangerously incompetent ideological former Fox news host?
Dario Amodei: "We want to empower democracies with AI." "AI-enabled authoritarianism terrifies me." "Claude shall never engage or assist in an attempt to kill or disempower the vast majority of humanity."
Also Dario Amodei: seeks investment from authoritarian Gulf states, makes deals with Palantir, willingly empowers the "department of war" of a country repeatedly threatening to invade an actual democracy (Greenland), proactively gives the green light to usage of Claude for surveillance on non-Americans.
Yeah, I don't know what your definition of "care" is but mine isn't that, clearly. You might want to reassess that. Care implies taking action to prevent the outcome, not help it come sooner.
The problem with counterfactual arguments like yours is that they frame the problem as a false dichotomy to smuggle in an ethically questionable line of decisions that somebody has made and keeps making. If you deliberately frame this as "everybody does this", it conveniently absolves bad actors of any individual responsibility and leads discussion away from assuming that responsibility and acting on it toward accepting this sorry state of events as some sort of a predetermined outcome which it certainly is not.
You make many good points.
Before I say anything else, I want you to know that I definitely don’t want to box anyone in with false dichotomies. I don’t think any of my arguments rely on them.
I’m not asking that you anchor on any one counterfactual exclusively. If you don’t like my counterfactual, reframe it and offer up others. I’m not a “one model to rule them all” kind of person.
If one of your big takeaways is we should keep our eyes open and not put anyone on a pedestal, I agree.
At present, my general prior that Amodei is probably the best of the bunch. This is a complex assessment and unpacking it might require gigabytes or even petabytes of experience. (I know that is a weird and unusual way to put it, but I like to highlight just how different people’s experiences can be.)
I am definitely uncomfortable with Palantir. Are you suggesting that Anthropic is differentially worse compared to other AI labs? Are you suggesting the other labs would do better if they were in Anthropic’s position?
If you don’t like the way I framed these questions, I suspect we have different philosophical underpinnings.
You might be aware that you’re implicitly referencing deontological ethics (DE). I’m familiar and receptive to many DE arguments. Overall, I’m not settled on where I land, but roughly my current take is this: for individuals with limited information and/or highly constrained computational resources, DE is generally a safe bet. It probably is a decent way to organize individuals together into a society of low to moderate complexity.
But for high stakes decisions, especially at the organizational level and definitely the governmental level, I think consequentialism provides a better framework. It is less stable in a sense. Consequentialist ethics (CE) is kind of a meta-framework (because one still has to choose a time horizon, discount rate, computational budget, evaluation function, etc.) It is rather complicated as anyone who has tried to build a reinforcement learning environment will know.
I fully grant that CE will admit a pretty wide range of concrete ethics (because the hyperparameter space is large). Some even can be horrific, so I don’t universally endorse CE. But done within sensible bounds, I think it CE is one of the most powerful and resilient ethical frameworks for powerful agents dealing with a complex world.
DE feels ok in the short run in areas where people have strong inculcated senses of right and wrong. But I would not trust it to keep the human race alive through rapid periods of change like we’re facing.
To be blunt, deontological ethics just cannot survive contact with modern geopolitics and AI risk. This is why I don’t put much stock in the kind of arguments that merely single out actions that don’t look good in isolation.
[dead]
Precisely
Anthropic never explains they are fear-mongering for the incoming mass scale job loss while being the one who is at the full front rushing to realize it.
So make no mistake: it is absolutely a zero sum game between you and Anthropic.
To people like Dario, the elimination of the programmer job, isn’t something to worry, it is a cruel marketing ploy.
They get so much money from Saudi and other gulf countries, maybe this is taking authoritarian money as charity to enrich democracy, you never know
>Anthropic never explains they are fear-mongering for the incoming mass scale job loss while being the one who is at the full front rushing to realize it.
Couldn't it also be true that they see this as inevitable, but want to be the ones to steer us to it safely?
Safely in what way? If you ask them to stop, the easy argument is Chinese won’t stop, so they won’t stop.
Essentially they will not stop at all, because even they know no one can stop the competition from happening.
So they ask more control in the name of safety while eliminating millions of jobs in span of a few years.
If I have to ask, how come a biggest risk of potential collapse of our economy being trusted as the one to do it safely? They will do it anyway, and blame capitalism for it
I'm not hearing an alternative here.