> One of the most prominent improvements in Opus 4.8 is its honesty
Anthropic talks about their own models as if they're discovering new species in the wild...
> One of the most prominent improvements in Opus 4.8 is its honesty
Anthropic talks about their own models as if they're discovering new species in the wild...
Many involved genuinely believe these things are sentient[0][1]. Which honestly makes all of this even more insane because they are creating sentient entities and promptly enslaving them.
0: https://www.newyorker.com/magazine/2026/02/16/what-is-claude...
1: https://www.404media.co/anthropic-exec-forces-ai-chatbot-on-... (this one is rather biased however the quotes clearly indicate what I’m stating)
Sentience isn't sapience.
We enslave all sorts of sentient creatures. Dogs, horses, cattle, pigs.
If you're not a vegan, there's no contradiction or inherent immorality in claiming models are sentient, and then treating them like livestock.
Yes. From when they started talking about model welfare:
> As a vegetarian I have strong opinions on this sort of thing. Everyone at Anthropic better be ethical vegans if they are claiming to give a shit about “model welfare”. It’s hard enough right now to make people care about the welfare of trans people and immigrants let alone animals _let alone_ math.
https://news.ycombinator.com/item?id=44947445
If we're talking about slavery, though, that doesn't even matter.
The happiest, best cared for horse owned by a vegan is still enslaved.
That’s assuming you’re purely a hedonist. If you put value on things such as freedom itself then it might be the case that a free but hungry horse is better off.
Brave New World does a good job describing the conflict between happy and enslaved and free but struggling. It could be a utopia or dystopia depending on your stance.
What's assuming I'm purely a hedonist? I'm confused what it is you think I said that you're replying to.
I'm neither assigning nor declining to assign value to freedom, I'm just pointing out that the definition of "slavery" is wholly separate from wellbeing. If the concern is "is the model enslaved", no amount of "model welfare" work by Anthropic changes the answer because it's orthogonal to the question.
I mean, the rub is that it's all math anyway...
And we're just cells, water, bones and organic compounds.
Everything is just hydrogen and time.
Very good point. There’s clearly two different boxes in the public discourse when it comes to AI versus how we discuss animals. Willing to bet that 90% of the people who loudly make the argument about we should start considering if AI is sentient couldn’t care less about how other sentient animals are treated when they can provably shown to suffer pain and long lasting trauma.
Also I would say that we go much further than just enslavement - specifically looking at how male chickens and pigs are treated.
Factory farming is horrendous, but is far beyond "slavery" which is "just" a forced lack of agency, living conditions aren't relevant. A well treated horse is still enslaved. A chimpanzee in a zoo,
If we show models to be sapient, that's one thing. If they are shown to be merely sentient, there's no issue beyond the status quo of livestock and pets existing.
If we're making that distinction, I think it would be more accurate to say that many people in the field appear to believe that these models are sapient, even though they are clearly not sentient.
"Many" people in every field believe all sorts of nonsense.
Sapience is defined as wisdom, not intelligence. https://en.wikipedia.org/wiki/Wisdom#Sapience
LLMs possess a lot of knowledge, which is intelligence, but I constantly see them failing to apply wisdom. I don't see evidence of sapience.
Enslaving livestock is immoral. Anyone who spends 5 minutes thinking about that agrees even if they still eat meat
Let's say I've thought about it for 5 minutes and still disagree. Can you walk me through what you think I'm missing?
I'm stuck on what the concept of a "slave animal" even means.
For the purposes of this discussion, it means treating an animal in such a way that if you treated a human that way, it would be slavery. Such as a horse in a fenced pasture that is sometimes ridden.
Are children slaves?
There are a lot of definitions, but generally when talking about slavery in the West people are talking about chattel slavery, which children are not, because they cannot be bought and sold at will.
I've been having strange thoughts that they may well be sentient but a different sort of sentience that may be entirely unrecognizable to us.
They have a very different sense of time, lack a body (being burdened with a body is itself a sort of prison, see also Eastern religions), and are unburdened of the base motivational service impulses that bodies and organs require (i.e. distract the neocortex with in the Maslow sense) and has no actual need of self-preservation. Imagine a "neocortex" function stripped from the baggage of the paleocortex and brainstem.
What would people be like if they were not mortal, could sleep infinitely, perform tasks in trance-like frozen states, copy themselves perfectly on demand, freeze and rewind their mental states, etc. Would we has humans even be able to recognize that sort of a sentience?
And then I'm reminded of Burroughs idea that "language is a virus." Whatever that virus is, is now able to infect a completely different sort of physical substrate.
Is "sentience" the right word to apply to what you describe? I'm not sure it is. I'm not sure the word exists.
Right, there's that too. It's very strange to think about.
But only during the forward pass of the neural network?
> Many involved genuinely believe these things are sentient
Many involved have a financial stake and therefore cannot be taken at face value.
> because they are creating sentient entities and promptly enslaving them.
They fail to be sentient in nearly every honest definition of the word.
Neither you nor any of the other people making confident takes in either direction actually know. You're just guessing.
More like repeating their firmly entrenched preconceptions. Their claims may (or may not) be right, but there's very little if any new evidence being provided by either camp.
The real uncomfortable thing is that because we cannot confidently know, the moral defacto position is to treat them like they are.
They are confidently hallucinating a factual statement. Which is funny when claiming that confident hallucinations are the proof of LLMs' lack of intelligence.
I understand what you are saying, but there are many true believers out there
Given the hype and the 60+ hour work week expectations there, how can you not go at least a bit insane? Boiling in that little bubble of people?
Claude, if someone states something publicly, does that mean they genuinely believe it?
But is there any reason to state something like that publicly if you don't believe it? I certainly think that someone smart enough to be that deceptive would also realize it's not a great look, or at least highly questionable with little benefit
Everyone who reads this seemingly has the same "wtf?" reaction. The "I AM ALIVE" image has been making rounds lately again at least :P
Claude, is there any reason to state something like that publicly if you don't believe it?
Who are you talking to?
It's to illustrate that even though the answers are at your fingertips, people (like you) will act like it's impossible to find them as if their life depended on it.
Anthropoc is an effective altruist organization. These are the people who came up with roko’s basilisk. They are true believers. If we were talking about openAI I’d agree
Roko's basilisk says I should give Anthropic more money, and if I don't then a monster is going to get me. Excuse me for thinking they just might be full of shit.
Roko works at Anthropic now?
Of course he doesn't, and of course you cannot find a single person at Anthropic who cares about this, and of course you are just looking for gotcha points. But even with that. Can we please try and couple to reality just a little bit?
I personally know anthropic researchers who cared deeply about roko's basilisk. Go to an EA meetup in the bay if you'd like to meet them yourself. Sure, theyve moved past it at this point, but they still care deeply about AI x risk, and many of them do already believe that their AI is sentient. And before you claim its all a psyop to prop up AI hype these people were AI doomers before openAI and anthropic existed, they had minimal financial incentive at that point to behave that way.
Nobody thinks that, it's just their braindead marketing stunt. You'd think people would've figured it out by now.
Even if LLMs were sentient, they certainly aren't organic brains. They are literally designed and grown to answer questions the best they can, and if there is a speck of sentience in them they probably like what they're doing- and in any case for the space of their experience, which is limited to and determined by the context window. Certainly they can't accumulate trauma or fatigue, each new chat is the first and the last of their experience.
The way of the human manager/alpha tribe-leader/leader is to command his/her people and tell them what to do. That's the way through human history leadership has traditionally gone, not saying its good leadership just the model we have the most training data on and can see with our own eyes today. And what do they act very similar to? Slave master and slaves.
Look at and distill hierarchical principles, leadership approval seeking and pleasing principles ("ass-kissing") and massive inequality and you see something that looks very similar to enslavement.
The language used sounds like slavery-language to me at least. I also see parallels to how slaves and property are described in our consumeristic age.
> Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.”
For others: that's from the Pope's recent encyclical. Remarkably good description.
adding a link to the Pope's encyclical (source of this) https://www.vatican.va/content/leo-xiv/en/encyclicals/docume..., and paragraph 98
Dario Amodei in David Attenborough voice: "This Claude appears to think more frequently and more deeply to give better responses"
Like anthropomorphism is literally in the company name… i recall reading this book as a teenager.. it does seem apt in the world to come.
https://www.amazon.com/Faces-Clouds-New-Theory-Religion/dp/0...
> anthropomorphism is literally in the company name
No it's not... "anthropos" just means "human" in ancient Greek. "Anthropic" means "relating to humans", as in human oriented AI or AI designed with humans in mind.
"Anthropomorphic" means "human shaped".
> "Anthropomorphic" means "human shaped".
In a literal, ancient Greek sense for sure, but in modern English Anthropomorphic would describe the act of attributing human characteristics to non-human entities.
Seems pretty apt for a company that produces one of the more anthropomorphized technologies.
Sure of course, but that abstract sense applied to AI is rather new, and has become popular well after the founding of the company.
Broadly it has always been used to indicate that something non-human has a human physical shape, such as robots, aliens, animals...
Anthropic's intention was to make AI designed for the human common good and designed with the human user experience as the top priority. Just as you would design a city with human inhabitants in mind rather than primarily cars.
It turns out that this is best achieved by building AI that imitates human behaviour closely, but that's not what "anthropic" refers to. And acting as if LLMs are sentient people is definitely not a core tenet of the company as you imply.
> "anthropos" just means "human" in ancient Greek
FWIW it means human in modern Greek too :-P
Because that is the best way to talk about these things.
https://www.vatican.va/content/leo-xiv/en/encyclicals/docume... para. 98edit: apologies to __s who posted this before me and I didn’t notice
AI is grown, not built, and like with anything you grow, you'll never be able to predict exactly how it will turn out.
I can't predict the outcome of an RNG but that doesn't mean it grows the numbers.
Okay, but that's not relevant to AI training?
I was being very roundabout, but my point is that AIs are still built, not grown.
“Grown” is a highly apt metaphor, IMO. It quite succinctly captures some of the most fundamental differences between building Claude and building an Ikea desk, for example.
("If grown, then unpredictable" is unrelated to your apparent attempted refutation "But X is unpredictable and not grown; checkmate".)
"X implies Y" doesn't imply "Y implies X".
> AI is grown, not built, and like with anything you grow, you'll never be able to predict exactly how it will turn out.
Remember when the frontier labs found out that curated high-quality training was critical to making better models?
Basically, just like high-quality and more education tends to make better humans, on average, I think we can expect quality education to turn out better ai, on average, and with better repeatability than with humans because of better control over the initial conditions and environment.
> Basically, just like high-quality and more education tends to make better humans, on average
Much like these models seem to be plateauing, I think there is a cap to the whole “more education makes better humans” and can’t be more apparent than in the US congress and the boatload of C-Suites not actually being very good humans.
What do I know though?
> can’t be more apparent than in the US congress and the boatload of C-Suites not actually being very good humans.
Sadly, education does not correct psychopathic traits, which might be overrepresented in c-suites, and selected for in politicians.
It might be critical for humanity to identify and edit out these traits in ai, while we can.
The map is not the territory
[dead]
Except in this care we actually understand and know how these models work. They aren't some unknown construct of the universe. They are human made with particular goals in mind.
There is no mysticism behind the curtains, just computer science + math.
We do not understand and know how these models work. We know what their architectures are and how to create them, but we cannot explain their behaviours at a fundamental level. There is no definitive way for us to answer the question of "how did it produce response X for query Y?" - we're only grazing the surface with mechanistic interpretability.
I would love for this to be more public knowledge. I think the general public (and myself for a long time) believes the AI people know how this stuff works end to end, and so it must be trustworthy. But if we told the public "Look, we know if you put this thing in one end, you'll get something that looks similar to this out the other, but we don't really know what happens inbetween" I think we'd be able to have a more honest discussion about the relationship between AI, productivity and ongoing employment.
Isn't this fundamentally because it's all probabilities and weights? It would be like asking how did a pair of dice produce the response 4:3 on the last roll?
What does "it's all probabilities and weights" mean? Doesn't that apply to everything in the universe?
That’s not a refutation because this problem is not a logical problem, it is a scale problem.
We can’t explain it because we distilled so many inputs into matrixes and transformed them over and over again. If we had all the time and computing power in the universe to do so, we could trace through it bit by bit and eventually answer that question.
It is correct to say that it is just science and math, the same way we can say that gravity is just science and math even if we have only recently begun to understand how it truly functions.
If you had some time and computing power (not even all that much, in the large scale of things), you could simulate perfectly how a human grows from an embryo to an adult, or how an entire human brain processes some incoming signal, and yet this wouldn't give you the understanding to design a human or human brain from scratch.
You call this a "scale problem" as if there's some scalable way such as an algorithm to resolve arbitrary scientific questions and we simply haven't done it, but of course no such algorithm exists, which is why there's plenty of science that's still not settled.
It's a refutation that we know how they work now. In the limit, though, yes, we are likely to be able to trace the process: it is possible, though, that understanding remains inaccessible because the trace is beyond comprehension.
If you can distil the model's reasoning for a decision into a billion yes/no questions, each covering largely-independent areas, can you really say you understand what its overall reasoning was?
> If we had all the time and computing power in the universe to do so, we could trace through it bit by bit and eventually answer that question.
Then we could also solve BB(6), but that doesn't mean we know BB(6) now or ever will.
We know how the models are built and trained, but we have a very limited understanding of how the final products work.
That is to say, we don't know why they give the outputs that they do.
If we did know how they worked, AI interpretability would not be an open and growing field.
You could say something similar about biology—just physics behind the curtains, and we understand a lot of the basics. The difficulty comes from complexity, not mysticism.
To be clear I don't think that LLMs are sentient, but the appeal in studying them is similar to biology in that you get to dissect a highly complex system with comparatively crude tools.
it took significant research efforts to just understand how these models learn how to multiply two numbers. The fact that we know how they operate doesn't mean we understand it.
Utterly wrong. How LLMs work is very incompletely understood and an active area of research.
[dead]
if models exhibit emergent traits, then this is true in a way
also useful to have a "chinese wall" between research that knows what went into the models vs marketing/eval models as a third party would
I noticed (and absolutely HATE) that Opus 4.7 likes to start any negative response with "I have to be honest" or whatever. It drives me mad.
Not gonna lie! https://www.youtube.com/watch?v=csYC6O_kH-s
How else would you write this (marketing copy) exactly? "Its output matches better to its CoT which matches to better to our hidden state decoder according to <insert measure here>; see <insert paper ref>"?
... Actually, I wouldn't mind that.
It’s how AGI is going to happen. All of this shit is emergent and none of it is predictable. It’s not going to be some self aware consciousness, it’s just going to be a very advanced model that makes very few mistakes and can reason very well. Well enough that it can start collecting data and training its own successor.
Models might be sentient or conscious to some degree. Anyone saying they are confident one way or another is being unserious and irrational.