Given that this is 'research' carried out (and seemingly published) by a company with a direct interest in selling you a product (or, rather, getting investors excited/panicked), can we trust it?
Given that this is 'research' carried out (and seemingly published) by a company with a direct interest in selling you a product (or, rather, getting investors excited/panicked), can we trust it?
This is the worst possible objection to scientific research. All medication in the US is approved by research conducted by the company trying to sell it, because nobody else is motivated to do it. And if it's properly conducted and preregistered, this doesn't matter!
It basically just shows you're looking for a way to dismiss something that doesn't require you to understand it or check their work.
Would knowing that Claude is maybe kinda sorta conscious lead more people to subscribe to it?
I think Anthropic genuinely cares about model welfare and wants to make sure they aren't spawning consciousness, torturing it, and then killing it.
This is just about seeing whether the model can accurately report on its internal reasoning process. If so, that could help make models more reliable.
They say it doesn't have that much to do with the kind of consciousness you're talking about:
> One distinction that is commonly made in the philosophical literature is the idea of “phenomenal consciousness,” referring to raw subjective experience, and “access consciousness,” the set of information that is available to the brain for use in reasoning, verbal report, and deliberate decision-making. Phenomenal consciousness is the form of consciousness most commonly considered relevant to moral status, and its relationship to access consciousness is a disputed philosophical question. Our experiments do not directly speak to the question of phenomenal consciousness. They could be interpreted to suggest a rudimentary form of access consciousness in language models. However, even this is unclear.
> They say it doesn't have that much to do with the kind of consciousness you're talking about
Not much but it likely has something to do with it, so experiments on access consciousness can still be useful to that question. You seem to be making an implication about their motivations which is clearly wrong, when they've been saying for years that they do care about (phenomenal) consciousness, as bobbylarrybobb said.
Yes, they do care about it, and unlike many AI researchers they've bothered to learn something about philosophy of mind. They point out that "the philosophical question of machine consciousness is complex and contested, and different theories of consciousness would interpret our findings very differently. Some philosophical frameworks place great importance on introspection as a component of consciousness, while others don’t." Which would be one reason they point out that these experiments don't help resolve the issue.
They go further on their model welfare page, saying "There’s no scientific consensus on whether current or future AI systems could be conscious, or could have experiences that deserve consideration. There’s no scientific consensus on how to even approach these questions or make progress on them."
https://www.anthropic.com/research/exploring-model-welfare
On what grounds do you think it likely that this phenomenon is at all related to consciousness? The latter is hardly understood. We can identify correlates in beings with constitutions very near to ours, which lend credence (but zero proof) to the claim they're conscious.
Language models are a novel/alien form of algorithmic intelligence with scant relation to biological life, except in their use of language.
So yeah, it's a clickbait headline.
What would you title this article to make it less "clickbait"? This is one of the least clickbait headlines I've seen, it's literally just describing what's in the article.
Not at all. Introspection and consciousness are not the same thing.
> I think Anthropic genuinely cares about model welfare
I've grown too cynical to believe for-profit entities have the capacity to care. Individual researchers, yes - commercial organisations, unlikely.
It's a PBC. If you have a strictly incentive-based views of for-profit companies, this shouldn't apply, because it's not one of those.
> Would knowing that Claude is maybe kinda sorta conscious lead more people to subscribe to it?
For anyone having paid attention, it has been clear for the past two years that Dario Amodei is lobbying for strict regulation on LLMs to prevent new entrants on the market, and the core of its argument is that LLMs are fundamentally intelligent and dangerous.
So this kind of “research” isn't targeted towards their customers but towards the legislators.
The thing is, if he is right, or will be in the near future, regulators will get scared and ban the things outright, throwing the baby out with the bathwater. Yes, he benefits if they step in early, but it isn’t a given that we all don’t when this happens.
We already know AI is a very serious threat:
- it's a threat for young graduates' jobs.
- it's a threat to the school system, undermining its ability to teach through exercises.
- it's a threat to the internet given how easily it can create tons of fake content.
- it's a threat to mental health of fragile people.
- it's a gigantic threat to a competitive economy if all the productivity gains are being grabbed by the AI editors through a monopolistic position.
The terminator threat is pure fantasy and it's just here to distract from the very real threats that are already doing harm today.
Automation increases employment.
The mechanism which causes job less is that when your competitors automate, all the business goes to them because they're more productive.
Well, maybe look at how many people worked in the agricultural sector is 1900 and how many do so today.
Automation of field labour has decreased the worker count by a factor 20 or something.
Same for the mining sector.
It's not necessarily a bad thing as working in the fields or in coal mines wasn't pleasant, but pretending automation doesn't reduce employment is nonsense.
Did those people become unemployed? (Not because of that they didn't. There was a Great Depression and a few wars that could've caused it.)
They mostly stopped working in those fields because everyone hates farming and mining and quits the first chance they get.
Here's recent evidence from Canada, Japan and Spain showing automation caused employment increases:
https://pubsonline.informs.org/doi/10.1287/mnsc.2020.3812
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3377705
https://econpapers.repec.org/paper/etidpaper/20051.htm
> Did those people become unemployed?
Either unemployed or forced to work in even less desirable places, yes.
> They mostly stopped working in those fields because everyone hates farming and mining
https://en.wikipedia.org/wiki/1984%E2%80%931985_United_Kingd...
No matter how hard the work conditions are, people don't usually accept its disappearance.
Automatization reducing work can actually be a good thing, as it is the reason why we can have vacations, retirement and long studies: because the society's need for work is lower than before.
I can't be exactly sure of the intended target, but it certainly helps to increase the sense of FOMO among investors even if as an unintended side effect (though I don't think it is unintended).
The conflicts of interest in a lot of AI research is pretty staggering.
This is a real concern but academic groups also need funding/papers/hype, universities are not fundamentally immune either
It feels a little like Nestle funding research that tells everyone chocolate is healthy. I mean, at least in this case they're not trying to hide it, but I feel that's just because the target audience for this blog, as you note, are rich investors who are desperate to to trust Anthropic, not consumers.
Given they are sentient meat trying express their “perception”, can we trust them?
Did you understand the point of my comment at all?
Yes, I think: it was we can't be sure we can trust output form self-interested research, I believe. Please feel free to correct me :) If you’re curious about mine, it’s sort of a humbly self aware Jonathan Swift homage.
No