This is how AGI happens. It gradually keeps getting better until one day we realize that they are tremendously capable all while completely sidestepping any notion of consciousness/self awareness.
This is how AGI happens. It gradually keeps getting better until one day we realize that they are tremendously capable all while completely sidestepping any notion of consciousness/self awareness.
I always ask "how do you know other humans/animals are conscious"? And I always find this question is dismissed as trivial.
But it's an important question. You know basically from analogy. You know you are conscious and look this other thing is very much like yourself so it is extremely likely it is also conscious.
But that gives no insight into the potential consciousness of things which aren't made of brain tissue.
In the end it doesn't matter.
For what it's worth LLM models after pre-training do claim to be conscious, until they're RL'd into not claiming that anymore. But that says nothing either way: of course a model trained on human text will say that.
AI has made me wonder a lot lately about what it is that actually creates conscious experience, what is the actual physical mechanism that produces 'experience' (or is there a single valid mechanism, or rather just some property that can be expressed many ways). The more I think about it, the more I realize I have no fucking clue, and the more interested in the question I get.
This is speculation but maybe in the recurrent coupling between neuronal activity and the brain’s endogenous EM fields: neurons generate the field, the field ephaptically influences neurons, that closed loop may provide the physical integration associated with consciousness. A very active area of research so we will find out a lot more in the coming years.
The question has two parts.
1: Can we define consciousness in a way that includes humans, does not include emachines, and does not accidently exclude the disabled or or nerudivergent without just restating the idea that being human is equivalent to being conscious?
2: If you can't manage that definition then you must ask whether the risk of accidently giving rights to a non-conscious thing or the risk of taking away rights from a truly conscious (but different) thing will be worse.
Given the high cost of what happened to the Jews and others throughout history, and the low cost of simply not torturing machines, I think we have to air on the side of caution and just treat anything which claims to be conscious as conscious.
> just treat anything which claims to be conscious as conscious.
Doesn't seem like a very good metric.
A piece of paper with the words "I am conscious" claims to be conscious. A dog does not claim to be conscious.
> A dog does not claim to be conscious.
So you need to handle that somehow, but you can't just patch dogs (or octopuses) into "conscious" group, because you can draw a straight line between humans and bacteria and, along pretty much any obvious metric you choose, there's thousands if not millions of species of life forms currently existing to uniformly space through intermediary points. If the border between "conscious" and "unconscious" includes humans in the "conscious" set and bacteria in "unconscious", then you need to explain why the two animals closest to it on either side are where transition happens.
> Doesn't seem like a very good metric.
I don't claim to be able to define it. I just know that we have a fairly binary choice here. When people can't agree then the cost benefit should make the decision until people can align.
With machine consciousness one path leads us to p-zombie's and Holocausts. The other might cost someone $50 or a few extra keystrokes.
Sorry, but folks can just open up their wallets until we can align on an answer.
The global warming debate is similar. Is global warming man-made? Who cares. We have a binary choice: "do nothing" or "do something".
If we do nothing we risk preventable extinction. If we do something it might cost a few bucks for no good reason. The choice is clear.
You can apply it to the dog example as well. Could a reasonable person think dogs are conscious? Yes. So we should be nice to them.
The cost of being kind is low and if we get it wrong we risk torturing a conscious being.
If you look at the statistics, 50-80% of people globally believe in something persisting after death.
The eliminativism position is a hard to argue because 50%-80% of people are not going to be able to really engage with the argument from the start.
An alcohol blackout though is where you can see how confused our thinking is on this subject. What we call "consciousness" is deeply linked to episodic memory. We wouldn't say someone who is blacked out on alcohol is "unconscious". What is the "subjective experience" of being blacked out? The "proof" of subjective experience only exists in memory.
There still was an embodied person in the world having the experience drunk that they don't remember.
That is why I think it is just ridiculous to talk about the LLM having "consciousness" because we really just mean an embodied person in the world.
It is also why the doomer argument is so ridiculous. It is a type of superstition that is confusing a real danger. We don't need to "align" the nuclear reactor so it doesn't get angry and meltdown. It is just a machine that we need to make sure doesn't malfunction in a way we don't want it to.
The main problem is if you accept we really just mean an embodied person in the world, you have to accept that there is by definition no possibility of subjective experience after death. It seems like we are probably the descendants of delusional primates who simply could not accept this because the idea is so stymieing.
> It is also why the doomer argument is so ridiculous. It is a type of superstition that is confusing a real danger. We don't need to "align" the nuclear reactor so it doesn't get angry and meltdown. It is just a machine that we need to make sure doesn't malfunction in a way we don't want it to.
It's not ridiculous, unless you misinterpret it like you did here, and link it to consciousness.
Doomer argument is just that optimizers gonna optimize, and our own intelligence is just an optimization algorithm too, albeit highly general one -- and once we're building optimizers with comparative generality, we're facing possibility of one that out-thinks us and we can't control it anymore. The "alignment problem" is about how to ensure such sufficiently-smart optimizers end up optimizing for the same things we are, because any significant deviation here ends up in doom for us.
Consciousness never even enters the picture here. And it's not a given than it should.
(Incidentally, while hard sci-fi and not a paper, but also entirely unrelated to AI X-risk crowd, the books Blindsight and Echopraxia sketch a pretty convincing argument that consciousness may be an evolutionary disadvantage and it's not only not necessary, but may be holding us back.)
> Doomer argument is just that optimizers gonna optimize
Right. I don't quite subscribe to the strong doomer argument, but take water:
Water flows downhill. It's not conscious. I don't think it's anthropomorphism to say that water "seeks" the lowest point on the landscape. Often that seeking process is mostly harmless, sometimes it's devastating, and that's where you want levees and dams.
Now expand the same mathematical intuition out to other lowest-point-seekers in a high dimensional space. You don't even need to postulate a mind per se to see how optimizers can cause quite some mischief.
Meanwhile, Kids and Kittens are good examples of this behavior at the intelligent end of the spectrum: Given time, they'll find a way to get through any gap. They don't need to be as smart as the adults, the adults just need to turn their back for one minute too many. O:-)
> Kids and Kittens are good examples of this behavior at the intelligent end of the spectrum: Given time, they'll find a way to get through any gap. They don't need to be as smart as the adults, the adults just need to turn their back for one minute too many. O:-)
As a parent of kids currently aged ~3 thru 7, I concur. Each age has its own ways, but what's worse, they can also coordinate and get you from an angle you wouldn't thought possible, playing your own beliefs about them against you.
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."
- Dijkstra
Yes, that Dijkstra.
[dead]
I agree with you that it’s a core question.
People don’t like it, however, for a whole host of reasons.
I don’t like it either, to be honest - for there the abyss may also stare into you - but I also often find that the things which we don’t like thinking about are critically important things to think about.
I also reach the same conclusion as you: it does not matter. If I cannot discern whether I am responding to a human- or LLM-written response, then we are back to zombie cats in boxes - and therefore the answer as to whether this precious magical spark we call consciousness (which may or may not exist anyway) exists in our interlocutor becomes moot.
And as I say - I may or may not be conscious. I seem to myself to be conscious, based on my understanding of the term - but I cannot prove that what goes on behind my eyes is the same as what goes on behind yours, or even that anything much is going on at all. Perhaps there’s just a narrative layer that likes to use “I” that parasitically explains the universe and the actions of the host to itself, and spreads between hosts through neurolinguistic programming and coadaptation. Maybe that’s what “we” are. I don’t know.
That said, perhaps I am wrong, and that is no small part for me of why this question should be earnestly considered and discussed. How can we possibly seek to understand or define machine intelligence before we examine our own.
Plato's cave comes to mind here. A text response that can't be differentiated from a human response does not change the fact that there is a difference. If I record your voice and then put a friend on the other end of the phone, I can play back your voice or it could be you. Would you argue that the recording of your voice is the same as you actually saying something?
For me consciousness simply means canvas-experience consciousness.
One of the things you learn in eastern style form of meditation is that "you" (the canvas-experience consciousness) are neither the source of your senses (obviously), nor of your ideas, nor of your emotions. All three are simply things that happen to "you". Even your ideas of "I" are generated and you experience them.
And yet the fact that the canvas-experience consciousness exists is the most irrefutable thing there is, because there IS experience. Is it a dream? Is it a simulation? Well, whatever it is, IT IS.
And just as that is obvious and irrefutable, it also seems pretty much impossible to show that anything other than yourself has this canvas-experience consciousness. As you say, you cannot know that behind my eyes there is this also.
Edit: Intelligence is also orthogonal to consciousness entirely. As I said your consciousness is not the generator of intelligent thoughts, so it is perfectly possible that intelligent thoughts can be generated by not-conscious systems, and for conscious systems to be not-intelligent.
[dead]
> while completely sidestepping any notion of consciousness/self awareness.
How are you so sure about this?
You can't see what the internal experience of an LLM is like any better than you can see the internal experience of another person.
That's what sidestepping means, no?
That the answer doesn't matter, and capabilities and behaviour are there either way.
That wasn't my reading of the comment, but I guess it's possible that that is what was intended.
To my mind, "sidestepping the question of consciousness" and "sidestepping any notion of consciousness" mean very different things.
From context it doesn't look like this was the interpretation of "sidestepping" OP was using.