This entire essay basically boils down to one simple contention: LLMs can’t be conscious because only humans can be conscious and LLMs are not human.
What an unimaginative argument: essentially ruling out the possibility of the title by definition. And the author fails to touch on the really interesting question posed by the article’s title: what if human consciousness is more like the working of LLMs than we think?
What? Where are you seeing such claims? I.e. that only humans can be conscious, that AI in general is not conscious by definition?? Seems to me you've put up an easy-to-dismiss strawman.
Here's how I would summarize crux of the argument: LLMs (specifically) are, by construction, chameleons. More precisely, role-playing machines. When compelled to have a conversation as "themselves", all they do is take on the role of "themselves", as inferred from the training data plus the text of the conversation so far, just like they do for any other role. This is fundamentally different than we humans. When we have a (normal, non-roleplaying) conversation, we don't put aside whatever role we might have been acting out before and instead take on the role of "ourselves" and act from this new perspective. Acting as ourselves is a completely different way of interacting than any play-acting we might do for example as an actor in a play, or when trying to predict the future behaviour of a third-party. Even in terms of energy expenditure, acting as ourselves is much less strenuous than trying to simulate someone else's perspective.
IOW, we have an inherent perspective, an I like an inner eye which looks at all things with a certain slant or tendency. LLMs don't. There's no I inside them, instead they can take, and in a sense are, all Is possible.
Err… right here in TFA?
>> The first requirement is that the computer program has a body (either physical or virtual) and sense organs
LLMs may be chameleons, but, to steal an excellent quote, if you can’t tell does it really matter.
My personal take on this is that consciousness is a thing induced, not a thing evoked. If it generates a response on the observer as a conscious thing would it is, by my definition, conscious.
If TFA says "only humans can be conscious," you may know many more humans that have a virtual body made up of virtual organs than I do.
why does AI/LLM deserve special scrutiny regarding consciousness or the lack of?
why not publish a paper stating - 'the radio is not conscious just because it can speak words'. or "the phone is not conscious because it can speak and listen".
The argument doesn't state that at all. It very clearly admits consciousness to nonhuman animals, for example.