> disguise the underlying dogma, which serves as an unsupported conclusion: humans are assumed to be completely entirely unique in every way whatsoever

Is that the argument the paper is making? In my reading they seem to primarily be making the point that assigning anthropomorphic concepts to LLM is dangerously misleading, and more importantly, not needed to properly study and evaluate LLMs.

I don’t think you have to make the assumption that humans are unique for that argument to hold up. I would argue that really it’s a comment on how loose and poorly defined all anthropomorphic attributes are. At the end of the day we have to make the assumption that other humans feel and experience broadly the same mental activity as each other, because we’ll never directly experience anyone else conscience, we can only experience our own.

We can barely link our own mental experiences to concrete empirical measurements. The vast majority of the measurements we make are entirely self-reported, and we simply assume strong correlation between self-reported measurements and the individuals actual experiences. We also have to assume that somehow all of our self-reported measurements are “calibrated” to some reasonable degree. Even measuring anthropomorphic properties in humans is pretty fuzzy and inaccurate, the only reason accept such poor data is because it’s the best we’ve got, and there enough signal in there for us to develop useful tools like talking therapy, physiological profiles, mental health scores etc which have some level of predictive and healing power when applied to _humans_.

It’s honestly amazing that what we have works for measuring and predicting humans, and we only know that works through decades of empirical measurement and study. But to then try and directly apply that fuzzy mess to a completely different system, and just assume the same level of predictive power, strikes me as kinda crazy. It requires huge assumptions, which effectively can never be tested (because even the human mind is a total mystery to us), to be made, and if we can study these systems without making those assumptions, then why make the assumptions at all?

You're missing the satire.

And the 'argument', which is a funny way to recast the chinese room argument, which has also been discussed to death.

And you're also assuming any kind of position other than your own dogma -- that AI has Intelligence In Its Name and Humans Have Intelligence Therefore AI Has Human-Like Intelligence -- is based on some religious belief in the specialness of humans instead of pointing out where this analogy between Intelligence in its two senses breaks down.

Hacking meme aside, lots of computer scientists are overextending their domain expertise into an area that has been well studied by philosophy and biology. It isn't surprising the software is good and the philosophy looks like outsider art.

Here are some relevant pointers to connect this discussion to the existing philosophy on the subject: https://en.wikipedia.org/wiki/Emergence

https://en.wikipedia.org/wiki/Emergentism

Basically there's a lot of cases where some properties arise from sets of a thing, which would otherwise not be present in a single or few things.

One classic example is that a single molecule or drop of water does not express fluid mechanics.

And of course, a bit more basic would be materialism, but maybe you are one of the lucky hundreds:

https://en.wikipedia.org/wiki/Materialism