I don't think this point is well stated. We have things like ECC to help with math being imperfect so that feels solvable. The problem with LLMs is much more subtle. They induce cognitive errors in humans. I'm doing it right now I'm saying "them" like they have individual identities, but they do not.
If you know they're just mirrors, then you know that looking in the mirror isn't the same as exploring the world.
It turns out, though, that many people can be trapped in a hall of mirrors. They will happily buy a condom for their brain.
Right. And when we talk "to" the LLM we address it as "you" or give it commands as if there is someone there to respond.
Maybe we need a new pronoun for it. 2nd person inanimate. YOU -> YAI?
This week I was experimenting with leaving out "you" pronouns by changing everything to passive/subjunctive and leaving out the subject.
E.g. when referring to something the AI has written or done: The code that was written... Instead of: The code you wrote.
For something I want it to do: May the code be modified to include the following constraints... Instead of: Modify the code to include the following constraints
I don't know if that's really helpful but it's good to be cognizant I'm not really talking to anyone.