As far as I can tell, the actual argument in the paper is that an LLM instantiated in AoE II would (a) be very slow, (b) maybe not actually input or output text, and (c) just generally look silly. Therefore observers would not naturally ascribe anthropomorphic characteristics to it. But you don't need the Turing-complete embedding for this argument. You can run inference as slowly as you like and detach the tokenizer. Show some silly representation of the internal computations. Now it's just a cute art project producing sequences of numbers. No human characteristics there!
Exactly. It actually is a cute art project and I quite like seeing their in game perceptron running, but if it were presented as a serious argument then it’s just an appeal to absurdity that rejects the notion of substrate independence. It’s trying to get the reader to reject the premise on the basis of a gut feeling of what a game is, the gut feeling of course hides that it’s just another way to do computation. But I guess that’s the point.