I completely understand what you are saying but ip does make an interesting point.

Why would chain of thought work at all if the model wasn't gaining something by additional abstraction away from binary?

Maybe things even go in the other direction and the models evolve a language more abstract than English that we also can't understand.

The models will still need to interface though with humans using human language until we become some kind of language model pet dog.