Indeed this is one of the key problems here. If models start doing CoT in "neuralese", or use it when talking with each other, we lose what little observability we have.