This is complete speculation, but I wonder if the "thinking step by step" paradigm, when under reinforcement learning pressure, also encourages grammatically ambiguous sentences (like garden path sentences in the extreme case) that lets the model "keep its options open" until very far into the sentence.