Wouldn't they do exactly that if they were trained on enough text with punctuation mistakes?

No because of post training