+10, I can't stand AI-generated comments. Half the time they are paragraph-length translations of every 5 lines of code into English prose, or just regurgitating whatever prompt they were given. When they actually try making high level comments they are worse than useless. It's really hard to put into words exactly why. It's something about how they are very verbose, but at the same time vague and stated in a slightly-off manner. Or they will introduce new terminology not reflected in the code's nomenclature, sometimes terms that are just synonyms for terms that the code already uses. Something about it makes me instantly distrustful of any information the comment may actually convey.
locally correct and globally incoherent is how they seem to me. I, and a number of folks I have talked to, find some of them incomprehensible. Like each sentence makes sense, but the overall thing does not.
It's because they don't really have a mental model. RL with Verifiable Rewards can reward working code, but it can't tell you whether the prose make sense or not. In theory RLHF should be good with this. In practice, it appears to be a measure difficult to quantify and steer. I am not an insider though. So it's just a theory.
I have the same impression. It is an interesting phenomenon. Is it one of the ways we will gradually learn that AI in the form of LLMs is fundamentally lacking?
I use AI all the time, it gives a tremendous boost in speed for many tasks and it is most certainly here to stay, but there is something off about certain things, like code comments. Like a weird person.
I kind of wander if the comments are intentionally bad, providing programmers something to bike shed about. A sneakier approach to the flattery the models used to so use so heavily.