Sometimes, but that's an edge case that doesn't seem to impact the productivity boosts from LLMs

It doesn't until it does. Productivity isn't the only or even the most important metric, at least in software dev.

Can you be more specific with like examples or something?