Except LLMs are "System I" writers. You'll be more satisfied with what they've written if you haven't really understood it and just let it wash over you.
It only falls apart when you actually try to figure out what's being said.
I find the most passionate advocates of LLMs right now to be the people who are
1) also the most passionate advocates of not reading or reviewing their output in any detail. They seem to be actively annoyed by the idea that people think they should. And
2) people who have a strange theory of mind about LLMs that excuses the low quality of their output. I've heard many times people advancing the theory that LLMs are coding in a particular way because the way that LLMs code is the way that they (the LLMs themselves!) are best able to work with the code.
In fact, I get the impression from these people that they think that their touching the code would actually ruin this delicate balance that the LLM is trying to make. This is actually crazy thinking. On a pretty obvious level, they already think that any LLM they encounter is a super-intelligent God acting in mysterious ways, and that they only way you can ruin its plan is by questioning it.
> I find the most passionate advocates of LLMs right now ...
Whether or not that's true, you have to keep in mind that it's not true of everyone.
I'm a passionate advocate of LLMs. I implemented my first neural net, for image recognition, in the late 1980s (based on Bart Kosko's work, I'm not claiming to have invented anything.) Now, I use LLMs daily for coding and other tasks. I don't usually "vibe code" unless it's a simple enough project that the code quality really doesn't matter - a simple example would be data integration, where as long as the data in the target system matches the data in the source system, I don't care much about how the sausage in between is made.
But otherwise, I'm approving or denying every tool call, like a caveman, and when they mess up, I explain to them why. I interrupt them and redirect them when they need it. I change code myself and let them know. With all that, they're still incredibly useful - more useful than most "junior programmers", realistically. No human software developer can hope to match LLMs on many significant dimensions of development. But that doesn't mean experienced and competent humans have no role, at least currently.
The disconnect here is that your characterization of "the low quality of their output" is too simplistic. The quality of their natural language writing is different, for the kinds of reasons stated in the OP. Some of the accusations of "lower quality" really boil down to "too compressed for humans to easily understand", again as the OP points out. And the quality of their code has a lot to do with how they're prompted.
One of the biggest barriers to good LLM output - both writing and code generation - is that they're trained on data generated by humans who fit George Carlin's observation that half of them are stupider than average. When it comes to coding, on average they don't have enough education not to think that imperative code with minimally-restricted mutation all over the place is a good idea. They think dynamic typing, so that code can't be statically verified, is a good idea. They reflect our own collective limitations. If you recognize that and redirect them to sources of good practices, they can be much better.
In short, they're much more of a mirror than most people realize or admit.
With all this noise, it should not be forgotten that we appear to have figured out roughly how a significant component of intelligence works, and have real life, useful proofs of concept. All the technical criticism, to me, is just coming from people who don't seem to understand how intellectual advancement works. The political criticism is another matter.