AI has a way of using abstract language, sounding impressive and eloquent, but in reality it is mostly incoherent fluff. You can see this in humans too. Some politicians and academics are good at expressing themselves in clever ways but when you actually try to parse what they say it makes no sense what-so-ever.
I don't think the analogy holds because AI still produces good code.
No it doesn't. It produces working, but sloppy code, and it's able to do so because of deterministic test suites that run quickly and give a reliable score of the result, letting an agentic loop iteratively improve its output. But we have no test suites that answer "is this code maintainable" or "does this feature make sense in the context of our business", so AI still fails those tests.
If we had a way to write a test suite for English that could reliably answer "is this sentence useful to my target audience" or "does this paragraph make any goddamn sense", you might be able to run some sort of agentic loop to approach good writing, but so far we don't.
The code is very maintainable, just not by humans.
> But we have no test suites that answer "is this code maintainable" or "does this feature make sense in the context of our business", so AI still fails those tests.
I've been struggling with this because I do believe maintainability of code is important. But the way it is going, if AI is writing all the code and no human needs to read it anymore, does it really need to be maintainable? Does it matter if it looks like alien code since we can throw it away when using the next AI model? I have no answers of my own to these questions...
In my work I use AI for code generation in two distinct ways:
- For throwaway prototypes I pretty much vibe code, only making manual tweaks occasionally if the agent gets stuck or something looks wrong visually. - For production code I stay in the loop, reviewing and iterating on the code with an agent.
This leads to me having experience with “messy” prototype code and “maintainable” production code. Whenever my prototype code lives longer than expected it gets to a point where the agent starts introducing regressions or churning on changes.
This happens much less often with the “maintainable” code. From my perspective it seems pretty clear that LLMs are faster, more effective, and less error-prone in a well structured codebase
> But the way it is going, if AI is writing all the code and no human needs to read it anymore, does it really need to be maintainable?
It's more important than ever, because code is just a way of expressing intent. If the code becomes overly complex, it de-coheres, and intent is lost across the system you are building.
This is why vibe coded projects turn into slop and eventually stop iterating: at some point that actual intent is too vague and contradictory because of the many layers of code that have been added.
No, it also produces impressive looking code that is actually not that great. It's exactly the same.
I agree except that it usually is correct code (at least in the sense of it passing some unit tests) because you also RL directly against that.
But yes, in my experience the code is often quite bad in other, harder-to-quantify directions: readability, maintainability, simplicity.
You can verify that code is good if it works - it might be ugly or hard to understand, but if it completes the goal and doesn’t crash then it’s good. You can’t really do this with prose though because the text is the thing itself - it needs a human to score it (perhaps another way would be to have other agents ask it questions about something they don’t know, and see if the agents can discern the answers from the output - but that’s still not going to determine if it’s good for human consumption).
it is not because it work that it is good. It still need to be maintained and this is a huge deal
Sometimes it doesn't, but it's tough to see because of the volume of what you have to wade through
I think it is more about comprehension. It reveals that AI does not actually understand what it does. It gets code right mostly because it is probabilistic and has seen enough sound precedents to land in something that works, but it does not come from an internal comprehension.
For code it also has tools to verify correctness. From specifications to compilers and tests.
With humans language, sometimes "wrong" is in fact "right."
Good point. I also think that the training data is different in this regard. It has trained on (mostly) verified, correct code, but when in comes to free text it has trained on a lot of unfounded opinions.
Unlike human languages, computer languages have tightly-specified boundaries so LLMs do a lot better. See also math.