I feel this sentiment is wishful thinking,but I want to start by saying I hope it turns out to be correct.
I find that often bugs will be created when using an LLM, like others have said. Saying that this can then be fixed by identifying all the bugs created by an LLM with an LLM doesnt guarantee another bug is not introduced when the LLM is addressing the initial problem.
Also, what if the LLM has a blind spot. They certainly also could be incapable of finding or fixing a bug. They dont pass any benchmark at 100% right now. Also also, guaranteeing there are no bugs in your code is like saying you have 100% test coverage, all of the tests pass, and they are written perfectly. Saying that you can simply identify and fix the bugs also assumes there is enough time and energy to find all of the bugs that exist within a project and then to address them. Even LLMs use time and energy. In a sufficiently complex system that is certainly wishful thinking.
Considering the size and complexity of a lot of modern software (like web browsers, 3d modelling software, game engines, etc.) software is just too complex to not have bugs even when created and managed by LLMs.
There will continue to be bugs in code and we will simply have to live with the fact that LLMs make it easier to exploit computer systems. I mean consider a hardware bug like Spectre [0]. If bugs like this become easier to find does that mean our existing hardware will just become obsolete more quickly? that type of problem can be addressed, but at quite a high cost.
Not sure what all of this means for the future.
0. https://en.wikipedia.org/wiki/Spectre_%28security_vulnerabil...
If LLMs can trivially find bugs, then they can trivially find bugs. If they can't find any bugs that doesn't mean there are no bugs but it suggests that others can't easily find them either. So the LLMs find all the bugs problem is fixed by asking the LLMs to find them before you ship.
Read what wrote, I didn't say your program will be bug free. I said, if the LLM can trivally find the bug it will. If it can't then we're at worst, back to the state of before LLMs could find bugs, but likely much better since we fixed so many of them
So, the fact that LLMs can trivially find bugs is enough to get the bugs fixed.
You, and several others, seemed to think I was saying LLMs would fix all the bugs. I never said that. I said they'd help. Finding them is help. Writing a possible fix is often help. Writing a possible fix and seeing if they can detect a bug after the fix is applied is also help. Automating the entire things and letting LLMs fix them without review is likely not help.
The bugs I can trivially find, the bugs Claude can trivially find and the bugs Codex can trivially find are not necessarily the same. The most obvious bugs would be obvious to any of us three, but beyond that we would't agree on a definition of 'trivial bug'.
This gets worse if you factor in different harnesses built for the task, and future model updates
Sure, things will get calmer than they are now, but shipping without bugs someone else considers trivial will still require more effort than most are willing to invest
But by using a LLM everyone is automatically devoting that much effort, and so while we don't fix all bugs we fix a lot that wouldn't have been.
Ok, I get a better idea of what youre saying from this reply than your original comment. It wasn't helpful to me that you suggested I reread your original comment.
I agree that LLMs make finding bugs take far less time and energy. I also agree that this should mean in the long run there are less trivial to find bugs IF everyone adopts the usage of LLMs while writing and reviewing code.
It does also seem possible that LLMs are better at finding bugs than fixing them.