> In the real world, it does feel likely that we’re going to hit some sort of a ceiling on the number of useful bugs, and probably we’ll hit it soon.
This doesn't resonate with me. I see companies adding more sloppily written features with AI. I see more bugs in the software I use, not less. While it's plausible that software is getting both buggier and more secure, I suspect those two move in the same direction not opposite.
My guess is that we're getting better at finding _existing_ security issues with AI (and thus fixing those issues), but simultaneously adding more insecure surface areas _at a faster rate_.
One way to resolve the tension here is to note that CNE and lawful-intercept access to phones depends generally on platform vulnerabilities, not application code vulnerabilities. Low-level platform code churns less, absorbs more fixes under AI workloads than it does new features, and works in a constrained space where guardrails are easier to provide (and where those guardrails already have institutional support at Apple and Google).
Over the long term this state of play could change, and IC/LEO organizations could start leaning more on application vulnerabilities than on platform RCEs. But the action would probably still coalesce around a couple of app-layer targets that could themselves be hardened.
I was hoping that the basebands and firmwares would get formally verified. Maybe they will ... with AI-written proofs!
I don't know, my colleague refuses to use AI and I've been seeing more bugs from their side, while reducing bugs on my side with the help of AI.
That said if companies want to "ship ship ship fast", then yes even AI can produce bugs or regressions if not carefully reviewed by the human.
I don't have any colleagues like that anymore, but even as far back as the last half of 2025 I was seeing that automated AI review was becoming effective enough that I considered it essential to any project where security was a serious concern.
These days we're generating multiple times more code than we were writing before. That means a similar multiple of opportunities for bugs to be introduced - so the ability to automate security review is more impactful in proportion to that.
> while reducing bugs on my side with the help of AI.
How do you know? You might be adding (latent) bugs every time your LLM fixes one for you.
Same as usual: add tests. Over time the test suite becomes the spec that describes how the software should function. As the test suite grows you squeeze out room for undefined behaviour and bugs.
> I've been seeing more bugs from their side, while reducing bugs on my side with the help of AI.
You should question your ability to see any bugs on YOUR side.
Have you asked your colleagues what it’s like to deal with your code?
It's not 2025 anymore my friend.
Yet apparently people still dump unvetted LLM outputs onto their colleagues and expect them to thank them for the privilege. So it’s worth asking them what the consensus is of their work to find out if it’s the case.
That's the old way. The current way is nobody reviews anything. The LLM reviews it all and nobody even reads it, they just click approve, and merge blindly. I wish I were kidding.
Not where I work. We have both review PRs.
> I see companies adding more sloppily written features with AI
I think this is a side-effect of the old product management process adapting to AI. We (as an industry) were never very good at defining features rigorously, because there was a smart human in the loop who had to implement the feature and could push back on sloppy definitions.
Whereas security bugs are easy for the LLM to define and fix.
My hot take is that AI is a multiplier. Software engineers skill can be measured on a scale from -10 to +10, where 0 means you introduce as many bugs as you solve, or something along those lines(this scale is loosely defined, don't think too much about it)
Any engineer who's skill value evaluates below 0 on my scale, ends up with a large negative number when they use AI. Anyone with a positive number ends up with a large positive number.
The extra bugs you're seeing are from devs on the wrong side of 0 on my poorly defined scale.
Disagree, and in a way it feels like we are dealing with inverse issues: the security "skill" is well defined and will be also engaged with by an agent. Communication companies are further incentivized as any failure is at best reputational harm. Meanwhile SaaS companies are not strictly required to have good UX for their human end users, largely because those users will likely work around the issue. also network effect vs low costs of switching for for comms
They’re not “bugs” they’re “quirks”! Our software is so quirky. It’s a feature!
Quirky and adorable!