I dislike AI, but if AI finds real bugs then this is in my opinion objectively a positive thing. Of course the question is what constitutes a real bug.

[deleted]

Unfiltered models will help build exploits for the bugs they find, so there is some means of measuring their efficacy.

If you're just talking about security bugs.

There are also non security bugs that don't have exploits but just make the user experience worse.

A.I. is useful for this. But it would be even more useful if all new code were written in Rust or some other memory-safe language.

A.I. could also be used to port C/C++ codebases to Rust, which isn't economically feasible at the moment.

AI will have plenty of security bugs left to find in Rust codebases.

I mean I get the sentiment but Rust won't save you against division by zero, it'll just panic at runtime like every other language.

From a security perspective, panic at runtime is not that bad for security. Much better than continuing to run with undefined behavior. If someone sends a malformed video in and it crashes the ffmpeg process you can just log it and restart it. Vs potentially exploiting the system.