It's also that humans are very bad at repetitive detailed tasks. Sitting down with a code base and looking at each function for integer overflow comparison bugs gets boring really fast. It's a rare person who can do that for as long as it takes to find a bug that they don't already have some clues about.
It's the flaw in the "given enough eyeballs, all bugs are shallow" argument. Because eyeballs grow tired of looking at endless lines of code.
Machines on the other hand are excellent at this. They don't get bored, they just keep doing what they are told to do with no drop-off in attention or focus.
idk man, pay me enough money and I’ll look at as much code as you want looking for integer overflows
Would it be cheaper than Claude Mythos doing it? No idea. Maybe, maybe not.
But it’s weird how we’re willing to throw away money to a megacorp to do it with “automation” for potentially just as much if not more as it would cost to just have big bounty program or hiring someone for nearly the same cost and doing it “normally”.
It would really have to be substantially less cost for me to even consider doing it with a bot.
> idk man, pay me enough money and I’ll look at as much code as you want looking for integer overflows
So would I, but it doesn't negate that we, humans, are bad at this. We will get bored and our focus will begin to drift. We might not notice it, we might not want to admit it, but after a few continuous hours we will start missing things.
And there aren't enough security researchers in the world to review ALL the files from OpenBSD.
And if there were, the cost would be more like $20M than 20K.
Having all code reviewed for security, by some level of LLM, should be standard at this point.