> The loudest reaction to Mythos Preview from other security leaders has been about speed - scan faster, patch faster, compress the response cycle. More than one team we have spoken with is now operating under a two-hour SLA from CVE release to patch in production [...] If regression testing takes a day, you cannot get to a two-hour SLA without skipping it, and the bugs you ship when you skip regression testing tend to be worse than the bugs you were trying to patch.
Over time, I wonder if these models will be able to generate more secure code by default by doing this kind of exploitability testing before ever merging their code.
I don't know, but it always seems weird to me when people notice AI isn't performing super well and then they conclude that the solution to problem is to try using more AI
Yeah why not? That's how I work. If I don't review my work, it's way worse than if I do review it and revise and iterate. I don't see why AI should be different: in fact it very clearly seems to be the case that is isn't.
I mean, I was sold something different. Something super human, vastly more intelligent, world changing. The reality is not that. Am I allowed to be disappointed and discouraged?
It has changed the world in major ways, although its not entirely visible because we've become numb to the idea of AI and AI being in everything.
It hasn't changed the way we sleep, wake up, eat, walk and talk so its not "life changing" or "world changing" in the sense a meteorite hit us, but each day thousands of mini meteorites are hitting Earth and we're becoming normalized to it one step at a time.
You are allowed to be disappointed and discouraged! For all the good tech that has come out of the AI revolution, most of it is ignored or shelved for things that can squeeze more and more money out of us and make our lifes worse, not better. Despite there being real potential to generate nice code, assist with biomedical research, self-driving cars, etc.
Which is it? Major changes or a bunch of small changes. I'm well aware of the small changes. I worked for an autonomous drone company back in 2008. It was really cool! In 2020 I started working for an autonomous car company. Again, amazing! None of it was a quick step function improvement. It was a lot of hard work. None of it was quite superhumanly smart either. LLMs are impressive pattern completion machines but they kinda suck at producing anything truly novel. Plus they are compulsive liars about that, lol!
Or they don’t, and they* sell access to Mythos and successors through their services company or network of partners and charge a premium.
* they, I mean all foundation models providers, as OpenAI seems to go in the same direction