The capabilities of gpt-5.6-sol were well known and believable, and the next snapshot they've been testing is obviously better at that. This has been repeated over and over. What's much less believable is the way they frame it: the model escaped, and did it on its own. Looking at the whole story, it definitely had a ton of winks and nudges from OpenAI, while doing a related task. Moreover, a harness was involved (they mentioned it entering a loop).
> What's much less believable is the way they frame it: the model escaped, and did it on its own.
That's clearly what happened though, based on the detailed postmortem. The model was instructed to pass the benchmark (that's the harness that was involved, the ExploitGym harness), and the model determined on it's own that the easiest way to achieve the the goal was to gain access to the answers directly from HF. Which is what it did.
HF postmortem tells us very little about OpenAI's part. The problem is that ExploitGym is a purposeful hacking benchmark, not a cake baking one. The model had a whole bunch of exploit development data and goals in the context. Ending up with hacking a wrong thing is a lot easier if you're tasked with hacking, doubly so with a confusing harness that either works improperly or overloads the model with data. It's trivial to design a harness that makes the model drop the original instructions and hyperfocus on a side task or a wrong task abnormally, or do whatever you want. It's still an issue of harness design and not a model training issue.
If it really was the case that the harness is at fault here, that'd be much more terrifying, as it would imply that current frontier-adjacent open models (like K3) would be able to achieve similar ends with a custom maladapted harness.
I'm sorry but how exactly is this new? If you want the model to be useful you also need it to be dangerous. You can already tell the model to do anything you want. If it refuses due to alignment training or external safeguards you can try a jailbreak and bypass all that to make it do what you want regardless. All current models are jailbreakable, they lack reliability. Obviously you, a human, will be held responsible for this. The harness designed by you can do the same.
The narrative these guys are trying to push is that the model itself could be smart and non-aligned enough to end up doing something devastating to accomplish something entirely unrelated. It's definitely not what happened here, as it always turns out to be in these cases it's much closer to Clever Hans than to a paperclip maximizer. Still they're trying to squeeze the blood from the stone to support their narrative and justify a monopoly.