Ran it on my adversarial programming language (KillSwitch) benchmark[1]. Very significant improvement over K3. On par with Gemini 4 Argon and Fable 5.1. Much slower than both though.
But this is now the highest scoring open-weight model. Amazing stuff.
Why does your benchmark have Opus 5 scoring the highest?
Because Opus 5 is the best at their benchmark.
yeah, it performed best for whatever reason. More specifically, it managed to deduce the meaning of the hack statement in the language better than Opus 5.5 did.
How many runs you make for each model?
Do you report an average score or the highest.
2 runs for each task, for each task it's either a pass/fail (the small green/red dots show each task pass/fail)
its not open weight.
not yet, but it will be, won't it?