I'm surprised Sonnet 5.5 did noticeably better than Opus and Fable. Same pattern in their other benchmarks.
It didn't really do any better, the difference is well within the error bars for their benchmark.
I like their benchmarks. They ask the right, not easy questions: https://surgehq.ai/blog/sudo-l7
Has anyone verified their figures to see if they hold up?
It didn't really do any better, the difference is well within the error bars for their benchmark.
I like their benchmarks. They ask the right, not easy questions: https://surgehq.ai/blog/sudo-l7
Has anyone verified their figures to see if they hold up?