Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.
Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark
Which is why I don't take the benchmarks at face value...
So you have decided the model is bad without having used it yet?
If they had a model on par with Opus 5.5 there would be no need to undercut it by that much, that's probably the logic.
Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.
No, I have decided I don't trust benchmarks without using the model first.
Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark
...no it's not? At max effort, it scores 50% on terminal bench vs. Opus 5.5's 70%.
ALE (top benchmark in post) is 60% for Terra vs 63% for Opus - I guess we'll have to wait for AA Index!
48 for GPT-6.1 Terra Max vs 58 for Opus 5.5 Max on AA Index