What about Sonnet? Haiku? Does it need Opus specifically?
Seems to me that the important factor is that Jev has well-calibrated confidence for this task. Which secondary expensive model it defers to is probably less critical in this technique as long as that model has higher performance than Jev.
Yeah exactly - very interesting that Luna is calibrated so poorly despite working better on easier problems.
Seems to me that the important factor is that Jev has well-calibrated confidence for this task. Which secondary expensive model it defers to is probably less critical in this technique as long as that model has higher performance than Jev.
Yeah exactly - very interesting that Luna is calibrated so poorly despite working better on easier problems.