Some AI company needs to create a model that can delegate simple tasks to 'stupider' models. I often encounter a task which is complicated enough to require a strong model like Opus, but which subdivides into a number of tasks - the vast majority of which are enough that 3.5 Sonnet could pick it up. All Opus would need to do is subdivide the task into easy and hard bits, then spin up a bunch of 3.5 Sonnets for the easy stuff.
This seems like such an obvious idea that I'm sure everyone is already working on it!
Claude code does utilize both the full Sonnet model and the lighter Haiku model in an automatic way. When you exit a Claude code session, it gives you the stats (tokens, cost, etc). I expect there’s a way to get this in-session as well.
I believe you can just hit /cost within a session for this
Only if you're using it without a subscription, and without one it doesn't take much usage to get to $20 in a month
I think Auto-GPT or a similar project did something like this in 2023.
Aider has an option where you can combine different models, so one does the thinking and one implements the changes.
https://aider.chat/docs/leaderboards/
e.g. o3 (high) + 4.1
There used to be more such combinations in this list but it seems they were removed. A while ago I think the best thing on the list was such a hybrid.
Maybe have the prompt tell the model emit the model "level" needed between 1 and 10 for each subtask it generates