barely acceptable for all use cases? like a speed issue?

Regardless, looking back even a year at SOTA local models that fit on a non-absurd system creates a crazy trendline toward the future. And I think model routing is the gap filler that can make/break now vs. Oct 2027

I suspect Anthropic is of this general thinking as well; they've clearly switched from a thoughtful progression toward a natural IPO to "omg sprint for it". Along with the regulatory capture attempts, it's hard to imagine what is breathing down their neck so hard outside of this.

Totally agree local inference of any major scale is a nerd's wet dream short of many stacked improvements in efficiency. But again, auto-routing smooths over the rough edges if you are ok with a little bit of API spend when needed.