I agree with your speculation to be honest. And yet I’ve tried several local open weights model now and none gives the same quality of answers as Claude gives me on a regular Sonnet model. Mind you: I am “running” 48GB of RAM so I can’t try every model. Where does this difference come from? Can we actually get close locally?
You’re running 48GB now but imagine a future where everyone has 512GB RAM or 1TB RAM in their computers (it might sound like a lot but also 20 years ago we had 512MB PCs).
It’s not hard to imagine what 5-10 years of pressure to increase RAM will do to specs, on top of the normal tech improvements.
That’s worth bearing in mind when thinking about local models.
Plus, local models keep getting better and better; 2 years ago what you could get out of those 48GB of RAM was embarrassing compared to what’s doable today.
We’re getting there. Just takes time.
Something I've noticed is that local models are giving better answers these days than they did a year or two ago, even if the size (in parameters and in the amount of RAM used) hasn't increased. I'm not familiar enough with the technical side of model training to explain how they're doing this, but I think in another couple of years, models that use up 48 GB will be able to squeeze out even more incredible performance.
Though on the level of something like Sonnet 5... well, maybe not.