> I suspect a possible future of local models is extreme specialisation - you load a Python-expert model for Python coding, do your shopping with a model focused just on this task, have a model specialised in speech-to-text plus automation to run your smart home, and so on.

I'd find this very surprising, since a lot of cognitive skills are general. At least on the scale of "being trained on a lot of non-Python code improves a model's capabilities in Python", but maybe even "being trained on a lot of unrelated tasks that require perseverance improves a model's capabilities in agentic coding".

For this reason there are currently very few specialist models - training on specialized datasets just doesn't work all that well. For example, there are the tiny Jetbrains Mellum models meant for in-editor autocomplete, but even those are AFAIK merely fine-tuned on specific languages, while their pretraining dataset is mixed-language.