This is really cool. Someone here commented about similarity between this and hardware advancements for AV encode/decode.
I think it's only a matter of time before miniaturization can have a thumbnail sized user-replaceable accessory that contains the LLM built onto the hardware. I admit I don't know how any of that works, but would be amazing to experience. Fully local, fully offline, ultra fast local inference better than any personal computing product.
https://chatjimmy.ai/ Is that. Company behind it just got acquired by AMD
In 5-10 years nice smartphones will be able to run ChatGPT (~gpt3-4) class models. A memory rich laptop (highend mac/framework) can run GPT-OSS:120b or full Gemma4 at very interactive speeds.
High end phones can already run the smaller models at enough speed to be probably useful, especially for background/overnight photo tagging and curation and things like that.
In 5 years the models will probably be so much better and more compact that phones will be running models equivalent at least to Opus 4.6 if not Fable, at least within the areas they are tuned for (which probably won't include coding).
I'm really excited for this future. Both what you said and datadrivenangel.
The fact that Apple shipped a more than capable laptop for most of the population using a last generation iPhone chip is just mind blowing. Silicon advancements are going to allow this, and I think the global majority will catch up and make their own chips that compete or exceed western performance. Especially when the US is scared of science, rapidly divesting and defunding it.