> IMO, Migrating to small-scale local LLMs would be a significant improvement over using data centers.
LLM serving is most efficient when you batch a lot of parallel requests together. Data center solutions also have the advantage of collecting queries from around the globe, so the hardware can be utilized around the clock.
Having everyone serve their own local LLMs would produce a lot more memory demand. Not less. The same memory would be idle most of the time, and when it was used it would be used for 1 person instead of a batch of requests.
There are other reasons to run local LLMs, but solving hardware demand problems is not one of them.
Yeah, local LLMs are probably an order of magnitude less efficient at a fixed level of "intelligence" if not more.
Inefficient per watt, yes, but local inference capacity is greatly underutilized in aggregate. If a model can run on a machine that already exists, that's a bunch of additional chips that don't need to be built.