I think we'll eventually get to the point where folks will have a local AI agent but I think people need to temper their expectations to a degree. You aren't going to have data center level tok/s from a box sitting under your desk and you don't need instantaneous responses for many workloads. Having a local agent that can execute tasks over a couple days with your supervision that might otherwise take you weeks is perfectly acceptable.
However I also think that Agentic AI is very much not an out-of-the-box solution, local or otherwise, and it takes a high level of technical knowledge to create an effective AI agent. And there's a problem now where most orchestration is fixed on what models are used for what tasks with no ability to weight constraints like cost, speed, and security.
The solutions will come in the past, big iron, thought they were immune and not too long afterwards, personal computers took over and big iron was history well, the same thing is going to happen all over again.
Those companies building those big data centers are going to find out that they overspent. Because we’re not going back to the mainframe era, no matter how much OpenAI, Anthropic, Microsoft, Meta or Google would like to.