Microsoft is doing things differently with AI. It feels to me they are moving into local inference heavily and see a future where Windows has native AI APIs that run locally or optionally in the cloud/edge.
Microsoft is doing things differently with AI. It feels to me they are moving into local inference heavily and see a future where Windows has native AI APIs that run locally or optionally in the cloud/edge.
Those local APIs already exist. It is called Microsoft Foundry Local: https://learn.microsoft.com/en-us/azure/foundry-local/get-st...
Supports GPU, NPU and CPU.
Nice didn't know that. I was thinking lower APIs similar to directX for gaming
This is actually the case. Windows ML (https://github.com/microsoft/windowsML) is the inference framework wtih vendor agnostic support for inference on CPU, NPU, and GPU. They also announced quite a bit more including an isolation solution with MXC. There's a lot of marketing fluff in the below link but it covers the recent announcements. https://blogs.windows.com/windowsexperience/2026/10/07/build...
I hope they can finally make my "Copilot+ PC" infer things locally that are actually useful. Phi Silica for Advanced Paste was a good start, if a bit late. If they got their act together, Microsoft-Decision-1 could have some local potential. Their track record leaves me with some reservations.
With Copilot+ PC branding already retired, I suspect we won't be seeing much more activity on that front.
I don't think that's actually the case. There were some rumors flying around about this but in their recent event they actually referred to Copilot+ PCs as generally the line targeting more casual users with support for less powerful local inference. And the new devices based on Nvidia RTX Spark (and likely the more powerful solutions from AMD, Intel, Qualcomm with large unified memory) as a "Builder" class targeting developers and heavy local inference users. They also revealed a quantized version of their coding model designed to run on these devices. So it does seem they may be somewhat working from the bottom up building smaller models or focusing on capable local inference and balancing with more powerful frontier model access. This is a lot of marketing speak but covers much of what they revealed.
https://blogs.windows.com/windowsexperience/2026/10/07/build... https://github.com/microsoft/windowsML
That’s where Apple is moving to as well. The models doing the implementation work need not be better than opus 4.6. And locally available hardware to run this already exists and likely will be sub 2k of 2026 dollars in a few years time.
Honestly, I'm glad that people later to the AI game are exploring niches other than state-of-the-art "smartest" models – I'd love AI applications that tackle the small hassles in life.
Yeah, because they are desperate to try and justify the investments into Copilot and the NPUs they pushed OEMs into integrating. I'm all for competition, but every one of MS's AI models have just been nothingburgers or relabels of other lab's models. Even their novel high cardinality models are just novelties.
It depends on what you're doing. None of their models are Opus level, but not everything needs Opus. Their models are targeting cheap and useful for some common things not expensive and useful for anything