This looks and works great, super impressed. This is what Siri/Apple AI should be like.
One concern I have is about sending private info to Anthropic. So it would be great to have the option to use local AI.
Local options should support the OpenAI API that seemingly is the standard for these connections, as provided by Llama.app and Unsloth Studio for example. That way one could run against a local model and keep personal information private.
That of course begs the question if a local model is smart enough for the tasks at hand.
I'm also trying out OOMU, a similar but closed source personal assistant app for MacOS. It emphasizes privacy with local models, and even has an 'Airplane' mode. The app has a built in llama.cpp server and uses Gemma-4-4B, their own router model plus a vision model.
Thank you so much!
Most daily commands like launching and moving around sites/apps happen without AI, but I agree with the privacy concern and will definitely look into utilizing local AI.
Smaller models, like Haiku, I have found to already struggle during multi-step tasks, so I expect local model capabilities to reduce performance further.
I'll look into how OOMU splits their work up, but what local model would you suggest starting with?
The choice of model could be up to the user, depending on the available RAM on their M class Mac. Newer and bigger usually being smarter.
Llama.app/.cpp and Unsloth Studio on Mac already provide guidance to pick the optimal model for the available memory, all model wheights hosted by huggingface. Both apps can use the local huggingface model cache locally.
If you make it configurable, the user could also run a separate machine with an NVidia card and Linux to serve tokens on the LAN. Or use OpenRouter or any other cloud hosted model offering (DeepSeek, Z.ai, etc.).
I would probably run qwen 27B or qwen fn but I would run the actual inference on one of my other boxes and not on actual desktop so initially it could be sufficient if you just enabled openai or anthropic compatible endpoint.
[flagged]