Why can’t the LLM you’re talking to on the web have access to your CLI tools?
When you talk to an LLM on the web, the harnesses spin up a fresh environment (I would hope it’s a VM…) so that the LLM can do stuff like run arbitrary Python and Bash scripts to complete the task you asked it for.
There’s no reason why you shouldn’t be able to customize this environment to add whatever CLI tools and credentials you need for the agent to act on your behalf.
The UX would be exactly the same.
Could be more expensive to provide a computing environment.
It can’t possibly be more expensive than doing equivalent work using context instead of RAM and inference instead of CPU.
Unless we’ve got the wrong balance of compute availability and inference availability right now, but I would expect the market to stabilize at some point.