I think part of the “just use a CLI” crowd might also be building similar agents as ChatGPT and Claude.ai web interface. I know at least 4 teams doing that in one company.
All those teams, including ChatGPT and Claude.ai, have figured out that you will eventually need to give your agent a small sandbox Linux environment to unlock the same level of “intelligence“ those coding harness exhibit. Stitching together the results of a cli command through scripting or coding gives the agent a ton more flexibility in what it can do as it can utilize its text generation capability into executable logic. toolcalls mostly work for actions rather than complex and novel problem solving. You are making the agent represent a programming control flow through toolcalls while carrying the context between them in a lossy, nondeterministic, wasteful, slow and rigid way.
It’s one thing if you want to artificially limit that agent to a very strict set of available APIs that it must use in a specific way while transferring context between them through the LLM and you don’t want to incur the cost of the extra sandbox compute. But coding harnesses have demonstrated that letting the agent write a small shell or python script can let the agents solve problems that you haven’t even really anticipated in your toolcall approach or that tool calls make prohibitively expensive or not even possible.
But also the token cost tends to dwarf the sandbox compute cost, so why not pay the $0.05/hour to have a sandbox where the agent can run free when you are already paying orders of magnitude more for the tokens
Hmm yeah but I think at some point ad-hoc code becomes a signal that something is wrong. eg. If your LLM is continuously writing python to join customers to orders at some point that's a signal that customers_aggregate('topspenders') needs to be a thing like a deterministic API call
Every arbitrary REPL or program written by an LLM to solve a problem can absolutely become a toolcall that does the same thing. Are you planning to provide hundreds, or thousands, of tools to cover all the possibilities an agent might consider AND have the foresight to include all of them?
I think that’s a future plenty of people see or want. But simpler and composable Linux commands are far more flexible than pre-defined Windows or Mac “Settings App”. The barrier to entry for the former was knowledge and familiarity, while the latter optimizes for simpler user experience. Agents don’t have to worry about knowledge or familiarity the way users do.
Ok, now you have customers_aggregate(‘topspenders’), but the prompt to the agent was “Get me the top spenders that are public companies and show me ones that increase their spend proportionally to their revenue increase and ones that don’t”. If the agent had access to the needed APIs (or CLIs) for your data and a finance API provider, it can generate a script that crunches down the top 10 or 10,000 for all that data. You could obviously come up with a toolcall that encapsulates that or rely on multiple toolcalls and on the LLM to fill in data correctly between them then do the math using inference.
Edit: even if you look at that customers_aggregate(‘topspenders’) tool, you wouldn’t have needed that if the agent could execute a SQL query. It can also express anything the data can do. You obviously trade the control and correctness guarantees you can give for a `customers_aggregate(‘topspenders’)` vs a potentially wrong SQL query generated by the LLM with the infinite flexibility of a SQL query to be able to express far more than you could in a customers_aggregate call.
I think MCP (or any kind of external API) serves a different purpose than ad-hoc code generation and scripting.
In my view, the MCP should be part of the specification. It should expresses intended actions and/or restrictions while the code generated and executed by the LLM is an implementation detail.
This distinction is completely obvious in some cases such as using an external payment gateway. But let's consider a more borderline case.
Say your database is multi-tenant and every query absolutely must include a tenant_id. Now that doesn't necessarily mean you have to use MCP. It could be some command line tool that requires a tenant_id parameter.
But what you can't do is ask the LLM politely to please include a where clause for the tenant_id in any and all SQL statements it decides to generate. You have to impose some things deterministically and MCP is one way to do that.
At that point you would add a `your-service-cli list-customers --order-by=spent` command, which would also be useful to humans and scripts, as opposed to an MCP tool call, which is only ergonomic to models.
Right we can quibble about the wire protocol but the point remains that the arbitrary REPL becomes suboptimal as a solution when there’s not a deterministic endpoint. So ‘scripting’ is not really what we are looking for; we just want the final data output and scripting is fine when it’s a one off
They’re not the same thing under a different wire protocol!
An MCP tool call that returns the top N spenders puts the result in the agent’s context window.
A CLI tool call on the other hand… The agent can pipe their results into other completely unrelated CLI tools. It can write a loop that calls multiple tools.
These things are not possible with MCP but are trivial with CLIs.
Anthropic proposed a way to programmatically chain MCPs together a while back, I'm not sure if it's been implemented much yet: https://www.anthropic.com/engineering/code-execution-with-mc...
I don't even need to read that to know they're re-inventing PowerShell now.
Edit: I read it. Yep.
We have text interfaces refined by humans for decades and there's an endless sea of training data for them, but they imagine these amateur-hour homegrown solutions will ever outdo an agent with shell access?
Fully agree that in the end sandboxes are required to get frontier performance out of the models.
But you can have both: rund the agent outside the vm/sandbox and orchestrate work on it, either directly via shell calls or kicking off an ephemeral subagent on the box.
This makes the agent and session that runs outside the vm more durable and opens new orchestration pattern.
I’m building the oss version of this here: https://github.com/smartcomputer-ai/lightspeed
We have those sandboxes already, it is called serverless, and those microservices are now extended with MCP endpoints metadata.