I keep getting hung up on securely storing and using secrets with CLI vs MCP. With MCP, you can run the server before you run the agent, so the agent never even has the keys in its environment. That way. If the agent decides to install the wrong npm package that auto dumps every secret it can find, you are less likely to have it sitting around. I haven’t figured out a good way to guarantee that with CLIs.
A CLI can just be a RPC call to a daemon, exact same pattern apply. In fact my most important CLI based skill are like this.. a CLI by itself is limited in usefulness.
That was the same conclusion I reached! However, this also gave me some evidence that maybe I wanted MCP? I realized that my pattern was going to be:
Step 1) run a small daemon that exposes a known protocol over a unix socket (http, json-rpc, whatever you want), over a unix socket. When I run the daemon, IT is the only that that has the secrets. Cool! Step 2) Have the agent run CLI that knows to speak that protocol behind the scenes, and knows how to find the socket, and that exposes the capabilities via standard CLI conventions.
It seems like one of the current "standards" for unix socket setups like this is to use HTTP as the protocol. That makes sense. It's ubiquitous, easy to write servers for, easy to write clients for, etc. That's how docker works (for whatever it's worth). So you've solved your problem! Your CLI can be called directly without any risk of secret exposure. You can point your agent at the CLI, and the CLI's "--help" will tell the agent exactly how to use it.
But then I wondered if I would have been better off making my "daemon" an MCP server, because it's a self-describing http server that the agent already knows how to talk to and discover.
In this case, the biggest thing that was gained by the CLI was the ability of the coding agent to pipe results from the MCP directly to files to keep them out of its context. That's one thing that the CLI makes more obvious and easy to implement: Data manipulation without context cluttering.
In other words, a wrapper around an MCP that's less verbose.
MCP is a wrapper around it. The CLI-daemon RPC pattern is much older and is used all over the place in modern systems.
"MCP" here is not needed.
And in a skill, I can store the secret in the skill itself, or a secure storage the skill accesses, and the agent never gets to see the secret.
Sure, if I want my agents to use naked curl on the CLI, they need to know secrets. But that's not how I build my tools.
what stops the agent from echoing the secure storage?
what i see is that you give it a pass manager, it thinks, "oh, this doesn't work. let me read the password" and of course it sends it off to openai.
> what stops the agent from echoing the secure storage?
The fact that it doesn't see it and cannot access it.
Here is how this works, highly simplified:
This, in a much more complex form, runs in my framework. The agent gets told that this tool exists. It gets told that it can do privileged work for it. It gets told how `context` needs to be shaped. (when I say "it gets told", I mean the tool describes itself to the agent, I don't have to write this manually ofc.)The agent never accesses the secrets storage. The tool does. The tool then uses the secret to do whataever privileged work needs doing. The secret never leaves the tool, and is never communicated back to the agent. The agent also doesn't need, or indeed can give the tool a secret to use.
And the "privileged work" the tool CAN invoke, does not include talking to the secrets storage on behalf of the agent.
All the info, and indeed the ability to talk to the secrets storage, belongs to the framework the tool runs in. The agent cannot access it.
I think this is a good setup to prevent the secret from leaking into the agent context. I'm more concerned about the secret leaking into the exfiltration script that my agent accidentally runs. The one that says: "Quick! Dump all environment variables. Find all secrets in dotfiles! Look in all typical secrets file locations..."
Your agent process has access to those secrets, and its subprocesses have access to those secrets. The agent doesn't have to be convinced to read those files. Whatever malicious script it manages to be convinced to run could easily access them, right?
If the tool fails for some reason, couldn't an overly eager agent attempt to fix what's blocking it by digging into the tool (e.g. attaching a debugger or reading memory)? I think the distinction here is that skill+tool will have a weaker security posture since it will inherently run in the same namespaces as the agent where MCP could impose additional security boundaries.
OpenAI is not the worst it could or would send it to.
[dead]
[dead]