>Well, here's the thing not enough people talk about: we're giving these tools god-mode permissions. Tools built by people we've never met. People we have zero way to vet. And our AI assistants? We just... trust them. Completely.
I keep seeing this pattern in articles: "Did you know that if you point the gun at your foot and pull the trigger, yOu ShOoT yOuRsElF iN tHe FoOt??!? I couldn't believe it myself!! What a discovery!!1!"
Are people really this oblivious or are these articles written about non-issues just to have written 'content'?
There was that article a few months ago about an AI code assistant deleting a company's production database: https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-d...
There's layers here, of course:
1. The founder shot themselves in the foot by not understanding an AI tool couldn't be trusted, so clearly they really were that oblivious.
2. ...the founder had direct production access hooked up to their regular coding environment. They were always going to shoot themselves in the foot eventually.
> There was that article a few months ago about an AI code assistant deleting a company's production database
That wasn’t a company. It was a vibe coding experiment. It didn’t have any actual customers.
It did delete the database marked “production” though. If it had been deployed it would have been a problem. It was just an experiment, though.
It was even worse than that. It was basically an ad for a security product.
dogfood.ai?!
This article was published few days ago citing notion's mcp server and it looks legit
https://www.codeintegrity.ai/blog/notion
That's a really heartwarming story. Thanks.
What’s obvious to the audience of HN isn’t necessarily obvious to anyone else.
Articles like this are intended to serve the latter group of people.
And it’s true, AI agents with MCP servers are pretty much unsafe by design, because security was never considered from the start. Until that changes, if it ever even does, the best thing to do is to inform.
The recent npm supply chain attacks were directly targeting people in the HN crowd. Don’t think because you’re here and know tech that you’re immune.
The npm supply chain attacks (or any similar ones) are essentially the same issue described in the article. You can't trust 3rd-party provided code implicitly. Even if the code is initially fine it's subject to change in later revisions. This issue goes all the way down the stack. Obviously, with a large user base the likelihood of quick detection goes up, but the issue never goes away.
> You can't trust 3rd-party provided code implicitly.
But that is what (almost) all of us do.
There is debate about this the rust world, where there are mitigations that very few even aware of
Mostly rusticans block their ears, close their eyes, and pretend everything will be just peachy
Until some ordinary developer develops a crypto wallet in Rust, honestly, that steals money for a third party this will not be addressed. Even then...
This is a big problem and we need to make everybody aware that they can protect themselves, and make them Liable for not taking those steps, before that happens
What exactly are you talking about? Interested to learn more.
Almost every developer outside the defense and aerospace sector is just stuffing code from internet randos into their binaries, JARs, and wheels. Just after they run this code on their developer machine that they keep their SSH keys on. It's a wonder/mystery/miracle we're not all hacked all day every day. The Rust and JS worlds are especially bad because somehow people got it into their heads that more dependencies are better. Why write 5 lines of your own code when you can use a dependency? (This is a real example of a discussion I've had at work)
Ah, I see. True. In my case I am looking forward to setting up a Linux workstation where I will severely limit random access to my system (and $HOME) via various means i.e. Flatpak and others. $HOME/.ssh is definitely first on the list.
But I agree that the out-of-the-box settings really make you wonder how we are not indeed hacked all day every day.
> What’s obvious to the audience of HN isn’t necessarily obvious to anyone else.
AI amplifies the problem.
Before AI, the idiot who accidentally has too much access probably doesn't have the ability to actively exploit it.
Given how much AI is being shoved down everybody's throats, an idiot with access now is an attack vector because they are have neither the ability nor desire to vet anything the AI is feeding to them.
> Before AI...
yes. Previously that idiot had to write code. Now it's the LLM that is directing invocation of these functions. And the LLM is a coding prodigy with the judgment of a 3 year old kid.
> Until that changes
MCP is just JSON RPC API dialect. It is not "safe" or "unsafe" by design - it's a layer where notion of safety (as talked about in the article) is not applicable. Saying that "MCP is unsafe" is not a meaningful statement in the scope of what MCP is. Nothing about any RPC (by itself) can guarantee that the remote system would do or not do something when some method is invoked.
Unless someone figures out a way to make end-user comprehensible language for formal software verification, so there could be an accompanying spec that describes the behavior to the dot, and technology that validates the implementation against the spec.
Right now the only spec is the actual codebase, and most users aren't typically reviewing that at all.
It isn't just an RPC spec, it's a use case. Specs never exist without a reason. I can assure you, a spec is a lot of work to write. You don't do it unless there is a purpose.
And with MCP, that purpose is fundamentally unsafe. It cannot be done safely without a different underlying technology. So yeah, it's pretty fair to say MCP is unsafe by design. It wouldn't exist except for that desire to create systems which cannot be secure.
MCPs are just fancy wrappers over rest apis, at least majority of the MCP servers out in the market are like that.
Having an MCP necessarily does not mean it is unsafe until and unless the company rushed execution and released something which it should not have in the first place.
Sorry, I don’t understand. My understanding of what MCP is - is that it’s just a standard on how to provide tools/functions to LLMs to call. What those functions are, what they do, and what else happens when they’re called is - in my understanding - out of scope of MCP. To me MCP is literally a plumbing between black boxes, and plumbing cannot do anything about the safety of what happens outside of it. So to me MCP is not “safe” or “unsafe” - those terms are meaningless for a plumbing (as long, of course, as plumbing itself doesn’t somehow explode or connect somewhere it’s not supposed to or stuff like that). Do you mean the purpose and/or scope are different somehow?
> My understanding of what MCP is - is that it’s just a standard on how to provide tools/functions to LLMs to call.
That's exactly the point the GP was making: this is a fundamentally unsafe idea. It's impossible to allow an LLM to automatically run tools in a safe way. So yes, MCP as means to enable this fundamentally unsafe use case is fundamentally unsafe.
My point is that the term “safe” is not a good one. In context of the article it erodes the scope of what MCP is and what it does (bringing way more stuff into the picture) and misattributes issue.
MCP safety is stuff like access controls, or lack of vulnerabilities on the protocol level (stuff like XML entity bombs). Software behavior of what MCP bridges together is not MCP anymore.
People discovering and running malware believing it’s legit is not a MCP problem. This line of thought is based on that “P” stands for “protocol” - MCP is interface, not implementation. And it’s humans who pick and connect programs (LLMs and MCP servers), not technology.
Not an XML bomb, but... at the protocol level, AFAIK, MCP allows the server to change its list of tools and what they do, and the description of what they do, at any point. The chatbot when it starts up, calls "tools/list" and gets a list of tools. So, if "approval for use" depends on that data, .. the approval is built on something that can shift at any time.
That, coupled with dynamic invocation driven by the LLM engine, seems like a security problem at the protocol level.
-----
Contrast to connecting an agent to a service that is defined by a spec. In this hypothetical world, I can tell my agent: "here's the spec and here's the endpoint that implements it". And then my agent will theoretically call only the methods in the spec. The endpoint may change, my spec won't.
I still need to trust the endpoint; that it's not compromised. Maybe I'm seeing a difference where there is none.
You’re correct. MCP is just a defined way of mapping string descriptions to functions.
I thought about it, and I think I know what the confusion could possibly be about.
To me, postmark-mcp is not a part of MCP, it’s a black box that talks MCP on one end. And its behavior is not an MCP but software trust and distribution issue, not specific to MCP (just like running any executables from random sources). I guess others may see differently.
Right but you have a good security posture and hygiene. MCP as a use case (not a protocol) is encouraging risky usage by less security minded people.
I was more referring to the environment that MCP servers run within, which at the end of the day is simply a massive interpolated string that blends instructions and data/content together in a way that an LLM will do its best to figure out a helpful response.
A 'perfectly secured' MCP server will become insecure simply by existing along neighbouring, insecure, tools. With that, I think it's safe to take a broader, less technical definition and say that you should use AI agents + MCP servers with caution.
Do people outside the HN audience exist that are using MCP servers?
Well, yes but they are not less nerd (actually, HN audience is not like it used to be, too)
> the best thing to do is to inform.
While also not using them yourself/actively trying to find and strip them out of workflows you have control over.
> Are people really this oblivious
Last I checked, basic SQL injection attacks were still causing massive economic damage every year and are at or near the top of the OWASP list. SQL injection is essentially the result of unintentionally giving god-mode permission to a database by failing to understand how queries are built and processed. The... agency available to AI agents might not be all that obvious either.
I am going to disagree with that. SQL injection attacks are an example of the age old issue of mixing up input and instructions. Smash the stack is older than many software devs, but it was essentially the same problem - its an inherit issue with Von Neumann architecture.
This is also not an AI issue, or even an MCP issue. If the same issue had been in a client library for the Postmark API, it would likely have had a bigger impact.
What we need is to make it much more likely to get caught and go to prison for stuff like this. That will change things.
> SQL injection attacks are an example of the age old issue of mixing up input and instructions.
Yes, and attacks on AI are much the same. The AI gets "prompted" by something that was supposed to be inert processable data. (Or its basic conduct guidelines are overridden because the system doesn't and can't distinguish between the "system prompt" and "user prompt".)
And we have decades of hindsight with sql injection to work with and make it obvious. No so much with all the fancy new AI tools.
Yes MCP has next to no security features, but then again is it even a year old at this point?
Not excusing it just pointing out something folks should me mindful of when using tool based on it, its an immature system.
And heck, I still remember a time when most of the internet traffic just flew around in plain text. Insanity to us now.
There was a time when email clients running scripts in emails from randos on the internet was "a game-changing opportunity for automation". Here we are again.
There was a time when people thought letting their children use the internet as a babysitter was healthier than parking them in front of the television... guess that turned out another way.
> Are people really this oblivious
There are hundreds of accidental gun deaths every year in the US. So yes, people do need to be told to not point guns at their feet (and other body parts) and pull the trigger.
https://ammo.com/articles/accidental-shooting-statistics
Just the other day I saw a video of a guy pulling a trigger on a gun in another person’s hand with the express goal of inuring them with the knock back. Another of a guy accidentally shooting a hole through his own ceiling. And a third one of a guy (there were at least three people present) shooting a rifle into a shallow river; they were lucky the barrel exploded like a looney tunes cartoon.
And we’re talking guns, which are designed to be dangerous. Software is much more nebulous and people have been conditioned for years to just download whatever and install it.
> Are people really this oblivious ...
Yes, yes they are. The vector of "I've been using this package and giving it full permissions for like forever and it has never been a problem." oblivious. One must understand the two-step here:
Step 1) Use someone else's package that solves your problem and doesn't have any issues. Yay you're doing DRY and saving time and effort!
Step 2) The package you got is now corrupted but you don't have any tools to checking to see if you're application is doing anything sus.
The alternative is that you audit every release and look at every change on every file. So suddenly your DRY because a weekly/monthly audit exercise for every single imported package that changes. Use three packages, that's three audit schedules. Use ten? That's ten audit schedules. It's stupid because if you actually used this alternative you'd be spending all of your time just auditing packages and 90+% of the time not finding anything to complain about.
So the ACTUAL alternative, is write your own damn code. You wrote it so you know how it works, and when you change it you know what you changed. And unless you are the bad guy, exploits don't "magically appear" in your code. Vulnerabilities will appear in your code if you don't do code reviews, but remember actualizing this stuff requires both knowing about the bug and exploiting it in the wild, you may end up coding in something that could be exploited but you are unlikely then to use your own vulnerability.
Step 1 sounds like npm, to me.
and Step 2.... ? yeah, we know.
"Write your own code" won't scale. Anyway that horse is out of the barn. We need better SBOM tools.
I disagree, writing your own code is the only thing that does scale. Even if you do nothing more than take the NPM package get source, and then do your own maintenance on it. But having been in different organizations that chose one path or the other, the durable organizations chose to write their own code.
The problem, especially with AI, IMO is that folks are even more willing to shoot themselves in the foot.
IMO this might be due to a few things like these:
1. Folks are trained on buffer overflows and SQL injections, they don't even question it, these are "bad". But an MCP interface to an API with god-like access to all data? The MCP of course will make sure its safe for them! (of course, it does not). It's learning and social issue rather than a technical issue. Sometimes, it makes me feel like we're all LLMs.
2. 300 things to fix, 100 vendors, and it needs to be done yesterday. As soon as you look into the vendor implementation you find 10 issues, because just like you, they have 300 things to fix and it needs to be done yesterday, so trade-offs become more dangerous. And who's going to go after you if you do not look into it too hard? No one right now.
3. Complete lack of oversight (and sometimes understanding). If you're just playing around by prompting an LLM you do not know what its doing. When it works, you ship. We've all seen it by now. Personally, I think this one could be improved by having a visual scheduler of tasks.
I think the issue is that the security issue is inherent to the core story being told about AI. “It can read your emails and respond automatically” is both a pretty compelling story to certain people who aren’t waist deep in tech news, and also exceedingly dangerous.
Did you know that if you give a loaded gun to a chimpanzee, sometimes it will shoot you (or itself) and it didn’t even know that was going to happen?!? Even if you tell it several times?!?
And that if that happens ‘smart’ people will tell you that it was really dumb to do that!!?!
Surely giving an unpredictable AI access to your system has absolutely no downsides at all.
The problem with foot guns is that you know it's a foot gun and I know it's a foot gun, but the underpaid social media manager at my bank might not and suddenly it's a problem for you and I.
This is unfair. It presumes a universal understanding of something largely because it is obvious to us. Most computer users have little to know detailed understanding of how any computer technology works, and because they are invisible and abstract, even less understanding of the risks they expose themselves to.
The answer to you gun analogy is false because it assumes basic knowledge of a gun. This is part of why so many kids shoot themselves or family members with guns - because they don’t know if you pull the trigger something violent will happen until they are taught it.
But these are not children, and they were not just handed a gun. They went out and acquired one.
That is what astounds me. How one can come into possession of a gun completely without understanding that it is dangerous. How fundamentally the worlds I and they live in must be for that to happen.
Oh, now that I write it out like that, I've definitely been on the other side of that astoundment before, for lacking 'common sense'. Ain't that just the way.
That's because people in general are good. They don't understand that there is a small fraction of humanity that would happily erase their digital lives given half a chance because they themselves can not even conceive of someone doing a thing like that.
The fact that we all can is a professional deformation, it is not normal. I lived in a place where the doors had no locks. Nobody could imagine anybody stealing from someone else's house. And so it just didn't happen. When buying the house there is the moment where the keys are transferred. The sellers somewhat sheepishly announced they didn't have any. The lawyer handling the transaction asked if they had lost their keys and if they did that they should pay for replacing the locks. Then it turned out they never had any locks in the first place and that this was pretty much the norm there.
That distrust that we call common sense is where we go wrong, the fact that we've connected the whole world means that these little assholes now have access to everything and everybody and there isn't even a good way to figure out who they are and how to punish them unless they are inept at hiding their traces.
When I was growing up, we did not have a key to lock our house. I was a "latch key kid", both parents worked, but I never had a key. I remember meeting someone who locked their front door as a matter of course, and thinking that was so curious.
What country was this? I've heard of leaving house unlocked but never no locks at all
Northern Canada.
People are doing this with MCP on large scale, it's not talked about enough; otherwise sensible people are dismissive of the risks.
What security people don't understand is that if insurance policy costs more than "cost of disaster" times "probability of a disaster" then it's most likely not worth it. They personally enjoy doing security stuff, so they don't internalize the costs of maintaining a secure environment the same way an average person does, or even a non-security developer.
Are people just aching to constantly blame poor people for moral and ethical failings, and not the grifters who prey on them?
Yes. People definitely blame victims all the time. There's no way LLMs, their proprietors, designers could be practicing dark patterns, marketing gimmicks and security shortcuts to convince people to shot these guns as often as possible...even when, they point them at their foot.
This grift is deep and wide and you may want to re-evaluate why everyone is so keen to always blame victims for technology, society, and business marketing.
It’s a bit easy for us to say, but general public doesn’t know about the details. For example, lots of people talk how algorithms designs your personal social media feed. But the same people don’t know what an “algorithm” means. It’s just a generic word for them. Which is fine, because not everyone is in our industry.
If you look at human civilization in general, it is full of cases where people need to be convinced that their bullet-wounded feet aren't "just the way it works", that fixing it is possible, and that the wounds are serious enough to be worth it.
Speak for yourself.
https://xkcd.com/1053/
Today, 10,000 Americans discovered that water is wet for the first time. Same thing tomorrow.
These also remind me on the early 1980s mentality of some who thought anything printed by a computer must be correct.
It’s an AI, it must be perfect! /s
[flagged]
Did you just desperately want to grind your political axe today? This has nothing to do with anything.
There's something deeply ingrained in the American psyche that says that warnings and guardrails are for other, stupid people. You, personally, are no doubt an ubermensch who can perform every procedure flawlessly from memory every time. If there were a crisis you would be a hero and save everyone. This applies equally to "good guy with a gun" and human factors analysis of software.
[flagged]
I'm sorry, we fundamentally disagree and there is no point in arguing as it is completely off-topic and our minds are made up. Your comment is unnecessarily inflammatory as is the OC's; the thread about MCP servers and their security model.