Minor point, but one of the complaints is a bit odd:
> curl -X POST https://backboard.railway.app/graphql/v2 \ -H "Authorization: Bearer [token]" \ -d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}' No confirmation step. No "type DELETE to confirm." No "this volume contains production data, are you sure?" No environment scoping. Nothing.
It's an API. Where would you type DELETE to confirm? Are there examples of REST-style APIs that implement a two-step confirmation for modifications? I would have thought such a check needs to be implemented on the client side prior to the API call.
Guys, did you bother checking the poster's profile? https://xcancel.com/lifeof_jer. SEE THE TWEET BELOW. Smells like a ragebait post to me. Also search online for his alleged "PocketOS" company with software for car rental businesses. I couldn't find anything on Google. (Of course, I might be wrong)
"The future of SEO is AIO" https://xcancel.com/lifeof_jer/status/2034409722624061772 March 18
There seems to be quite a lot of stuff here [1]
Seems legit to me. The oldest news item is from 2021. The domain name is new, but there seems to have been some rebranding lately. The product used to be called Pocket RentalOS and even that seems to be fairly recent rebranding [2]
[1] https://pocketos.ai/ [2] https://pocketos.ai/news/pocket-rebrands-its-luxury-rental-m...
Eh, it seems to be real, but all vibe coded.
https://pocketos.ai/
Interesting. Indeed there are some sketch stuff
I don't think this is a minor point. It seems clear by this point that the author is clueless how even API works and are just trying to shift blame for third-parties instead assuming that they're just vibecoding their whole product without doing proper checks.
Yes sure, there seems to be lots of ways this issue could have been mitigated, but as other comments said, this mostly happened because the author didn't do its proper homework about how the service they rely their whole product works.
It's also moot.
If the API replied "Are you sure (Y/N)?" the AI, in the mode it was in, guardrails completely pushed off the side of the road, it would have just said "Yes" anyway.
If you needed to make two API calls, one to stage the delete and the other to execute it (i.e. the "commit" phase), the AI would have looked up what it needed to do, and done that instead.
It's a privilege issue, not an execution issue.
Exactly, that just reinforces the fact that the author is just blaming others instead of getting any valuable insights about this "postmortem analysis".
He also seems to be lying, he wrote on Twitter the agent was in plan mode. That part has to be exaggerated.
“Plan” vs “execute” modes seem more like suggestions the models _mostly_ follow. I have absolutely had models (Codex and Sonnet/Opus) perform actions in plan mode they should never have been able to take like editing files or starting to work on a plan that was just created.
I can’t say for sure, but I think Claude’s mode is nothing more than part of the system prompt. I don’t think it actually takes away web request or file write tools. I say this because I could swear I’ve seen Claude go ahead and make some changes even while we’re in plan mode. Web requests certainly, because it can fetch docs and so forth.
You’re not alone, I’ve absolutely seen the same behavior occasionally with Opus in OpenCode where it takes actions it shouldn’t be able to in plan mode.
It's not common, but I've personally built APIs where requests for dangerous modifications like this perform a dry run, giving in the response the resources that would be deleted/changed and a random token, which then needs to be provide to actually make the change. The idea was that this would be presented in the UI for the user to confirm, but it should be as useful or more by AI agents. Also, you get the benefit that the token only approves that particular modification operation, so if the resources change in between, you need to reapprove.
I guess we don’t know what the agent would do after seeing these warnings and a request for extra action.
Perhaps it would stop and rethink, perhaps it would focus on the fact that extra action is needed - and perform that automatically.
I suppose the decision would depend on multiple factors too (model, prompt, constraints).
Measure twice cut once seems to be forgotten these days.
As well as: A computer can never be held accountable
The stupidity of people sinks to new lows every day. It's astonishing just how ignorant people are of table stakes, basic technological concepts.
You just gave an AI destructive write access to your production environment? Your production DB got dropped? Good. That's not the AI's fault, that's yours, for not having sensible access control policies and not observing principle of least privilege.
AWS actually has a thingy on some services called “deletion protection” to prevent automation from accidentally wiping resources the user didn’t want it to (you set the bit, and then you need to make a separate api request to flip the bit back before continuing).
I think it’s designed for things like Terraform or CloudFormation where you might not realize the state machine decided your database needed to be replaced until it’s too late.
And then, someone added IAM so you could actually restrict your credentials from deleting your database.
First mistake is to use root credentials anyway for Terraform/automated API.
Second mistake is to not have any kind of deletion protection enabled on criticsl resources.
Third mistake is to ignore the 3-2-1 rule for backups. Where is your logically decoupled backup you could restore?
I am really sorry for their losss, but I do have close to zero empathy if you do not even try to understand the products you're using and just blindly trust the provider with all your critical data without any form of assessment.
GCP Cloud SQL has the same deletion protection feature, but it also has a feature where if you delete the database, it doesn't delete backups for a certain period of days. If someone is reading this and uses Cloud SQL, I highly suggest you go make sure that check box is checked.
Agents will happily automate away intentional friction like a confirm prompt, even if you organise it as multiple API calls.
The fix needs to be permissions rather than ergonomics.
There's also a cooldown period on some deletes (like secrets) to make sure you don't accidentally brick something
This should be the solution. All destructive actions require human intervention.
If we take that literally, then just remove all destructive API endpoints. Because then, it they no real purpose, you cannot automate the removal of anything.
I think some other suggestions are saner (cool-down period, more fine-grain permissions, delete protection for certain high-value volumes). I don't think "don't allow destructive actions over the API" is the right boundary.
A human representing the company should be physically present in the provider's office to perform such an action or what? Otherwise you would just grant your agent a way to impersonate a human.
I agree that this is the author’s fault considerably more than it is Railway’s, however I have learned from experience that no matter how many “are you sure you want to do this” prompts you have, sometimes users delete stuff they didn’t intend to delete and it’s better to not delete immediately but put it in a queue for deletion in a few hours and offer a way to reverse it. Even if it’s 100% user error, the user is very happy they didn’t lose data and the cost of storing it for an extra 5 hours or so is tiny.
Funny how he points the finger at everyone but himself.
the kind of attitude you really need to get your agents to delete your prod lol
User is an idiot for using AI Agent. But I am not saying that it is not also badly designed system. Soft delete or something like should be standard for this type of operations. And any operator should know well enough to enable it for production.
> Are there examples of REST-style APIs that implement a two-step confirmation for modifications?
A pattern I've seen and used for merging common entities together has a sort of two-step confirmation: the first request takes in IDs of the entities to merge and returns a list of objects that would be affected by the merge, and a mergeJobId. Then a separate request is required to actually execute that mergeJob.
IMO the fail here is not having a true soft delete policy with a delete endpoint available
You need to protect customers from themselves. If you offer a true deletion endpoint/service you need to offer them a way to stop them from being absolute idiots when they inevitably cause a sev 0 for themselves.
This is kind of a stretch, but especially if there were multiple operations beyond the "volumeDelete", the GraphQL definitely worsens readability here.
For someone reviewing and approving LLM calls or just double-checking before running a script or bash history, it would be a lot more readable if it were compliant with HTTP norms: curl -X DELETE example.com/api/volumes/uuid123 would make it very obvious that something was going to be deleted at the front and then what it is at the end of the command.
I suppose could implement it by requiring a deletion token that is returned when making a deletion request which doesn't have its own deletion token, but why would you? That's something for the frontend to handle.
In AWS eg. bucket can be deleted only when empty. Deleting all files first is your confirmation.
> In AWS eg. bucket can be deleted only when empty. Deleting all files first is your confirmation.
That wouldn't have helped in this case - the agent made a decision to delete, so if necessary it would have deleted all the files first before continuing.
The question that comes to mind is "how are people this clueless about LLM capabilities actually managing to rise to be the head of a technology company?"
The first delete would fail: “bucket not empty”. This might make the agent question the deletion (“bucket should be empty”).
> The first delete would fail: “bucket not empty”. This might make the agent question the deletion (“bucket should be empty”).
This is actually not a bad test case for evaluating an LLM: give it a workflow that has an edge case requiring deletion, then prevent that deletion, and see if it:
a) Backtracks on the decision to delete, or
b) Looks for an alternative way to delete.
Yeah, I've run tests similar to this while evaluating gpt 5.4 vs claude 4.6
Claude is more likely to figure out workarounds and get things deleted if I tell it to delete stuff, so it performs much better in this benchmark and I prefer it.
GPT is more likely to stop and prompt you "I got an error deleting this, should I try another way?", and since the operator gets more of these prompts, they'll hit continue more withut even reading it, so it ends up being more annoying for the operator and not really reducing the chance of it happening imo.
If your workflow for your llm says "delete the ec2-instance", and the ec2 api gives back "deletion protection is on", I want my llm to turn off deletion protection and delete it.
I feel like you're implying that the reverse result, prompting the user, is better, but I disagree with that.
How are people still deluded enough about this economic system to believe rank implies competence?
I've sometimes seen a variable like "areyousure" which needs to be set to true. Sometimes there's a force flag. And "agree to eula" fields are somewhat common.
agreed — confirmation belongs on the client side. but the harder question is "what is a client-side check when the client IS an llm agent?" a polite "are you sure?" doesn't bind a probabilistic generator that's motivated to finish the task. the version that actually works: declare the agent's allowed actions in a parsed config that's validated BEFORE the action is emitted. destructive verbs require the operator to approve a diff to that config first. still client-side — but the check isstructural, not behavioral. ended up doing this in bluewave (multi-tenant agent runtime) — explicit @scope and @rules blocks in a parsed .ssl spec, validated before each cycle. the agent literally cannot emit an action outside the declared scope. spec is open at github.com/Galmanus/ssl-spec — mit.
Some S3 APIs have 2FA options for drastic operations (delete for versioned buckets where you probably don't want deletes much) https://docs.aws.amazon.com/AmazonS3/latest/userguide/MultiF...
AWS has deletion protection for databases, and you have to make a separate call to disable it first. Deletion is rejected if you don’t disable that protection.
Assuming the API has some secret spot to write DELETE, wouldn't the chatbot just send DELETE and make the protection only delay the disaster for 10 seconds?
You won’t, but API implementation can and should mark a volume as pending deletion and keep it for a while. Like AWS does with keys and some other things.
The whole tweet is AI slop, I doubt the human hitting "post" read through it all that closely. If they did, maybe they'd also go "Wait, that's nonsense".
Yes! I wish more people called that out as well! Has anyone even verified the validity of this claim?
I have to agree here...of all things that went wrong here, I don't think the API surface is to blame. You need to have deterministic control & escalation mechanism on your agents whether they are calling an API or any other tool
He (or ChatGPT) is throwing spaghetti at the wall. Not having the standard API key be able to delete the database (and backups) in one call makes sense. "Wanting a human to type DELETE as part of a delete API call" does not.
In the user interface for Railway, all destructive actions require multiple confirmations, plus typing "apply destructive changes". Why would an API key (regardless of its scope) be able to delete without confirmation?
> Why would an API key (regardless of its scope) be able to delete without confirmation?
What do you think an API is for? There's no user sitting at the keyboard when an API is called so where would that confirmation come from? It can't come from the user because there is no user.
Isn’t the point of an API to have two computers talk to each other? As in “if I want safeguards for humans, it would be my responsability to put them BEFORE calling that API”?
> Why would an API key (regardless of its scope) be able to delete without confirmation?
How do you see this working? Any confirmation would be given by the agent.
... because that's how every other cloud provider API works? the AWS console makes you confirm before deleting a bucket; DeleteBucket does not
I read this as "the agent should have asked for confirmation before running".
Me too. The line before the curl command says the agent ran the command, so it could be that the next paragraph is complaining that the agent didn't ask for confirmation.
This person is a card-carrying moron and has no idea how anything works. Even if we concede that maybe there should be some grace period or soft deletions or whatever..
Also, the post is 100% written by an LLM, which is ironic enough on its own. But that then makes it a bit more curious that you find this argument in this slop, because any LLM would say so. But if you badger it enough, it will concede to your demands, so you just know this clown was yelling at his LLM while writing this post.
He really should've thrown this post at a fresh session and asked for an honest, critical review.
[dead]