I can't believe that readers of HN actually think that that is how it does or should work.
Google/GCP can only make very general statements and in this case we want more than that.
They need to tell Railway and Railway needs to tell us, or Railway can tell us that Google is refusing to tell them.
Either way, we need to hear about this from Railway.
It's entirely plausible Google won't tell Railway without an NDA to prevent them disclosing exactly what set it all off.
The bigger point though is Google really need to flag any business account as not subject to these suspensions until checked into by several humans. Back when I had a team that used a lot of App Engine they would even call us when we caused all their pagers to go off, and then conspire to keep the lights on while things got fixed. It's sad they have ended up like this.
> It's entirely plausible Google won't tell Railway without an NDA to prevent them disclosing exactly what set it all off.
That case is called:
> or Railway can tell us that Google is refusing to tell them.
I'm not paying you (which let's face it, that's what an NDA is) just to find out why you messed up about as severely as one can imagine. In theory, there was a contract here: $ for cloud services, and the rug got pulled. One should get a very clear, and very apologetic explanation as to why, with no strings attached, or one should be voting with their wallet.
Now, whether Railway will do any of that, who knows.
They did:
"We take full responsibility for the architectural decisions that allowed a single upstream provider action to cascade into a platform-wide outage, and detail below what happened, how we recovered, and the changes we are making to prevent this from happening again."
My guess is they will be switching away from Google Cloud. Because anything else would be nuts.
I know hating GCP is hip, but why do only a minority entertains the possibility that Railway did really something off to trigger some alarms?
The calibration of the alarms might be off, and that's acceptable, but in the end if something can be held wrong, somebody will hold it wrong.
I did the same thing in the past, albeit in a much smaller scale. There's no shame in being wrong and admitting as long as it results in progress, so this stance of "we do nothing wrong" from both parties is getting a bit old now.
Of course it doesn’t work this way but why shouldn’t it? It doesn’t because Google is a massive company and could kill dozens of Railways before they notice an issue to their bottom line. However in a world where companies care I’d expect them to make a statement.
I took this a different way which was that to google railway is their customer and out of a variety of professional and security considerations want the communications to come from their customer and not them.
I think there’s been far too many Google/GCP ‘suspensions with no human in the loop’ that Google does need to put out a statement about their practices.
The only way is for PR to think they should open this
Google fucked up. They should convince people that it won't happen again.