The footnote which says that is the design durability against equipment failure literally begins:
> In the unlikely case of the loss or damage to all or part of an AWS Availability Zone, data in a One Zone storage class may be lost. For example, events like fire and water damage could result in data loss
I'm going to reword my Terms of Service this second to add "any military operation" next to "acts of war". But I'm sure we'll then have to demonstrate whether paramilitary are assimilated to the military.
> US refuses payout for soldiers who died in Iran 'because it isn't a war'
> In response to another question, Mr Vance rejected using the word “war” to characterise US operations in Iran, saying there was “no active shooting”.
If a senior leader at Doordash swore to God that your sandwich would 100% guaranteed make it to you, regardless of whether or not there was a car crash, then yeah, maybe you should complain
Yeah I agree with you. If they made a specific promise to some unlikely case out of their control, then I would expect compensation.
Did Amazon make such a promise? They didn't as far as I know. My understanding is they provide specific guarantees like given an AZ outage, your data is still safe (provided you architect correctly).
Somehow I feel like the biggest post-apocalyptic problem will be the loss of home equity due to uninsured damage causing a collapse of financial markets.
Are you saying that most data loss happens because your data center gets blown up in a shooting war? Like, AWS is the first digital service provider to lose data in decades?
I'm saying that if you have eliminated more mundane failures like dying harddrives, cosmic rays and so on from your systems and your calculation ends up with 11 nines then actually those "force majeure" events are probable enough that they dominate whatever other residuals are supposedly hiding in those last 0.0000000001%.
The region has seen a bunch of wars in the last 100 years, so the annual war-rate is > 1%. Even if we generously add the assumption that only 1 in 100 wars affects a datacenter you can see that wars become a major source of correlated hardware failures that they need to solve to actually deliver that kind of reliability.
You don’t want to blend probabilities like this, because the tactics you use as a consumer vary between the two. If you consider 11 9s like “object AFR”, you might build systems that are resilient to very occasional single object loss. And it’s useful to know at what rate that might occur.
Whereas with these force majeure events you’d want a complete DR setup, and it’s typically an async recovery. Here it is useful to understand the fault domain (single server or single building or multi-building) so you can plan.
Blending the two numbers doesn’t help you build better against the systems. And the force majeure events are rare enough that they won’t happen … until they do. I’m not sure that knowing the precise probability that Iran would attack a gulf nation would change the fact that if they do, you need to have a DR story.
The sales pitch should change from "probabilistically we will NEVER lose your data" to "you are most likely to lose your data due to wars, terrorists, software bugs, someone losing the master encryption key, the government forcing us to...".
Offsite backups are sadly rare these days, and aws sales is the main reason why.
Creative accounting works and is good because it works. If your customers give you more money because you lied to them, but it's legal, then it's good.
This is very on point. While I am not legally trained in US law, and especially not in any Middle Eastern law, I have a enough knowledge on the law in my country here where I live…
Invoking force majeure requires the entity to prove all three following to be true:
A. That the event was unexpected and therefore unavoidable.
B. That the event was outside the control of the entity.
C. That the event made it impossible for the company to resolve the issue.
War in the region is as you say rather common unfortunately. The fact that AWS is used by the IDF (https://www.972mag.com/cloud-israeli-army-gaza-amazon-google...) should be considered a factor whether or not their data centre became a more likely target or not. What remains is the ability or not for AWS to do multi-location reduncancy.
But if they want to design for extreme probabilities you need to account for tail risks, so their design should have included a missile defense system.
At some point you need to start worrying about asteroid defense too.
Hot take: if you have a service that is 11 nines reliable, but there is an underlying component whose reliability is lower, cap the nines to that component.
The footnote which says that is the design durability against equipment failure literally begins:
> In the unlikely case of the loss or damage to all or part of an AWS Availability Zone, data in a One Zone storage class may be lost. For example, events like fire and water damage could result in data loss
Even if they have payable SLA on this, most SLAs have Acts of God and Acts of War exemption.
But do they have Act of Special Operation exemptions?
I'm going to reword my Terms of Service this second to add "any military operation" next to "acts of war". But I'm sure we'll then have to demonstrate whether paramilitary are assimilated to the military.
> US refuses payout for soldiers who died in Iran 'because it isn't a war'
> In response to another question, Mr Vance rejected using the word “war” to characterise US operations in Iran, saying there was “no active shooting”.
https://news.ycombinator.com/item?id=49596955
It seems unreasonable to blame Amazon here. The AZ was destroyed. Are they supposed to have missile/drone defense?
I'm not going to complain to DoorDash if my order is delayed due to a car crash
If a senior leader at Doordash swore to God that your sandwich would 100% guaranteed make it to you, regardless of whether or not there was a car crash, then yeah, maybe you should complain
Yeah I agree with you. If they made a specific promise to some unlikely case out of their control, then I would expect compensation.
Did Amazon make such a promise? They didn't as far as I know. My understanding is they provide specific guarantees like given an AZ outage, your data is still safe (provided you architect correctly).
The SLA excludes force majeure.
Making a probabilistic claim while excluding a factor that dominates those statistics is... is quite creative accounting.
Force majeure carveouts are really common in every type of contract.
You should check your home insurance contract, for instance... It likely would not cover an ICBM strike.
Somehow I feel like the biggest post-apocalyptic problem will be the loss of home equity due to uninsured damage causing a collapse of financial markets.
Are you saying that most data loss happens because your data center gets blown up in a shooting war? Like, AWS is the first digital service provider to lose data in decades?
I'm saying that if you have eliminated more mundane failures like dying harddrives, cosmic rays and so on from your systems and your calculation ends up with 11 nines then actually those "force majeure" events are probable enough that they dominate whatever other residuals are supposedly hiding in those last 0.0000000001%.
The region has seen a bunch of wars in the last 100 years, so the annual war-rate is > 1%. Even if we generously add the assumption that only 1 in 100 wars affects a datacenter you can see that wars become a major source of correlated hardware failures that they need to solve to actually deliver that kind of reliability.
You don’t want to blend probabilities like this, because the tactics you use as a consumer vary between the two. If you consider 11 9s like “object AFR”, you might build systems that are resilient to very occasional single object loss. And it’s useful to know at what rate that might occur.
Whereas with these force majeure events you’d want a complete DR setup, and it’s typically an async recovery. Here it is useful to understand the fault domain (single server or single building or multi-building) so you can plan.
Blending the two numbers doesn’t help you build better against the systems. And the force majeure events are rare enough that they won’t happen … until they do. I’m not sure that knowing the precise probability that Iran would attack a gulf nation would change the fact that if they do, you need to have a DR story.
Cosmic rays and dying hard drives are not force majeure though.
I think it's what most people comparing provider SLAs would expect
Are you suggesting that their technical documents have separate availability numbers to predict geopolitical events and war?
The sales pitch should change from "probabilistically we will NEVER lose your data" to "you are most likely to lose your data due to wars, terrorists, software bugs, someone losing the master encryption key, the government forcing us to...".
Offsite backups are sadly rare these days, and aws sales is the main reason why.
Creative accounting works and is good because it works. If your customers give you more money because you lied to them, but it's legal, then it's good.
Excluding war as force majeure in the Middle East is the same as excluding high tide as force majeure building a sandcastle at low tide.
This is very on point. While I am not legally trained in US law, and especially not in any Middle Eastern law, I have a enough knowledge on the law in my country here where I live…
Invoking force majeure requires the entity to prove all three following to be true:
A. That the event was unexpected and therefore unavoidable.
B. That the event was outside the control of the entity.
C. That the event made it impossible for the company to resolve the issue.
War in the region is as you say rather common unfortunately. The fact that AWS is used by the IDF (https://www.972mag.com/cloud-israeli-army-gaza-amazon-google...) should be considered a factor whether or not their data centre became a more likely target or not. What remains is the ability or not for AWS to do multi-location reduncancy.
This
If they haven't changed it recently, 11 nines is the durability target by design, but it is not set in any SLA. the S3 SLA is focused on availability.
I don't think this has any teeth. They don't compensate in the event of loss afaict.
Considering all the data they have globally, they might still be compliant.
They say it's "designed for" 11 9s, not guaranteed.
But if they want to design for extreme probabilities you need to account for tail risks, so their design should have included a missile defense system. At some point you need to start worrying about asteroid defense too.
Ah so that's why we need a lunar base. To uphold S3's 11 9s of availability
Hot take: if you have a service that is 11 nines reliable, but there is an underlying component whose reliability is lower, cap the nines to that component.