https://aws.amazon.com/message/41926/
Unfortunately, one of the inputs to the command was entered incorrectly
and a larger set of servers was removed than intended.
So, basically "sudo rm -rf --no-preserve-root /" ;-)
S3 has experienced massive growth over the last several years and the
process of restarting these services and running the necessary safety
checks to validate the integrity of the metadata took longer than expected.
No periodic DR testing?
From the beginning of this event until 11:37AM PST, we were unable to
update the individual services’ status on the AWS Service Health
Dashboard (SHD) because of a dependency the SHD administration console
has on Amazon S3. Instead, we used the AWS Twitter feed (@AWSCloud) and
SHD banner text to communicate status until we were able to update the
individual services’ status on the SHD.
AWS infra should have backup capacity on GCP and Azure, just like
everyone else! :)
ilya