EC2 outage...excellent customer service!

3 views
Skip to first unread message

Ross Cooney

unread,
Apr 7, 2008, 7:25:51 AM4/7/08
to AWS User group
Hi,

There was an EC2 outage this morning...it lasted around an hour and
effected around 25% of our instances. The issue lasted 56 minutes. The
issue was odd as it was not a full outage...just the public IP address
of the instances was unreachable...it was as if somebody messed with
the IP allocation tables :)

The outage didn't really effect us as we have redundancy with our
facility in Databanx (www.databanx.net)....but during the outage we
were very concerned.

The AWS support team were amazing during the outage....they provided
updates on their Forum every 15 minutes and had the issue resolved in
under an hour. You can see the posts here:
http://developer.amazonwebservices.com/connect/thread.jspa?threadID=20932&start=0&tstart=0

The Forum is proving a really good way to provide support to EC2
customers. It provides one place to look when you need help and also
allows you to check back on the status very easily.

Well done Amazon, excellent customer service,

Ross

Ian Tasker - Clariner.com

unread,
Apr 7, 2008, 8:33:51 AM4/7/08
to AWS User group
Hi,

It would be better if amazon gave a bit more information in regards to
issues.

With reference to your link.

It was 30 mins before they acknowledge the issue and 1 hours to put a
fix in place.
This is a quick fix but information as not forthcoming. Apart from the
2 post detailed above they were just posting
"we're looking at it".

There should be a status page of the aws services highlighting any
issues and affected instances.

Ian



On 7 Apr, 12:25, Ross Cooney <ross.coo...@gmail.com> wrote:
> Hi,
>
> There was an EC2 outage this morning...it lasted around an hour and
> effected around 25% of our instances. The issue lasted 56 minutes. The
> issue was odd as it was not a full outage...just the public IP address
> of the instances was unreachable...it was as if somebody messed with
> the IP allocation tables :)
>
> The outage didn't really effect us as we have redundancy with our
> facility in Databanx (www.databanx.net)....butduring the outage we
> were very concerned.
>
> The AWS support team were amazing during the outage....they provided
> updates on their Forum every 15 minutes and had the issue resolved in
> under an hour. You can see the posts here:http://developer.amazonwebservices.com/connect/thread.jspa?threadID=2...

jdbra...@gmail.com

unread,
Apr 7, 2008, 8:41:14 AM4/7/08
to AWS User group

On a similar topic - I came across the following interesting article
about "The mythical 99.99% uptime SLA" which I thought might amuse.
It was posted after AWS's first major outage earlier in the year.

http://dealarchitect.typepad.com/deal_architect/2008/02/the-mythical-99.html

He was a little light on the detail between planned and unplanned
outages ;-) But did make a valid point nonetheless.

jon


On Apr 7, 12:25 pm, Ross Cooney <ross.coo...@gmail.com> wrote:
> Hi,
>
> There was an EC2 outage this morning...it lasted around an hour and
> effected around 25% of our instances. The issue lasted 56 minutes. The
> issue was odd as it was not a full outage...just the public IP address
> of the instances was unreachable...it was as if somebody messed with
> the IP allocation tables :)
>
> The outage didn't really effect us as we have redundancy with our
> facility in Databanx (www.databanx.net)....butduring the outage we
> were very concerned.
>
> The AWS support team were amazing during the outage....they provided
> updates on their Forum every 15 minutes and had the issue resolved in
> under an hour. You can see the posts here:http://developer.amazonwebservices.com/connect/thread.jspa?threadID=2...

Ross Cooney

unread,
Apr 21, 2008, 4:08:39 AM4/21/08
to AWS User group
As you may have seen, last week Amazon announced a few commercial
support options for AWS services.

Over the past week I have been thinking about this and wondering
why....why release this service? The online forums have been used very
effectively to provide support to the tens of thousands of AWS users
in the past. Well, if you read into the statement from Jeff Barr
closely you will see the reason. He said:

“Increasingly, we see that organizations of all sizes are putting AWS
to use in new, innovative, and mission-critical ways. These
organizations have told us that they need a more direct and more
discreet way to request assistance and to report problems.”

The hint is the word “discrete”.

On April 7th there was a major outage of the EC2 service from AWS. For
some reason about 20% of the EC2 instances lost connectivity. A post
was placed on the AWS forum and within around one hour the issue was
resolved. At the time I was very impressed at the speed and efficiency
of the support….but one thing stuck in my mind….the whole episode was
very public. During the outage several developers posted information
about their problems, all of which were posted on a public forum and
still visible on the internet….something that really does not concern
me with my small business but would be completely unacceptable for an
enterprise.

A commercial support package gives enterprise users the ability to
report their problems and have them dealt with in a discrete way. Well
done, Amazon, for finally making Cloud Computing acceptable for the
enterprise.

Ross
(http://www.spoutingshite.com/index.php/2008/04/21/amazon-web-services-
gets-serious-about-enterprise/)

Alex Kavanagh

unread,
Apr 21, 2008, 4:52:22 AM4/21/08
to aws-use...@googlegroups.com

On Mon, 2008-04-21 at 01:08 -0700, Ross Cooney wrote:

>
> A commercial support package gives enterprise users the ability to
> report their problems and have them dealt with in a discrete way. Well
> done, Amazon, for finally making Cloud Computing acceptable for the
> enterprise.

Of course, on the downsize, they've just created the (inevitable?)
consequence: tiered support depending on how much you pay. I can only
guess who will get the priority now.

It also has the side effect of reducing public PR contests between
enterprises and Amazon. Maybe I'm just a cynic at this point. Perhaps
the whole point of EC2 was to woo the enterprises?


Reply all
Reply to author
Forward
0 new messages