OUTAGE ALERT: awe.sm Create and Stats APIs unavailable

9 views
Skip to first unread message

Jonathan Strauss

unread,
Apr 21, 2011, 2:28:25 PM4/21/11
to awes...@googlegroups.com
As you've probably heard (see http://www.techmeme.com/110421/p28#a110421p28), our hosting provider AWS has been experiencing issues since last night. Our team has been up all night battling to keep this extensive and prolonged AWS outage from affecting you, but the continuing AWS issues have now forced us to shut down our Create and Stats APIs as well as our web interface.

While you cannot currently create new links or retrieve stats, per our failover plans, redirections on existing links are not affected at this point and we will continue to do everything in our power to keep these links from breaking.

We will continue to update this list with more information as we have it.

Our sincere apologies for the disruption to your business and thanks for your patience,
-jonathan

--
Jonathan Strauss, Co-Founder
http://awe.sm

We're hiring! http://awe.sm/jobs

Blog: http://jonathanhstrauss.com
Twitter: http://twitter.com/jhstrauss

Jonathan Strauss [Staff]

unread,
Apr 21, 2011, 8:09:56 PM4/21/11
to awesm-api
We have just restored access to the Create APIs.

In order to minimize risk to the Create APIs, we are holding off on
bringing back Stats APIs until AWS is more stable. Until then, we are
working on serving properly formatted JSON error responses in the
Stats APIs.

-jonathan


On Apr 21, 11:28 am, Jonathan Strauss <jonat...@awe.sm> wrote:
> As you've probably heard (seehttp://www.techmeme.com/110421/p28#a110421p28), our hosting provider AWS has been experiencing issues since last night. Our team has been up all night battling to keep this extensive and prolonged AWS outage from affecting you, but the continuing AWS issues have now forced us to shut down our Create and Stats APIs as well as our web interface.
>
> While you cannot currently create new links or retrieve stats, per our failover plans, redirections on existing links are not affected at this point and we will continue to do everything in our power to keep these links from breaking.
>
> We will continue to update this list with more information as we have it.
>
> Our sincere apologies for the disruption to your business and thanks for your patience,
> -jonathan
>
> --
> Jonathan Strauss, Co-Founderhttp://awe.sm
>
> We're hiring!http://awe.sm/jobs

clawfire

unread,
Apr 22, 2011, 5:03:49 AM4/22/11
to awesm-api
hum AWS Guidelines recommand to make several replicating server on
different data-center to prevent that kind of outage O:-)

On Apr 21, 2:28 pm, Jonathan Strauss <jonat...@awe.sm> wrote:
> As you've probably heard (seehttp://www.techmeme.com/110421/p28#a110421p28), our hosting provider AWS has been experiencing issues since last night. Our team has been up all night battling to keep this extensive and prolonged AWS outage from affecting you, but the continuing AWS issues have now forced us to shut down our Create and Stats APIs as well as our web interface.
>
> While you cannot currently create new links or retrieve stats, per our failover plans, redirections on existing links are not affected at this point and we will continue to do everything in our power to keep these links from breaking.
>
> We will continue to update this list with more information as we have it.
>
> Our sincere apologies for the disruption to your business and thanks for your patience,
> -jonathan
>
> --
> Jonathan Strauss, Co-Founderhttp://awe.sm
>
> We're hiring!http://awe.sm/jobs

Vladimir Oane

unread,
Apr 22, 2011, 5:30:16 AM4/22/11
to awes...@googlegroups.com
They actually recommend different availability zones. But that wouldn't have helped here. 

PS: The good part is most of the API consumers are down too :) We are.

Best,

Vladimir OANE 
CEO uberVU
......................................................................
+44 753 21 50 976 
Skype:  vladimiroane
Twitter: ubervu
www.uberVU.com
......................................................................




--
You received this message because you are subscribed to the Google Groups "awesm-api" group.
To post to this group, send email to awes...@googlegroups.com.
To unsubscribe from this group, send email to awesm-api+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/awesm-api?hl=en.


Jonathan Strauss [Staff]

unread,
Apr 22, 2011, 12:24:54 PM4/22/11
to awesm-api
Our Stats APIs are now back online, as well as our Web Interface. New
click data is now being collected, processed, and returned in real-
time.

However, there is currently a gap in the data from April 21 @ ~11:30am
to April 22 @ ~8am PDT. We are beginning to reprocess that data and
hope to have it all restored in the next 12 hours.

With regard to AWS best practices, Vladimir is correct that Amazon's
recommendation of having servers in multiple availability zones (as we
do) did not save us from this outage (see
http://www.networkworld.com/news/2011/042111-amazon-ec2-zones.html).
The reason that it has taken so long for us to fully restore service
(beyond the fact that EBS is still down in our primary availability
zone) is that we needed to re-architect our system to span multiple
regions, which is a much bigger hurdle in the AWS system.

We can't apologize enough for the impact this may have had to your
business. While this was a 'black swan' type of outage that impacted
many companies, we take our responsibility to provide you with
reliable service as our own and we will be following up with a more in
depth post-mortem shortly detailing what steps we will be taking based
on what we learned from this outage.

Thank you again for your patience,
-jonathan & Team awe.sm
Reply all
Reply to author
Forward
0 new messages