Google App Engine Downtime Notify Google Group

App Engine outage last night

2008-10-01T17:43:17Z

Yesterday evening between roughly 4:15 and 5pm PDT (GMT-7) we
experienced an outage during which a significant percentage of
requests to some apps resulted in spurious quota denials. All apps
are now functioning smoothly, and we're investigating what caused
this.

Pete Koomen, App Engine Team

App Engine outage tonight

2008-09-30T03:40:07Z

This evening at around 7:20pm PDT (GMT-7) we experienced a 15 minute
outage during which a large percentage of requests resulted in
spurious quota denials. This issue has been isolated and fixed, and
all apps are currently functioning smoothly.

Pete Koomen, App Engine Team

Re: Brief index-building outage tonight

2008-09-24T19:56:48Z

On closer inspection, this issue was not resolved last night. As of
12:45pm PDT (GMT-7) today, app deployments are functioning smoothly.
If you tried to deploy an app that required new indexes (or tried to
vacuum old indexes) at any point between 9:50pm last night and 12:45pm
today, you would have seen a 500 error--this has now been fixed, and

Brief index-building outage tonight

2008-09-24T06:37:02Z

Tonight beginning at around 9:50pm PDT (GMT-7) apps with new indexes
were unable to deploy and apps were unable to vacuum old indexes. The
issue was resolved at 10:30pm PDT, and all app deployments are
functioning smoothly again.

Thanks,
Pete Koomen, App Engine Team

Re: Increase in mail API error rates

2008-09-16T14:28:08Z

This issue was resolved completely at 4pm PDT (GMT-7) yesterday. The
mail API is now functioning normally.

Pete Koomen, App Engine Team

On Sep 15, 1:35 pm, App Engine Team <appengine.nore...@gmail.com>
wrote:

Re: Datastore writes experiencing errors and increased latency

2008-09-16T08:34:23Z

This issue has been addressed. Datastore writes are back to normal as
of 1:30am PDT.

Datastore writes experiencing errors and increased latency

2008-09-16T07:51:54Z

09/15/2008 11:30PM PST-PRESENT [ONGOING]:
Partial System Outage.
Percentage of Applications: All
Datastore writes for all applications are experiencing elevated
latencies and error-rates. Engineering is currently troubleshooting
the issue.
Expected Next Update: 1hr

Re: Increase in mail API error rates

2008-09-15T17:35:44Z

Correction: this issue began at 6am PDT (GMT-7)

Pete Koomen, App Engine Team

On Sep 15, 1:26 pm, App Engine Team <appengine.nore...@gmail.com>
wrote:

Increase in mail API error rates

2008-09-15T17:26:50Z

Today at roughly 9am PDT (GMT-7) around 10% of
google.appengine.api.mail.send _mail() calls started returning errors.
We're currently investigating this issue, although you can handle
these errors using Python's try syntax like this:

from google.appengine.api import mail
from google.appengine.runtime.apipr oxy_errors import

Datastore latency increase for small percentage of apps today (20 minutes)

2008-09-11T05:50:30Z

This morning between 9:25 and 9:45am PDT (GMT-7) a very small
percentage of apps experienced an increase in datastore latency, which
may have resulted in an increase in datastore timeouts for affected
apps. We're working on identififying the issue that caused this, and
note that some apps may have experienced this issue to a lesser degree

Re: Scheduled Maintenance on Friday 8/29 at 5pm PDT (GMT-7)

2008-08-30T00:23:10Z

This maintenance is beginning now. The read-only time for any given
application should last between 10 and 20 minutes.

Pete Koomen, App Engine Team

On Aug 29, 10:06 am, App Engine Team <appengine.nore...@gmail.com>
wrote:

Re: Scheduled Maintenance on Friday 8/29 at 5pm PDT (GMT-7)

2008-08-30T00:04:15Z

This maintenance is beginning now. The read-only time for any given
application should last between 10 and 20 minutes.

Pete Koomen, App Engine Team

On Aug 29, 10:06 am, App Engine Team <appengine.nore...@gmail.com>
wrote:

Scheduled Maintenance on Friday 8/29 at 5pm PDT (GMT-7)

2008-08-29T17:06:52Z

On Friday, August 29th (today), App Engine will undergo scheduled
maintenance at 5pm PDT (GMT-7) for approximately 15 minutes. As a
result, during this period all App Engine apps will revert to read-
only mode, and all datastore writes and transactions will throw an
exception. If you'd like to modify your app to catch this exception,

Re: Scheduled Maintenance on Wednesday 9/27 at 5pm PDT (GMT-7)

2008-08-28T00:32:40Z

This was completed at 5:28pm PDT (GMT-7). All applications are
functioning normally.

Pete Koomen, App Engine Team

On Aug 27, 5:02 pm, App Engine Team <appengine.nore...@gmail.com>
wrote:

Re: Scheduled Maintenance on Wednesday 9/27 at 5pm PDT (GMT-7)

2008-08-28T00:02:22Z

This maintenance is beginning now.

Pete Koomen, App Engine Team

On Aug 26, 9:31 am, App Engine Team <appengine.nore...@gmail.com>
wrote: