App Engine Down Now? 2:55pm PT
22 views
Skip to first unread message
Scott
Nov 9, 2009, 5:57:18 PM11/9/09
to Google App Engine
Hitting our app and getting 500 Server Error 100% of the time.
It is 2:55pm PT on Monday, 11/9.
Anybody else seeing issues?
It is 2:55pm PT on Monday, 11/9.
Anybody else seeing issues?
bvelasquez
Nov 9, 2009, 6:02:47 PM11/9/09
to Google App Engine
Scott
Nov 9, 2009, 6:04:01 PM11/9/09
to Google App Engine
osn-prod.appspot.com
www.onshortnotice.com
There are other apps down, but also looks like most apps from
appgallery.appspot.com are working.
www.onshortnotice.com
There are other apps down, but also looks like most apps from
appgallery.appspot.com are working.
Cynthia Kurtz
Nov 9, 2009, 6:08:10 PM11/9/09
to Google App Engine
Mine too, but only two out of three are broken. Very disconcerting, I
was just setting up a new group for somebody and it dropped out.
Grrrrrr. I wouldn't mind as much if I could catch an exception and
send people to a status page like I do for Timeouts.
Cynthia Kurtz
was just setting up a new group for somebody and it dropped out.
Grrrrrr. I wouldn't mind as much if I could catch an exception and
send people to a status page like I do for Timeouts.
Cynthia Kurtz
Scott
Nov 9, 2009, 6:10:21 PM11/9/09
to Google App Engine
Looks like the App Engine System Status is not based on reality. It
is showing 100% availability:
http://code.google.com/status/appengine/
is showing 100% availability:
http://code.google.com/status/appengine/
Tim Hoffman
Nov 9, 2009, 6:11:30 PM11/9/09
to Google App Engine
All of my apps are out too, with Error 500
however they are not DeadlineExceeded Errors
But
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details.
But I never have that much traffic, no traffic is showing in the
statistics, and no requests except for my test visits.
The other site that is failing no one even knows about. So the over
quota isn't possible.
Looks like the problem is pretty bad and widespread,
T
however they are not DeadlineExceeded Errors
But
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details.
But I never have that much traffic, no traffic is showing in the
statistics, and no requests except for my test visits.
The other site that is failing no one even knows about. So the over
quota isn't possible.
Looks like the problem is pretty bad and widespread,
T
Tim Hoffman
Nov 9, 2009, 6:16:43 PM11/9/09
to Google App Engine
Now the dahsboard is even returning Error 500.
Looks like the whole infrastructure is on the way out.
T
On Nov 10, 7:11 am, Tim Hoffman <zutes...@gmail.com> wrote:
> All of my apps are out too, with Error 500
> however they are not DeadlineExceeded Errors
>
> But
>
> Request was aborted after waiting too long to attempt to service your
> request. Most likely, this indicates that you have reached your
> simultaneous active request limit. This is almost always due to
> excessively high latency in your app. Please seehttp://code.google.com/appengine/docs/quotas.htmlfor more details.
Looks like the whole infrastructure is on the way out.
T
On Nov 10, 7:11 am, Tim Hoffman <zutes...@gmail.com> wrote:
> All of my apps are out too, with Error 500
> however they are not DeadlineExceeded Errors
>
> But
>
> Request was aborted after waiting too long to attempt to service your
> request. Most likely, this indicates that you have reached your
> simultaneous active request limit. This is almost always due to
bvelasquez
Nov 9, 2009, 6:25:09 PM11/9/09
to Google App Engine
"Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details."
I get this in the Log only under "Info." Same thing as a previous
http://code.google.com/appengine/docs/quotas.html for more details."
post..
Barry
On Nov 9, 3:16 pm, Tim Hoffman <zutes...@gmail.com> wrote:
> Now the dahsboard is even returning Error 500.
>
> Looks like the whole infrastructure is on the way out.
>
> T
>
> On Nov 10, 7:11 am, Tim Hoffman <zutes...@gmail.com> wrote:
>
> > All of my apps are out too, with Error 500
> > however they are not DeadlineExceeded Errors
>
> > But
>
> > Request was aborted after waiting too long to attempt to service your
> > request. Most likely, this indicates that you have reached your
> > simultaneous active request limit. This is almost always due to
bvelasquez
Nov 9, 2009, 6:27:39 PM11/9/09
to Google App Engine
"Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details."
I get this message also in "Info" within my Log. Hope the failover
http://code.google.com/appengine/docs/quotas.html for more details."
kicks in soon, I need to do some work.
> Now the dahsboard is even returning Error 500.
>
> Looks like the whole infrastructure is on the way out.
>
> T
>
> On Nov 10, 7:11 am, Tim Hoffman <zutes...@gmail.com> wrote:
>
> > All of my apps are out too, with Error 500
> > however they are not DeadlineExceeded Errors
>
> > But
>
> > Request was aborted after waiting too long to attempt to service your
> > request. Most likely, this indicates that you have reached your
> > simultaneous active request limit. This is almost always due to
> > excessively high latency in your app. Please seehttp://code.google.com/appengine/docs/quotas.htmlformore details.
>
> Looks like the whole infrastructure is on the way out.
>
> T
>
> On Nov 10, 7:11 am, Tim Hoffman <zutes...@gmail.com> wrote:
>
> > All of my apps are out too, with Error 500
> > however they are not DeadlineExceeded Errors
>
> > But
>
> > Request was aborted after waiting too long to attempt to service your
> > request. Most likely, this indicates that you have reached your
> > simultaneous active request limit. This is almost always due to
Tim Hoffman
Nov 9, 2009, 6:30:02 PM11/9/09
to Google App Engine
6 out of 7 of my apps are experiencing the same problem
All with the possibly over quota message, none of which have hardly
any traffic.
The logs are are all the same
11-09 03:20PM 33.146 / 500 10028ms 0cpu_ms 0kb
All getting cut off at around 10sec, and no cpu time used to date.
T
On Nov 10, 7:25 am, bvelasquez <bvelasq...@gmail.com> wrote:
> "Request was aborted after waiting too long to attempt to service your
> request. Most likely, this indicates that you have reached your
> simultaneous active request limit. This is almost always due to
> excessively high latency in your app. Please seehttp://code.google.com/appengine/docs/quotas.htmlfor more details."
All with the possibly over quota message, none of which have hardly
any traffic.
The logs are are all the same
11-09 03:20PM 33.146 / 500 10028ms 0cpu_ms 0kb
All getting cut off at around 10sec, and no cpu time used to date.
T
On Nov 10, 7:25 am, bvelasquez <bvelasq...@gmail.com> wrote:
> "Request was aborted after waiting too long to attempt to service your
> request. Most likely, this indicates that you have reached your
> simultaneous active request limit. This is almost always due to
Scott
Nov 9, 2009, 6:34:08 PM11/9/09
to Google App Engine
Anyone from the App Engine team monitoring this thread? Can we get a
status update? Our app is still down.
I am not seeing anything related to quota. Our app has billing
enabled. Failure rate for accessing our app is 100%
osn-prod.appspot.com
or
www.onshortnotice.com
Dashboard was down, now working.
-Scott
status update? Our app is still down.
I am not seeing anything related to quota. Our app has billing
enabled. Failure rate for accessing our app is 100%
osn-prod.appspot.com
or
www.onshortnotice.com
Dashboard was down, now working.
-Scott
rpalmer68
Nov 9, 2009, 6:37:59 PM11/9/09
to Google App Engine
Down here too...
"The server encountered an error and could not complete your request."
Richard
"The server encountered an error and could not complete your request."
Richard
HelperOfTheBride Team
Nov 9, 2009, 6:39:56 PM11/9/09
to Google App Engine
HelperOfTheBride Team
Nov 9, 2009, 6:37:51 PM11/9/09
to Google App Engine
We (http://www.helperofthebride.com/) are also experiencing the same
issue.
Users see this message, and the report just redirects them to the GAE
front page:
Error: Server Error
error message and the query that caused it.
issue.
Users see this message, and the report just redirects them to the GAE
front page:
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it.
Cynthia Kurtz
Nov 9, 2009, 6:41:18 PM11/9/09
to Google App Engine
Same error, same warning in logs, nowhere near quota. And it's such an
ugly error too.
ugly error too.
Ikai L (Google)
Nov 9, 2009, 6:50:58 PM11/9/09
to google-a...@googlegroups.com
Hey everybody,
We've been following this issue and we believe it consistently is affecting a small number of applications. We're working on resolving the issue for those of you that have been impacted as we speak. I'll send an update once we've resolved the issue.
Thanks for your patience,
Ikai Lan, Developer Programs Engineer, Google App Engine
rpalmer68
Nov 9, 2009, 6:53:46 PM11/9/09
to Google App Engine
Thanks for responding....
Sander D
Nov 9, 2009, 6:44:51 PM11/9/09
to Google App Engine
The same error occurs on www.pleft.com (contact me for the AppSpot
ID). Requesting a page gives:
"""
Error: Server Error
The server encountered an error and could not complete your request.
"""
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details.
"""
No actual quota were exceeded and visitor numbers are still low.
ID). Requesting a page gives:
"""
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it.
"""
There are a lot of warnings in the logs like:
error message and the query that caused it.
"""
"""
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details.
"""
Sander D
Nov 9, 2009, 6:48:58 PM11/9/09
to Google App Engine
The same error occurs on www.pleft.com (contact me for the AppSpot
ID). Requesting a page gives:
"""
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it.
"""
There are a lot of warnings in the logs like:
"""
ID). Requesting a page gives:
"""
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it.
"""
There are a lot of warnings in the logs like:
"""
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details.
"""
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details.
"""
No actual quota were exceeded and visitor numbers are still low.
[Sorry if this was sent twice.]
RollingCircle
Nov 9, 2009, 6:56:46 PM11/9/09
to Google App Engine
metutil.appspot.com suffering also - cannot even log into the admin
console.
Interestingly, if I view the console for one of my other apps
(ukairspace.appspot.com), and then swap applications to metutil via
the drop-down list, it works. Mostly.
I'm not sure if it's even in the offing, but it might be an idea not
to try and host the App Engine developer forums on App Engine....for a
while...or ever.
Cynthia Kurtz
Nov 9, 2009, 7:04:29 PM11/9/09
to Google App Engine
My affected app_ids are rakontu and rakontu-francais. rakontu-sandbox
is working.
is working.
Ben Wu
Nov 9, 2009, 7:04:27 PM11/9/09
to Google App Engine
Agreed -- good to see you chime in here, Ikai. Resolving the outage
is the most critical item for us, of course, but to pass along 2 other
items of feedback:
1) The error page says "If the error persists, please _report_ your
problem..." The link takes you to http://code.google.com/appengine/
where it's not at all clear how/where one should report the problem.
As this is our end-user facing page and the jumping point I tried
myself, it would be nice if that page actually provided a way to
report the problem.
2) http://code.google.com/status/appengine shows everything on App
Engine as being normal. Granted it may only be a small number of
applications affected, but it'd be helpful if you updated this page
nonetheless so we know that Google is aware of the issue rather than
needing to wait in the dark until somebody like you jumps into this
thread.
Thanks very much.
is the most critical item for us, of course, but to pass along 2 other
items of feedback:
1) The error page says "If the error persists, please _report_ your
problem..." The link takes you to http://code.google.com/appengine/
where it's not at all clear how/where one should report the problem.
As this is our end-user facing page and the jumping point I tried
myself, it would be nice if that page actually provided a way to
report the problem.
2) http://code.google.com/status/appengine shows everything on App
Engine as being normal. Granted it may only be a small number of
applications affected, but it'd be helpful if you updated this page
nonetheless so we know that Google is aware of the issue rather than
needing to wait in the dark until somebody like you jumps into this
thread.
Thanks very much.
Ben from He3
Nov 9, 2009, 7:21:37 PM11/9/09
to Google App Engine
I am having the same issue for my application id 'my-web-brain'
"Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it."
This appears not only when hitting the application direct but also
even attempting to log into the dashboard (before even nominating an
application).
"Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it."
even attempting to log into the dashboard (before even nominating an
application).
Message has been deleted
Message has been deleted
Ben Wu
Nov 9, 2009, 7:30:41 PM11/9/09
to Google App Engine
HelperOfTheBride Team
Nov 9, 2009, 7:32:24 PM11/9/09
to Google App Engine
We are up too.
bvelasquez
Nov 9, 2009, 7:35:30 PM11/9/09
to Google App Engine
Cynthia Kurtz
Nov 9, 2009, 7:40:47 PM11/9/09
to Google App Engine
Both of my apps that were broken (rakontu, rakontu-francais) are now
working again. Thanks!
But ... if we could have a better way to deal with this issue in
future it would be grand.
working again. Thanks!
But ... if we could have a better way to deal with this issue in
future it would be grand.
Ikai L (Google)
Nov 9, 2009, 9:09:37 PM11/9/09
to google-a...@googlegroups.com
App Engine developers:
I just wanted to let you all know that as of 4:40 PM Pacific Time, we resolved an issue with App Engine that caused a minority of our Python users to experience application outages that started at around 1:30 PM. The issue caused these applications to become aggressively cycled out, which in turn caused instances to end up in a thrashing state and suffer from thread starvation. This resulted in HTTP 500 errors being returned to end users.
We apologize for any inconvenience this issue may have caused.
Ikai Lan
Reply all
Reply to author
Forward
0 new messages