appengine.google.com returns 500 error

[Sundar Raman]

Since around 2pm CST appengine.google.com keeps giving me:

Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it.

Basically this means I can't get to my dashboard, can't create new apps, and can't really do anything administrative. My apps seem to be running fine, but it's hard to say since I can't get to my logs!

[Sundar Raman]

[Daniel Upton]

My admin console is mostly returning 500 errors as well. When it
loads it is quite slow.

[Ikai Lan (Google)]

I'm seeing this intermittently. We're looking into it.

--

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blogger: http://googleappengine.blogspot.com

Reddit: http://www.reddit.com/r/appengine

Twitter: http://twitter.com/app_engine

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[coltsith]

I'm getting 500 errors too. Exactly on the day that I needed to use it
for a promotional event no less....

Honestly, I'm so close to switching to Amazon Web Services that it's
not even funny. Unless Google cleans up their act we will not be using
GAE for our next commercial application.

[dloomer]

Same here.

 

I think the larger issue is the reliability and reusability of the admin console over the past few weeks (possibly longer).  If my app performed like that, I don't know how many users I'd have left.  And given that I believe the console itself is written in GAE, it makes me a little unsure about the platform I've chosen.  I can't remember the last time it was consistently snappy and reliable every time, but I know that it was at one time.  I hope someone is doing something to address this.

[mathijs]

Lots of 500s here too.
Watchmouse (which IMO is much more reliable and matches my 500-
encounters much much better than the official AppEngine dashboard,
which always says that everything's ok) also
shows downtime: http://api-status.com/6404/117406/Google-App-Engine-API

Our pushes are also failing all the time now with 500s. Any official
reaction?

[dloomer]

(correction: "usability", not "reusability").

[dloomer]

Looks like for some of us, this isn't intermittent: it happens 100% of the time :)

[Francois Masurel]

Yep can't connect at all since a few minutes.

[Sundar Raman]

It's 100% unavailable for me at present.

On Wed, Dec 1, 2010 at 2:44 PM, dloomer <dlo...@gmail.com> wrote:

Looks like for some of us, this isn't intermittent: it happens 100% of the time :)

--

[Amir]

It's been down for quite some time now.  Any update?

[Andy Kim]

I'm having the same problem as well. Both my apps and its dashboard are returning 500 error. I haven't changed anything with the app in a about a week or so.

[vivpuri]

DeadlineExceededError all over the app.

[Ron Yust]

Of course I can't deploy right when I am having a production issue and need to get a new release out NOW... the timing for Google's periodic problems couldn't have happened at a worse time.

[datapharmer]

I am on the east coast and am experiencing this 100% of the time today. Perhaps http://code.google.com/status/appengine coule be updated to reflect that this isn't exactly working. api-status.com seems to have more accurate data.

some test responses:

$ curl -I http://appengine.google.com/

HTTP/1.1 500 Internal Server Error

Date: Wed, 01 Dec 2010 19:57:08 GMT

Content-Type: text/html; charset=UTF-8

Server: Google Frontend

Content-Length: 466

$ ping appengine.google.com

PING www3.l.google.com (74.125.45.138): 56 data bytes

64 bytes from 74.125.45.138: icmp_seq=0 ttl=56 time=33.117 ms

64 bytes from 74.125.45.138: icmp_seq=1 ttl=55 time=33.368 ms

64 bytes from 74.125.45.138: icmp_seq=2 ttl=55 time=32.445 ms

64 bytes from 74.125.45.138: icmp_seq=3 ttl=56 time=38.425 ms

64 bytes from 74.125.45.138: icmp_seq=4 ttl=56 time=32.204 ms

64 bytes from 74.125.45.138: icmp_seq=5 ttl=56 time=30.685 ms

64 bytes from 74.125.45.138: icmp_seq=6 ttl=56 time=36.050 ms

64 bytes from 74.125.45.138: icmp_seq=7 ttl=56 time=32.040 ms

64 bytes from 74.125.45.138: icmp_seq=8 ttl=56 time=36.140 ms

64 bytes from 74.125.45.138: icmp_seq=9 ttl=56 time=30.773 ms

--- www3.l.google.com ping statistics ---

10 packets transmitted, 10 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 30.685/33.525/38.425/2.412 ms

tcpdump -v -i en1 -ttttt host appengine.google.com

tcpdump: listening on en1, link-type EN10MB (Ethernet), capture size 65535 bytes

00:00:00.000000 IP (tos 0x0, ttl 64, id 4441, offset 0, flags [DF], proto TCP (6), length 1122)

    192.168.2.148.54485 > yx-in-f101.1e100.net.http: Flags [P.], cksum 0xb5da (correct), seq 1191499719:1191500789, ack 1198268946, win 33323, options [nop,nop,TS val 792741414 ecr 4231782916], length 1070

00:00:00.020660 IP (tos 0x0, ttl 56, id 10974, offset 0, flags [none], proto TCP (6), length 52)

    yx-in-f101.1e100.net.http > 192.168.2.148.54485: Flags [.], cksum 0x40f9 (correct), ack 1070, win 155, options [nop,nop,TS val 4231800521 ecr 792741414], length 0

00:00:00.078026 IP (tos 0x0, ttl 56, id 10975, offset 0, flags [none], proto TCP (6), length 679)

    yx-in-f101.1e100.net.http > 192.168.2.148.54485: Flags [P.], cksum 0x0264 (correct), seq 1:628, ack 1070, win 155, options [nop,nop,TS val 4231800577 ecr 792741414], length 627

00:00:00.078083 IP (tos 0x0, ttl 64, id 34894, offset 0, flags [DF], proto TCP (6), length 52)

    192.168.2.148.54485 > yx-in-f101.1e100.net.http: Flags [.], cksum 0xbdf6 (correct), ack 628, win 33009, options [nop,nop,TS val 792741415 ecr 4231800577], length 0

00:00:03.153283 IP (tos 0x0, ttl 56, id 23210, offset 0, flags [none], proto TCP (6), length 132)

    yx-in-f113.1e100.net.https > 192.168.2.148.54451: Flags [P.], cksum 0xd31d (correct), seq 439558393:439558473, ack 2771839305, win 654, options [nop,nop,TS val 3861146831 ecr 792741186], length 80

00:00:03.153395 IP (tos 0x0, ttl 64, id 20084, offset 0, flags [DF], proto TCP (6), length 52)

    192.168.2.148.54451 > yx-in-f113.1e100.net.https: Flags [.], cksum 0xd2ad (correct), ack 80, win 65535, options [nop,nop,TS val 792741445 ecr 3861146831], length 0

6 packets captured

6 packets received by filter

0 packets dropped by kernel

[mathijs]

I have just opened a new issue about the appengine dashboard being
very inaccurate with reporting downtime, as just experienced.

I still think AppEngine's benefits far outweigh the occasional
downtime, but I think it's fair to demand that
Google's own dashboard is accurately reflecting what we're all seeing
in our apps. An inaccurate dashboard
causes needless investigation by app authors into their own apps when
the problem really (temporarily)
lies with the platform, and I wonder whether people at Google get
notified at all about problems
when the official dashboard shows that everything is OK.

If you agree with me, please star the issue:
http://code.google.com/p/googleappengine/issues/detail?id=4176

On Dec 1, 9:04 pm, datapharmer <dataphar...@gmail.com> wrote:
> I am on the east coast and am experiencing this 100% of the time today.

> Perhapshttp://code.google.com/status/appenginecoule be updated to

> reflect that this isn't exactly working. api-status.com seems to have
> more accurate data.
>
> some test responses:
>

> $ curl -Ihttp://appengine.google.com/

[Rafael Sierra]

100% here too

On Wed, Dec 1, 2010 at 6:04 PM, datapharmer <datap...@gmail.com> wrote:
> I am on the east coast and am experiencing this 100% of the time today.
> Perhaps http://code.google.com/status/appengine coule be updated to reflect
> that this isn't exactly working. api-status.com seems to have more accurate
> data.
> some test responses:
> $ curl -I http://appengine.google.com/
> HTTP/1.1 500 Internal Server Error
> Date: Wed, 01 Dec 2010 19:57:08 GMT
> Content-Type: text/html; charset=UTF-8
> Server: Google Frontend
> Content-Length: 466
> $ ping appengine.google.com
> PING www3.l.google.com (74.125.45.138): 56 data bytes
> 64 bytes from 74.125.45.138: icmp_seq=0 ttl=56 time=33.117 ms

Jealous :`( -- ~200ms here --

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

--
Rafael Sierra
http://blog.rafaelsdm.com

[Wesley C (Google)]

things should be back to normal now:
http://groups.google.com/group/google-appengine-downtime-notify/browse_thread/thread/d2b4ce71754bade5

pls join that group if you haven't already!

[dloomer]

Thanks for posting that.

 

I've also noticed a lot of sluggishness in the performance of the admin console (when it is working) over the past few weeks as well, and would hope that response times of the console would be reported in the status.  Task queue, logs, and datastore viewer can all take an inordinate amount of time even though they generally pull up no more than 20 records at once.

 

Somewhat related, I'd like to plug an issue someone else opened for the GAE dashboard a few weeks ago, requesting that deployment issues (of which I've also had many over the past few weeks) get their own slot in the dashboard: http://code.google.com/p/googleappengine/issues/detail?id=3906

[dloomer]

I got in once, a few minutes ago, but since then I'm still getting the 500 errors.

[Jules]

I am still getting 500 errors too....

[dloomer]

I think the overarching theme here is that there might be a gap in Google's monitoring, requiring users to report their experience in order to determine if things are working or not (and this isn't the first time I've seen this scenario).

 

Hopefully this can be better automated.  If api-status.com is doing a better job in some cases, that provides hope that there are similar things thart Google can do.

[Daniel Upton]

Now I'm getting deployment errors, but the admin console is back.

[Joshua Smith]

+1

[Nick]

Since the admin console came back up, I can't deploy my application.  Here's the message:

Try Again (503)

An unexpected failure has occurred. Please try again.

Is someone still working on this?

Please help.

[Amir]

It seems to fail during precompilation...

output:

....

Precompilation starting.

2010-12-01 16:12:39,897 INFO appcfg.py:1814 HTTP Error (HTTP Error 503: Service Unavailable)

Rolling back the update.

Error 503: --- begin server output ---

Try Again (503)

An unexpected failure has occurred. Please try again.

--- end server output ---

[Amir]

it's up... and super fast for me :)

[InspiradoGames]

Oddly, mine's been working 100% all day until about 30 minutes ago. Now I get this 100% of the time:

Rolling back the update.

Error 503: --- begin server output ---

Try Again (503)

An unexpected failure has occurred. Please try again.

--- end server output ---

 Any news on a fix?

[Jing Huang]

Some urls of my application started to return server error from yesterday and they still do so now.

The home page can be loaded although slow. Some links from the homepage work, some always don't since yesterday.

Also, I deployed a new version with minor layout change, the problem is not solved and the app engine still presents the old one.

[Ikai Lan (Google)]

Whenever you see a 503 on deploy, can you try turning off precompilation and giving it a try? We've been having issues with this service lately.

--

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blogger: http://googleappengine.blogspot.com

Reddit: http://www.reddit.com/r/appengine

Twitter: http://twitter.com/app_engine

--