500 - Request was aborted after waiting too long to attempt to service your reques

110 views
Skip to first unread message

Leonard Cremer

unread,
Sep 21, 2021, 10:57:13 AM9/21/21
to Google App Engine
We are suddenly getting a lot of these type of errors on services that have not changed recently. 

Is anyone else experiencing similar errors. 

Kaan

unread,
Sep 21, 2021, 4:11:24 PM9/21/21
to Google App Engine
I've always seen them

Leonard Cremer

unread,
Sep 22, 2021, 4:01:38 AM9/22/21
to Google App Engine
This is how our server errors increased suddenly yesterday with no change on our side. 

Screenshot 2021-09-22 095905.png

Neelanshu Jain

unread,
Sep 22, 2021, 4:24:29 AM9/22/21
to Google App Engine
Yes, I've also started seeing these since very recently(past 1-2 days) on services served by App Engine. Glad to know it's not just me , meaning not somethng that I've done wrong. :) 

Joshua Fox

unread,
Sep 23, 2021, 7:46:57 AM9/23/21
to google-a...@googlegroups.com
On  deploying to GAE, an  image is automatically created in my Google Container Registry . Though tmp is in the name, it does not go away, even when I delete the service. For example, I created and deleted about 20 services, and got the images below. 

How can this be avoided? How can these be automatically cleaned up? I'd rather not write my own script, given that this is internal to GAE.

//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-arineqod/ttl-18h@sha256:7f9c736e3edf6d3b1cd28b2060316a5b180ea1722d4aacb778550cb391cb7019
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-arineqod/ttl-18h@sha256:7f9c736e3edf6d3b1cd28b2060316a5b180ea1722d4aacb778550cb391cb7019
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-bicfjzey/ttl-18h@sha256:58524bd0c3164006863964d2df87a4b19ec15e004a152af98aba63f16dc0bde4
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-bicfjzey/ttl-18h@sha256:58524bd0c3164006863964d2df87a4b19ec15e004a152af98aba63f16dc0bde4
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-gcdtukmv/ttl-18h@sha256:aa0f9f53463e6fbbaca540d6004c1b485dfc8b9d35c69a76a6ef6dcea78a09cd
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-gcdtukmv/ttl-18h@sha256:aa0f9f53463e6fbbaca540d6004c1b485dfc8b9d35c69a76a6ef6dcea78a09cd
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-gmahecwr/ttl-18h@sha256:af58f9793e09e46bdbcd9fb528d51bb91cadd1799519fb615c6cd78ed05b7925
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-gmahecwr/ttl-18h@sha256:af58f9793e09e46bdbcd9fb528d51bb91cadd1799519fb615c6cd78ed05b7925
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-hyqkcmoz/ttl-18h@sha256:596600a236b36cbded3cebb95f0e77b50f20e32188dd9184f7052dedb542ed57
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-hyqkcmoz/ttl-18h@sha256:596600a236b36cbded3cebb95f0e77b50f20e32188dd9184f7052dedb542ed57
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-idgobnau/ttl-18h@sha256:f6d7a15ff18d7acc31040beefa50b8d3189dd6246a5878605a6697c7130c54e4
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-idgobnau/ttl-18h@sha256:f6d7a15ff18d7acc31040beefa50b8d3189dd6246a5878605a6697c7130c54e4
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-inukagez/ttl-18h@sha256:ab500e25adf150495dec6d9ec7cc847b296a2d514fefbccbaef9c82afa5570ad
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-inukagez/ttl-18h@sha256:ab500e25adf150495dec6d9ec7cc847b296a2d514fefbccbaef9c82afa5570ad
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-lugvxqbd/ttl-18h@sha256:61ec3be120f078114c9dff08afd3d99d3ee43977f9e4ea345ff984d2d9698164
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-lugvxqbd/ttl-18h@sha256:61ec3be120f078114c9dff08afd3d99d3ee43977f9e4ea345ff984d2d9698164
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-tcmaelwy/ttl-18h@sha256:9cdd15d4deef455cd43927980c5aebbf06d7825cdac8eb6914c085d6c6dac961
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-tcmaelwy/ttl-18h@sha256:9cdd15d4deef455cd43927980c5aebbf06d7825cdac8eb6914c085d6c6dac961
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-tlwznvsm/ttl-18h@sha256:9602b1e5e2a6c2aa94c926ab61043a0777848208ab461954a0defd4c417de829
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-tlwznvsm/ttl-18h@sha256:9602b1e5e2a6c2aa94c926ab61043a0777848208ab461954a0defd4c417de829
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-xmlbkihy/ttl-18h@sha256:786378d1f9998270f6d593c33a756a9afcec897dc39cf3c6f5d118ba658a6fef
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-xmlbkihy/ttl-18h@sha256:786378d1f9998270f6d593c33a756a9afcec897dc39cf3c6f5d118ba658a6fef
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-ylewaxuz/ttl-18h@sha256:898a3cb8f68e13807a117d28bf325a576710a4b6799a23ed8b83e82f0f771988
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-ylewaxuz/ttl-18h@sha256:898a3cb8f68e13807a117d28bf325a576710a4b6799a23ed8b83e82f0f771988
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-ywaxosuk/ttl-18h@sha256:bc5ac147f33318f7a6f5941166b98af277afab04de9272b1f8640aa332c7ab23
//containerregistry.googleapis.com/us.gcr.io/myproject/app-engine-tmp/app/service-ywaxosuk/ttl-18h@sha256:bc5ac147f33318f7a6f5941166b98af277afab04de9272b1f8640aa332c7ab23

David Scott

unread,
Sep 24, 2021, 11:17:57 AM9/24/21
to Google App Engine
Yes, I also saw several services stop responding (error 500) around 12:10AM EST on 9/21/2021, also with no changes on our side.

Will Reiher

unread,
Sep 28, 2021, 11:50:06 AM9/28/21
to Google App Engine
Totally, not a huge amount of them but definitely more RPC errors and timeouts in the last week or so. I'd expect the odd one but it would seem that some infrastructure changes are causing issues.

Ludovic Champenois

unread,
Oct 18, 2021, 11:03:27 AM10/18/21
to Google App Engine
Can you tell us with runtime are you using (Java, py, go?) and which versions?
Are you app.yaml based or appengine-web.xml based?

Ludo

Elliott (Cloud Platform Support)

unread,
Oct 18, 2021, 1:54:43 PM10/18/21
to Google App Engine
Hello Leonard,

Thank you for your response. As Google Groups is meant for general discussions about products, I believe the best way to report an issue is to use Google Issue tracker to report a possible issue and if it may regard to your project in specific, I would suggest to you to contact support through our usual channel here.

Jonathan Giorgio Ghellere

unread,
Oct 19, 2021, 1:10:08 PM10/19/21
to google-a...@googlegroups.com
Hello,
You had this error you mentioned + some huge 100s latencies?

We received lots of these errors on October 13th, it lasted for 6 hours straight. We believe it was due to some Datastore slowness.
We opened a google's support case, still, they had no clue what happened and just pointed us at some random documents. We got very frustrated with their answer, I mean, datastore is a managed service, no logs or metrics for us mortals... then today we escalated the case to see if we can get a specialist and some advice on how to handle those errors in the future.

Have you realized as well that this error and the 100s timeout is not even documented?
geez



--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/47ab989d-7571-4361-8b2e-7e2816ae79dcn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages