502 backend error handling with Flask python on GAE

[Udit Sharma]

Hey all, 

I am using GAE for hosting my applications. I am facing a problem related to load balancing error of 502 bad gateway. 

When the requests on the server surpass the load and GAE are yet to spin up the instances to balance it, 502 bad gateway get thrown, which defines the requests are failing. I wanted to know the ways of handling this error from the backend/frontend, so that if the loads get more it will show to the user "Servers are down" instead of the application gets crashed.

I am handling the 500 backend error since whenever it happens response is a JSON type: "message":"internal server error".

Can I get something same for 502 error too like "message":"load balance error"?

I am a novice here, any help would be grateful.

Thanks

[Yannick (Cloud Platform Support)]

Hello Udit, you are most certainly getting those 502 errors because you've set a strict maximum number of instances even though it appears your application needs to handle heavy spikes in utilization. To mitigate these errors you could remove the maximum cap or rate-limit to stretch the spikes over a longer period of time.

As for the 502 response code, what's stopping you from handling it? How is your application used?

[Udit Sharma]

Hey Yannick, 

Thanks for the reply. 

There is no limit I have kept for the instances, I was load testing that where I got these error. I wanted to know the ways of handling that.As I said, I do not know much here. If you can tell me how should I handle that, I googled and got these results here and here.

I am sure how to handle this using Flask. If I need to do handle things using Flask, I will find it out, thanks for the help once again.

[Jordan (Cloud Platform Support)]

502 Bad Gateway usually means that the nginx proxy that is responsible for handling requests for an App Engine Flexible instance has not been able to get in contact with your application and deems it to be unhealthy. This is done via health checks that nginx performs (you can test this by disabling health checks). 

If your application code is completely busy handling requests, then it will not be able to respond to any health checks, and nginx will tell App Engine that the instance is not healthy and it will be turned down with a 502. App Engine will then automatically start up a brand new instance in its place. 

By default, the App Engine Flexible Python runtime uses sync workers. Sync workers force only a single request to be handled by your instance at a time. It is therefore recommended to ensure your application code use async workers so that it may always respond to health checks if it is indeed healthy. 

In terms of handling this on the frontend; you are correct in your assumption to perform exponential backoff retry. Whenever your frontend client receives a 5XX HTTP response code from your App Engine application, it should retry the exact same request but only after waiting a small delay. This exponential delay is used to give App Engine some time to recover and start a new instance, and will prevent your application from getting overloaded with requests.