Google app engine not distributing requests properly

[Shehan Jayasinghe]

Hello!
We have a dockized PHP Nginx application running on google app engine flex. 

We have automatic scaling and minimum 1 instance is always running. 

As soon as traffic spikes, users receive 50x gateway errors for some time. The autoscaler works as intended but after checking the logs,  all errors originate from the idle instance due to app engine sending most of the requests received to the first instance overloading it.

Why is app engine not distributing requests evenly? What can we do to make sure the service is available during high traffic bursts?

Thank you for your time!

[wokmou]

From what I understand , your instance is overloaded during spikes. My suggestion is to set target_concurrent_requests [1] parameter to limit the number of concurrent requests per instance. This will trigger the creation of a new instance as soon as the limit is reached.

There is also the max_concurrent_requests [2] use to specify when a new instance is started due to concurrent requests.

[1]https://cloud.google.com/appengine/docs/flexible/python/reference/app-yaml?hl=en#automatic_scaling

[2]https://cloud.google.com/appengine/docs/standard/nodejs/config/appref#scaling_elements