Troubleshooting downtime (potentially autoscaling-related) issues - causes 500

72 views
Skip to first unread message

Rainulf Pineda

unread,
Jan 22, 2019, 6:58:38 PM1/22/19
to Google App Engine
Hi,

We've been having downtime issues this past few months -- typically happen within few secs before it goes back up.

I suspect the cause of it is autoscale (looking at their start times) -- error 500 on the gateway.

Troubleshooting: I don't know what criteria app engine decides to autoscale (or kill instances) - is there any way to figure this out from the logs? Some sort of event log saying "CPU usage reached 30%, autoscaling to 3"?

Thanks!

Goran (Google Cloud Platform Support)

unread,
Jan 29, 2019, 2:43:38 PM1/29/19
to Google App Engine

Hello,


App Engine scales instances up when the volume of incoming requests increases and scales instances down when this volume decreases. This scaling helps ensure that all of your application's current instances are being used to optimal efficiency and cost effectiveness. For more information please see the documentation for scaling dynamic instances [1].


You can use several settings in the application configuration file to adjust the trade-off between performance and resource load. For the list of the available auto-scaling parameters, please see ‘Using performance settings’ section of this article [2].


A good recommendation is to benchmark performance with default scaling settings then run a new benchmark after each change. See this article[2] for some best practices to ensure that your app will scale efficiently to the volume of incoming requests.


[1] https://cloud.google.com/appengine/docs/standard/python/how-instances-are-managed#scaling_dynamic_instances

[2] https://cloud.google.com/appengine/articles/scalability

Rainulf Pineda

unread,
Feb 14, 2019, 4:56:35 PM2/14/19
to Google App Engine
Goran,

Thank you for your reply! I understand that App Engine scales up and down depending on certain criteria -- whether that's incoming requests, CPU usage, and more. 

But that is not my question. My question is if there's a way to determine its decision whenever it kills or create instances. A log such as this would be helpful: "incoming requests has decreased from 1000 to 10, therefore, killing 10 out of 50 running instances"

руслан мирзиеев

unread,
Feb 15, 2019, 4:07:30 PM2/15/19
to google-a...@googlegroups.com
Выйдете со мной на связь

пт, 15 февр. 2019 г. в 03:56, Rainulf Pineda <rainu...@gmail.com>:
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9e14deca-fdbf-42f6-8e6e-5ea24ba8b484%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages