Will AppEngine autoscale if current instance(s) don't have enough memory?

[Alex Kerr]

Hi,

I've got a PHP app running in standard AppEngine. It does use a bit of memory for in-app image manipulation. I'm wondering what happens when there are enough concurrent users of my web app that the current instance(s) don't have the memory to handle more users (i.e. so it's not a CPU load issue, or other things (?) that might cause the app to scale). I don't want the users to see errors or a non-working app, I just want the AppEngine autoscaler to add another instance(s).

Is this how things work? If not, is there a way to get autoscaling to happen on the basis of current instances running out of memory?

Many thanks!

Alex

[Patrice (Cloud Platform Support)]

Hi Alex,

Scaling doesn't really happen based on memory, but based on latency and number of requests. It's highly tied into your scaling settings (either manual, automatic, or basic), and which exact config you have. You can lookup this document that contains some information on scaling, or this one which explains the configuration you can setup (look a bit further down than my link, it includes how to setup the three types of scaling and the options available to you)

But to answer "can we have autoscaling when an instance run out of memory", not on App Engine directly. One thing you definitely can do is to use ManagedVMs and the beta autoscaler, which you can set to "target utilization level" based on CPU utilization.

Since app engine scales a bit differently, there is definitely a way to do it, but it would be a workaround that would be way more involved than simply switching to ManagedVMs, so I wouldn't recommend it, as it would be clunky, might degrade customer experience (by making your whole app a tad slower).

Cheers!

[Alex Kerr]

Thanks Patrice. Your second document refers to Python. I'm using PHP - are the same options applicable to that too?

I'm just wondering what to do when there are sufficient numbers of users of my website (at any given time) to cause the running instances to run out of memory. Logically, the thing to do would be to get more instances fired up while the extra requests are coming in. I really need a simple, automated as possible solution (hence why I'm using app engine). If it doesn't respond to memory usage directly, is there a way to instruct or trick the App Engine autoscaler to initiate more instances (i.e. trigger it to do so based on some other metric)?

Surely there has to be some alternative here to my users seeing my app fail due to out of memory errors!?

Thanks,

Alex

[Stuart Langley]

How many users could a single instance handle concurrently before it runs out of memory now? 

If you know deterministically how much memory a user request will require you can implement some logic in your app to slow/pause the request processing in the instance when it is close to running out of memory. As the request latency grows (because you're slowing requests at the instance level) then more instances will be added to handle the users load. Basically you just queue requests at the instance level.

[pdknsk]

> I don't want the users to see errors or a non-working app, I just want the AppEngine autoscaler to add another instance(s).

App Engine has a soft memory limit. So when a request exhausts 128MB on an F1 instance, the instance doesn't crash but allows the request to complete. Only then does the instance quit.

[Tapir]

I have a Java GAE project on F1 instances.
Its instances use more than 200M memory at most time.
Google never close these instances. :)

BTW, the memory consumption will be much smaller by using Golang.
I'm porting this Java project to Golang now.

[Alex Kerr]

Thanks Stuart. That sounds good and I was starting to think along those sorts of lines.

But how would I assess how much memory a user request uses? Are there particular PHP commands or AppEngine or Cloud Console features I could use to determine this?

And secondly, how do I assess how close to the instance memory limit I am (at least from the PHP perspective)?

Currently instances seem to be a bit of a black box to me - not sure how close to resource limits I am with any particular code I run - would love any tips on determining this! (e.g. also for using VFS etc.)

Many thanks

[Stuart Langley]

That why I said "If you know deterministically" ;) At a guess I would think most image manipulation operations would use a predicable amount of memory for a given image size/type.

However, I would suggest unless you're manipulating high quality images (e.g. > 25MB in size each) you should not be overly concerned about scaling based on memory usage. As others have mentioned there are soft and hard memory limits - you're users would only see an error if you exceeded the hard memory limit and the instance was killed. 

For soft memory limits the instance is terminated when the requests have completed.

[Alex Kerr]

Thanks. How can we find the correct figure for memory usage and availability from PHP, within the App Engine environment?

For mem usage seems I can use memory_get_usage() and/or memory_get_peak_usage() and either with or without (true) to (from PHP docs) "give the real size of memory allocated from system. If not set or FALSE only the memory used by emalloc() is reported."

For mem availability is ini_get('memory_limit') accurate - it reports "2G" (I assume 2 GB?) on App Engine but that can't be right can it, I thought basic instances only had 128 MB?

Alex