Policy for instance startup

[Tomas Alaeus]

I'm curious when exactly instances are started. I have two
applications running on GAE, one of them have billing enabled. The one
with billing enabled have been stress tested and have at most started
100 simultaneous instances. The other is just for testing and staging
purposes and have never handeled much traffic.

What I experience is that the staging server never starts more
instances than needed. If a single person views pages it will never
load more than a single instance. The other one however seems to start
about 5 instances before anyone can get hot responses, and it will
continue to start up to about 10 before realizing that ~1 QPS isn't
that much traffic (the requests finish in about 100ms each).

So, why does GAE boot up lots of instances even though 1 instance can
serve the incoming traffic without a problem (the requests doesn't
even overlap, so no waiting is needed)?

I realize that this isn't a very big issue, since when it gets lots of
traffic it will indeed need all the instances. I'm just curious why it
happens.

[nickmilon]

Not been a Googler can't help much with this.
Having said that, I suspect there is a kind of build in algorithm that
does some kind of application profiling taking into acount QPS,
response times, and other parameters which adjusts instance life time,
number of instances to start etc..
This could possibly explain the difference in behaviour between your
staging and production appls.

happy coding;-)

[Tomas Alaeus]

Probably. With some more thinking I got to this conclusion:

At first, an application can utilize up to 30 instances. After having
used all those instances for a few minutes, and it still isn't enough,
GAE will increase the maximum number of available instances to 40, and
then to 50 etc. Every time it increases the number of instances it
will probably also adjust how and when it boots up new instances.

For my production app it probably tries to keep at least 10 instances
running since it thinks a huge amount of users can use the site at any
given time. The staging haven't had any rush of users, and therefore
thinks it is safe with just 1 instance running.

I would still like to hear something from a Googler. If this is how it
works then it means that stress testing the application (with a lot
more pressure than it will normally get) may lead to worse (!)
performance permanently.

[nickmilon]

While waiting from a Googler to give us some light on this may be you
do your stress tests in the staging server ? (just an other version
will do as staging server)
Also I remember people from App Engine team stating here that the best
performance comes when load comes in a natural matter.

Still IMHO that there must be some more transparency into those
parameters within which our appls have to operate. - It helps both us
and the platform.

Regards
Nick

[Tomas Alaeus]

This problem doesn't seem to happen on the staging server (doesn't
have billing enabled), but I maybe haven't stressed it enough. Another
application I have for testing purposes seem to be affected by it too,
and it doesn't have billing enabled either. That application seems to
want at least 4 instances before serving hot instances to incoming
requests.

This is actually quite bad for everyone. If the load is low the user
experience will be worse, we have to pay more and Googles servers will
have to boot up/down instances for no reason. Can any Googler confirm
this behavior?

[Tomas Alaeus]

I hoped that Warmup Requests could be used to remedy the situation (at
least from a users perspective). Unfortunally, it does not help
anything.

Warmup requests seems to be inactive (or not react) when there are
less than 10 instances active for my live app. The warm up requests
starts to work when 10 instances are started.

It seems to me that my app-id have been marked permanently as a
"heavy" application, which isn't true yet.