scheduler routing to dynamic instance while resident instance is idle

[Andreas]

i tried different setups and played around with the instance slider but the most requests always get routed to a dynamic instances rather than the resident one that should be there to do the job. why arent requests sent to that instance instead of the dynamic one?

[alex]

aschmid, as far as I understand, resident instances are there for the times when there's no dynamic instance currently available to serve an incoming request.

Think of this scenario: suddenly a new request arrives to your app. It is too long to wait for a new dynamic instance to warm up so the scheduler route the request to your resident instance, starting a new dynamic instance in parallel. Meanwhile, another new request reaches your app. The scheduler will route it to the newly created dynamic instance (if it's available/started already). Why? Because if new requests kept arriving and scheduler routed them all to your resident instance, there would be no available instances to serve new requests anymore. At least, not fast enough.

So, resident instances are there to fill in t2-t1 waiting time period, where t1 is the time of a new request and t2 is the time when that request can be served by an available instance of your app. 

I might be over-simplifying this but it actually makes sense if you think about it for a moment. I could be wrong though.

What you're probably trying to do is limit the number of instances. I think you might want to try the sliders in Application Settings instead, if that's the case.

[Andreas]

ok that would explain what happens. but it remains really unclear and not logical from how things have to be setup.

another thing ist that idle instances do not get shut down if there is very little traffic and 1 or 2 instances are handling all the load.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/GzQ9MztgFjsJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[alex]

I don't know the answer to that but there's definitely some or a few algorithms to the scheduler. 

I'm sure they're constantly looking to improve it. For example, if I were on GAE team and had live access to all the stats I'd definitely try experimenting with e.g. Prediction API to add to the scheduler to predict when to start and shut down instances of an app. 

I'm just saying, there's definitely some logic to the behavior you're seeing. Also, don't forget that the system is distributed and huge so sometimes you might not be seeing numbers in real-time, at least not 100% accurate, on the app console.

To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

[Andreas]

do warmup requests still make sense or is it that part deprecated?

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/XxuZu7wfTFAJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

[alex]

Good question, actually. 

From where I stand, the (desired) results of warmup requests and resident instances do overlap. For instance, both are useful methods when you want your app to be *always* ready to respond quickly, even in traffic absence in the nearest past.

They are somewhat different, though. A warmup is just a guy sitting there and POSTing to your app endpoint once in a while. Resident instance is sort of a Plan B whenever the "normal" plan doesn't work ("route request to a dynamically available instance" so to speak.)

Again, this is my understanding from experiments, observations, reading the docs and confronting it with "how I'd do it if I was to design instances balancing of that size". I might be wrong.