50% increase on latency with "Max Idle Instances"=1 ?

[Raymond C.]

I tuned the value of "Max Idle Instances" to 1 as suggested by many people here since a few days ago. Since then the ms/request value has been increased by around 30-50% as you can see from the attached charts.  No code were changed during the period.  

I suppose "Milliseconds/Request" has latency included right? If so, is everyone having the same result by tuning down the max idle instance?  30-50% latency increase is not breaking the game, but still the less latency the better.

[Raymond C.]

Oh, it should be setting "Max Idle Instances" to 1 has increased latency by 200ms for each request (by average) according to the charts.  Am I right?  Anyone else seeing similar result?

[Robert Kluin]

Yeah I'd suspect reducing the number of instances and increasing the
min-pending latency to hurt and app's performance quite a lot. App
Engine instances don't generally handle many QPS, even for fast apps.
And the latency probably does matter for a lot of people. I think
Google actually published some analysis on that and found a few
hundred ms makes a big difference.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/AgP2fnxuc0gJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

[Raymond C.]

I haven't changed the "min pending latency" settings ("automatic").  And as I can there are still some "idle instances" even I have set the "max idle instances" to 1.  What concerns me is, is setting it to 1 really hurt the latency of the application in general?  That is plain increase of 200-300ms latency increase for each request in average.

[Robert Kluin]

Yeah, I've seen the same thing. I also find it less than desirable.

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/BeLVxGLKYfcJ.

[Jon McAlister]

Hi Raymond,

Indeed, this is the down-side to tuning the performance setting knobs
to favor cost over performance. Serving latency and reliability may
become worse.

The design of the scheduler is that the default automatic mode decides
to minimize pending latency, and provide spare capacity (to provide cover for
request spikes or load increases). Obviously, not all apps will want this,
especially in light of the new billing formula. This
is why there is now the ability to opt-out of this by using the
Performance Settings options of max-idle-instances and
min-pending-delay. With those options you can opt-out and signal to
the scheduler that pending latency and spare capacity are not as
important for your app as instance utilization. But the default is to
provide good performance and reliable serving.

I hope that helps,
Jon

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/wK7C600QgjMJ.