Are my QPS and latency typical? (related to new pricing)

[jon]

Hi guys,

I may be missing something here but looking at the below screenshot of my instances, it seems like GAE is overly eager to add instances.

Each instance seems to be serving a request every 8 seconds approximately, which feels high. Do the numbers look right to you?

Thanks in advance,

Jonni

[Alexis]

Hi,

I also have noticed something odd between QPS and latency, although my
numbers are not as bad as yours.
I generally have stats like this:

Total number of instances Average QPS* Average Latency* Average Memory
210 total 0.599 252.3 ms 72.1 MBytes

With a latency of 0.25sec, I'd expect to have a QPS close to 4, and
hence reducing the number of instances needed.
But QPS is still very low...

What makes QPS and Latency so different?

> <https://lh3.googleusercontent.com/-R73wUepWYAA/Tl-eYELjwMI/AAAAAAAAAA...>

[Jon McAlister]

Hi Jon and Alexis,

Indeed by default the scheduler will try to make space for spare
capacity in your frontend instances. There are usually two reasons for
this.

The first is that requests do not usually come in regularly, they
arrive in spikes and various irregular patterns. In order to
comfortably handle the spikes without pending latency, an app needs
more instances than would be expected by purely multiplying qps and
latency.

The second is that if the app were to receive more load, having spare
capacity is helpful to serve the additional load without pending
latency.

The design of the scheduler is that the default automatic mode decides
to minimize pending latency, and provide excess spare capacity, to
help deal with the two issues explained above. Obviously, not all apps
will want this, especially in light of the new billing formula. This
is why there is now the ability to opt-out of this by using the
Performance Settings options of max-idle-instances and
min-pending-delay. With those options you can opt-out and signal to
the scheduler that pending latency and spare capacity are not as
important for your app as instance utilization.

I hope that helps,
Jon

> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
>
>

[jon]

Thanks Jon for the reply. After a few days of tinkering I can see how
important it is to have that spare capacity. It's not fun to make a
request that cold starts an instance.