Frustrations with XMPP Presence and Frontend Instances

[Gerald Tan]

I've been trying to keep my Frontend Instances under control for a very-low traffic application that should remain under all other quotas. I have enabled multi-thread, set Minimum Pending Latency to 15s, and Max Idle Instances to 1, and hope to use only 1 Frontend Instance to stay under the daily quota for Frontend Instance Hours (because anytime a 2nd instance spawns, it is almost guaranteed that I will exceed the daily quota), but it seems impossible with the Scheduler deciding to spawn extra Frontend Instances for no reason other than to bill us higher.

What I've noticed happening is XMPP Presence Available requests always coming in pairs. Not sure if this is the correct behavior from XMPP clients or if this is a bug in the GAE XMPP engine double-sending XMPP Presence requests. This is fine if there is an Instance ready to handle them, but when I don't have an Instance up, the scheduler will spawn TWO (or even more if some other requests come in at the same time) instances to handle both requests simultaneously.

I'm guessing my daily Frontend Instance quotas will run out maybe 2-3 hours before the end of each quota-day because of all the unneeded 2nd Instances, and I'll have to pay $9/month to make it available 24 hours per day.

[Gregory D'alesandre]

Hi Gerald,

Have you looked at what the XMPP presence looks like for the pairs?  Many systems (including Google Chat) will send back 2 responses a basic one and then a more detailed one.  I don't believe this is a bug in our system but rather standard practice, but if you are getting 2 identical requests back and the other system is not sending those (in other words, you only get 2 when using our XMPP presence), let us know.  

The scheduler is not trying to spin up useless instances but if you don't have any instances running, then get 2 requests in, the scheduler will take you loading time into account and likely see that it won't be able to service both requests in the amount of time needed hence spinning up 2 instances.  Do you see a difference in the number instances created if there is already an idle instance running?  It might spin up fewer instances because it doesn't need to wait for the instance to be ready to serve the first request and therefore can have the second request wait.

Does that make sense?

Greg

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/ETL0ufhzwfYJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Gerald Tan]

Hi Greg,

I haven't looked into the details of the incoming requests, I will do that when I have a bit of time, but it probably doesn't matter too much to me anymore. I've pretty much decided I will have to pay $9/month for an estimated average 10-15c per day of usage, since 1k XMPP Stanzas/day is definitely not enough for my app... Don't suppose that $9/month minimum could be reconsidered?

I guess I understand what the scheduler is trying to do, and I don't see the problem anymore now that I've setup a 1 min heartbeat to keep the instance up. It does spin up extra instances occasionally when the XMPP or Datastore API chokes up and causes an ApiDeadlineExceededException. I guess it shouldn't matter after I enable billing and will be able to set min idle instance to 1.

Probably a minor issue though, but with Min Pending Latency set to 15s and my instance loading time being ~5-6s, should the scheduler be opening a new instance? I think it could be smarter because it shouldn't be necessary to open a new instance when it knows that another one is in the process of being opened. Not that it really matters now that I understand we will be charged for Max Idle Instance + Active Instances