unnecessary instance spinning causes latency issues

[Rajesh Gupta]

We are using Java appengine standard.

The appid is our test server with 1 min instance running always on F1

Even if there is no traffic, the instances are spinning.  This is not good.  Why should there be so many instance restarts.  

Low traffic website are facing the following latency because of constant instance spinning

Lets say, the instance is started with the /_ah/warmup and the instance is ready to serve.  Now a new request arrives and the newly created instance serves the request.  The first request here always takes a longer time, because of java loaders loading the classes and other things.  Had this request been served by a old instance, the serving will be faster.

Please see the following instance spinning with no requests at all.   Don't kill the instances, and try to serve the request always with a older instance.

 

--

Regards,

Rajesh

www.VeersoftSolutions.com

www.GainERP.com

Accounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and Mobile

[Nicholas (Google Cloud Support)]

If I understand this report correctly, you are pointing to the time taken from when the instance is ready to when the servlet is ready.  With App Engine, the loading of code/runtime is separate from the loading of the instance when it comes to instance creation.  One of the reasons for this choice is to better handle traffic spikes.  When runtime loading is included in instance creation, it may take longer for the instance to report as ready to receive new requests.  During that time, additional instances may be spun up.  This of course, depends on how long it takes to load the runtime and your application's servlets.

To reduce this, have you considered the <load-on-startup> property for the Java standard runtime servlets?  This will instruct App Engine to load said servlet when the instance is warmed up.  This seems like an appropriate way to address the issue you've presented here.

Alternatively, you could also schedule a cron task to periodically send requests to a specific endpoint.  While this won't ensure that every instance loads its servlets, it will ensure that at least one instance loads the appropriate servlet to handle the request.

Lastly, if you're really looking to micro-optimize this, keeping in mind the dictum that premature optimization is the root of all evil, you might consider using a runtime with faster load times like Go.  In general, coping your application's binary and running it tends to take less time than loading the JVM, all your classes and then your servlets.  While the slides may be a little dated, High Performance Apps with Go on App Engine is still relevant today and shows some of the performance benefits of using Go.