Unneeded instance started

[Francois Masurel]

I'm seeing lots of loading requests recently : every few minutes for a rather low traffic application (vncts1).

Some new instances are started even if there is already one resident and one dynamic instances running (cf. screenshot).  Strangely the already running dynamic instance doesn't seem to serve much requests.

What is annoying is that these loading requests seem to hang all other requests.

I have Idle instances set to 1 - 1 and Pending latency set to 10s - 15s.

I hope we are not going a few months back on that point.

François

[Johan Euphrosine]

Hi Francois, 

Did you turn on concurrent requests? This could help to optimize the usage of instance that are already started, and reduce the number of loading request for starting new one as a consequence.

Your loading request latency seems quite high (30s), was it always the case or is it something that changed recently?

Hope that helps.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Cs85Y9CwoKgJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations

[Francois Masurel]

Hi Johan, thanx for answering.

Yes, concurrent requests are activated in appengine-web.xml :

<threadsafe>true</threadsafe>

<warmup-requests-enabled>true</warmup-requests-enabled>

Usually warmup requests take less than 10s, but since a few days some warmup requests can go up to more than 70s:

What I was use to get and still get most of the time :

2012-02-22 14:29:41.731 /_ah/warmup 200 9910ms

2012-02-22 11:16:53.080 /_ah/warmup 200 9854ms

2012-02-22 10:25:33.876 /_ah/warmup 200 9372ms

2012-02-22 10:11:18.477 /_ah/warmup 200 9040ms

2012-02-22 10:06:49.817 /_ah/warmup 200 10830ms

And what I get now quite frequently :

2012-02-22 14:02:54.150 /_ah/warmup 500 71762m

2012-02-22 12:55:42.130 /_ah/warmup 500 74788ms 

2012-02-22 10:38:42.771 /_ah/warmup 500 73526ms

2012-02-22 09:51:34.343 /_ah/warmup 500 73667ms

My warmup request just loads and instantiates a few classes.

Application ID : vncts1

Thanx for your help.

François

[Johan Euphrosine]

Hi Francois,

Thanks for these information.

The engineering team is aware of loading request latency issues.

They are currently working on a general solution to this it that should hopefully roll out in the next release.

In the meantime I would recommend setting Max Idle instances to Automatic to prevent the scheduler to recycle (killing and restarting) your instances too much, this should significantly reduce the spikes in your instances graph.

Hope that helps.

François

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/lPEfJuaJO8gJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Francois Masurel]

I have set Max Idle instances to Automatic, just hoping it won't cost me too much and hit my daily budget.

Thanx for helping.

François

[Robert Kluin]

Hey Francois,
I'm curious, have you noticed an increase since setting this to
automatic? On several apps where I've bumped into this issue in the
past week or so I've seen a bunch of instances hang around through low
traffic times.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/vEjQDF-eBz4J.

[Francois Masurel]

Hi Robert,

I have set Max Idle Instances setting to full Automatic and my frequent instance restarts have disappeared.  No more "loading_request=1" in my logs every 3 minutes.

I now have to dynamic instances permanently "active" but only one seems to be "billed" (cf. attachments).  So, for the moment, it does not cost me more than before.

But is it normal behavior ?  I better keep an eye on it.

François

[Robert Kluin]

Hey Francois,
Interesting. On apps where I'd set it to auto, I generally see the
billed line track with the total line. I also generally see instances
linger for a long, long time being largely unused.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/ONbZnH5_VSEJ.