[CRITICAL] Hanging Python Instances Since Last Maintenance

[Alex Popescu]

Since the last maintenance, I'm seeing an elevated number of Python instances just hanging. My application (id: twimblr) has been running for a long time and I've never seen this behavior before. 

This issue is critical as:

1. they hang around indefinitely even if I configured the app to allow *1* idle instance

2. these instances are also consuming my credit (so they seem not to be in the "idle" mode either)

As a side note, I'd like to know what the best way to report these issues as a paying customer. Just posting to a forum doesn't feel the most effective/customer friendly way.

thanks

[Nikolai]

+1

[Fabiant7t]

+1

[Sharp-Developer.Net]

+1 - application: petaclasses

We are down - exceeded daily budget in 6 hours.

[Alex Popescu]

Interestingly when I've tried to pull out the instances graph I've discovered that there is no data older than 1.5 days is available--which I think corresponds to the period of the maintenance.

According to the documentation: The information displayed in these graphs gives you a snapshot of resource consumption per second over a period of up to 30 days. 

I would kindly remind you that I'm a paying customer and its scary to know that:

1. I will be billed for instance hours I'm not using

2. I have lost access to historical data about the app

3. most importantly there's no transparency and no customer support on these issues

I am looking forward to hearing something from Google support.

(PS: as I've already mentioned, my app id: twimblr)

[Alex Popescu]

After going through the logs I've started to notice a pattern:

1. elevated instance startup times

2. warnings related to the usage of Django 0.96

After some experimentation I have deployed a new version that is using Django 1.2 and start using this version for the last hour or so and I'm not seeing the reported behavior.

On the other hand considering the many similar threads in the forum, it's hard to say if this is the result of my change from Django 0.96 to Django 1.2 or work done behind the scenes.

Anyways I'm still waiting for a Google official answer as this issue 1) led to increased billed instance time; 2) lost of historical stats.

On Friday, March 9, 2012 1:35:12 PM UTC+2, Alex Popescu wrote:

[Christopher Wong]

+1

[Jeff Schnitzer]

On Fri, Mar 9, 2012 at 12:57 PM, Alex Popescu <alex...@mypopescu.com> wrote:
>
> 2. I have lost access to historical data about the app

Sorry if this seems incredibly obvious, but I just to make sure the
simple answer is covered: You know that logs are attached to specific
versions, right? If you make version 15 default, all the logs for
version 14 will still be attached to that version and you must
explicitly ask for it in the admin console.

Jeff

[Alex Popescu]

Yes, Jeff. I'm fully aware of the stats data being associated with major app versions.

My app has been at version 2 for a long time and I've attached the chart of instances for the last 4 days.

[Alex Popescu]

I'm taking this back... the erratic random instances behavior is still present.

I'm attaching a screenshot with the total number of instances: 5 with an avg. qps of 0.013 and avg. latency of 513ms (admin setting: 12s). In the last 30 minutes, the dashboard has recorded 1.5 instance time hours.

At this point I'm clueless and I'll just have someone to look into this (hopefully before I'm running out of budget).