Fwd: New Admin Console Released

[Marzia Niccolai]

Hi,

This morning we released a new Admin Console which includes a more descriptive accounting of your application's CPU stats.  We now display the amount of CPU consumed per request, as well as the avg amount of CPU used per handler.

In addition to the CPU stats, you can see the short-term request rate per handler, as well as the total number of requests a day for each handler.

Sign in to the admin console and check it out!

-Marzia

[Sylvain]

Hi,

It looks nice ;)
I've many questions about it :

* I'd like to know how the CPU is counted. Most of my requests are
high because in 1 request (i can't change it) :
- I fetch an url (and it is counted in the CPU)
- I get an XML and parse it (10 kb)
- I zip it and store it
So now everything (xxxxx mcycle) is orange even with no Warning (high
amount of CPU, and was roughly x.x times)

A strange thing is that now I can have a Warning (high amount of CPU,
and was roughly x.x times) but with a low mcycles number (the number
is not in orange)

* why did you remove the minor version everywhere ? It was a good
idea.

Regards

Sylvain

[Joel]

I like the added visibility.

But now I want to find out where my mcycles are going!

Is there an api analogous to time.time() that I can use to see how
many megacycles I've used to that point?
I would find that helpful.

Joel

[Larry....@googlemail.com]

I like the ease at which bottlenecks can be identified but what does
this mean:

"This URI uses a high amount of CPU and may soon exceed its quota."

Does each URI have its own quota now? Or are the quotas for the whole
application? Because I'm nowhere near using up my entire application
quota, but I get that warning for one of my URIs (which I knew was
inefficient).

[Marzia Niccolai]

Hi,

To address some of the comments in this thread:

1 - Minor Versions - We did eliminate listing the minor version. This allows you to always access the latest changes for each version of your app at http://maj_ver.latest.app_id.appspot.com, instead of having to enter a new URL with each minor push.

However, we understand that you would like some clarification as to what minor version is live in production, so we are planning on adding this number back to the 'Versions' column.

2. Concerning CPU accounting overall - The warning threshold for the logs and for the dashboard actually differ.  In the logs, warnings will only occur for high runtime CPU.  The warnings on the dashboard are triggered for high overall CPU, including CPU used in the runtime and per API, most notably being the datastore API.

If you are curious to know where your resources are going for each request, we encourage you to use the Python Profiler:
http://code.google.com/appengine/kb/commontasks.html#profiling

This combined with the information in your dashboard and logs should allow you to pinpoint where any inefficiencies in your app may be.

Also, please be aware as to how each column in the dashboard is calculated (details can be found on the FAQ: http://code.google.com/appengine/kb/general.html#currentload), but to reiterate some of the information:  The number of requests for the handler is per day, calculated since Midnight PST.  The %CPU is also calculated over the same time period.  However, the Average CPU is only accounted for per hour.

3. High resource requests - We understand your application may have, on occasion, the need to consume more resources than normal - such as for parsing feeds or doing more complex calculations.  As long as these calculations are relatively infrequent, this should not be a problem. Be sure to cache or store the results of the calculations for reuse to minimize the incidents of such requests.

Quotas are not calculated on a per handler basis, but are per application.  However, providing insight in to how specific requests consume resources allows you to optimize inefficient handlers.

Hope this helps,
Marzia

[Sylvain]

Hi Marzia,

Thank you for your answers.

For the quota/CPU, I've spent many times to understand it and
particularly with the profiler. But as I understand, there is "no"
link between the profiler score and the number of mcycles.

I've created an issue about it : http://code.google.com/p/googleappengine/issues/detail?id=680

The profiler is fine to check an anomaly, infinite loop,... etc,...
but the way mcycles are calculated seems to be very different than how
the profiler do it.

An example : urlfetch.fetch(url) seems to count a lot in the mcycles
and not (near 0) for the profiler.
In my new project, zip.compress/decompress seems to use a lot in
mcycles too, etc,...

So I'd like to have more information how it is count and if it is
possible to have the details (like the profiler do it) for each method/
function that are used inside the request. May be a "profiler" mode
inside the dashboard.

Regards.

On 18 sep, 23:11, "Marzia Niccolai" <ma...@google.com> wrote:
> Hi,
>

> To address some of the comments in this thread:
>
> 1 - Minor Versions - We did eliminate listing the minor version. This allows

> you to always access the latest changes for each version of your app athttp://maj_ver.latest.app_id.appspot.com, instead of having to enter a new

> On Thu, Sep 18, 2008 at 1:00 PM, Larry.Ches...@googlemail.com <

[mitnickcbc]

I'm quite curious about the high amount CPU quota. It seems some
simple put or URL fetch will make the request easily be a high CPU
request. In my case, any request that involves more than 1 put or any
URL fetch, it tends to display as a high CPU request in log. And 80%
of my requests need to do around 2-4 puts and 2 URL fetch. This makes
the average CPU usage for most of my requests around 3K-8K mcycles.
That means the high amount CPU quota (which is small) is actually
limiting my app instead of the general CPU quota.

My question is, why GAE need to have 2 different CPU quota? I think
every app is different and not every app is line to the normal/high
amount CPU rate that GAE defines. In my case, most of my requests are
high amount CPU ones. As long as user will pay for the CPU usage in
future, what's the point to add the additional high CPU quota?

On Sep 19, 5:11 am, "Marzia Niccolai" <ma...@google.com> wrote:
> Hi,
>

> To address some of the comments in this thread:
>
> 1 - Minor Versions - We did eliminate listing the minor version. This allows

> you to always access the latest changes for each version of your app athttp://maj_ver.latest.app_id.appspot.com, instead of having to enter a new

> On Thu, Sep 18, 2008 at 1:00 PM, Larry.Ches...@googlemail.com <

[Martynas Brijunas]

Hi Marzia,

I noticed something curious with the Applications Overview panel. It
says that my application is not deployed despite it working properly.
Is it because I was on version 0.19 prior to the upgrade? Does it not
like the zero version numbers? The app is called gentlecolours. I can
still get all the stats if I click on the app name though.

Thanks!

Best regards,
Martin

[Ravinder Thakur]

my suggestion would be to change this whole CPU scheme.

my application is not a high traffic (yet :) ) its only few folks
querying it occasionally. My daily CPU usage never goes more than 10 %
and yet i get these time out errors since few queries exceed some
unknown CPU limits. Whats the point of terminating few complex queries
when the whole CPU usage never crosses 10% per day ? Isn't it strange
that i am not able to use 100% of CPU and yet my queries are being
terminated due to too much CPU usage :(


I would suggest to create an alternative scheme where app developer is
responsible for CPU calculation. If he is not using the paid service
and has consumed 100% for the whole day, just shut down the service
for whole day and ask him to purchase some more CPU.


thanks
ravinder thakur

[Sylvain]

"prev" & "next" at the bottom of the dataviewer has been removed.
It was a good idea, iI think.

On 18 sep, 19:30, "Marzia Niccolai" <ma...@google.com> wrote: