Please Stop Throttling my App!

[Coding Social]

Hi,

I have had appid mapthislink for many months now. Recently my
extensions that use this web service to unwind urls have been featured
by Google Chrome and Apple Safari so usage is up substantially.

Can someone turn off the throttle? Causing latency and 13% error
rate.

Thank you.

[Nick Johnson (Google)]

Hi,

We don't throttle apps. If your average latency is over 700 milliseconds for user-facing requests, we may not schedule additional servers for your app, however.

What leads you to conclude that your app is being throttled?

-Nick Johnson

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047

[bFlood]

not for nothing, but isn't "we may not schedule additional servers for
your app" throttling?

when did 700ms become a magic number?



On Sep 15, 9:33 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

> Hi,
>
> We don't throttle apps. If your average latency is over 700 milliseconds for
> user-facing requests, we may not schedule additional servers for your app,
> however.
>
> What leads you to conclude that your app is being throttled?
>
> -Nick Johnson
>

> On Tue, Sep 14, 2010 at 9:47 PM, Coding Social <codingsoc...@gmail.com>wrote:
>
>
>
>
>
> > Hi,
>
> > I have had appid mapthislink for many months now.  Recently my
> > extensions that use this web service to unwind urls have been featured
> > by Google Chrome and Apple Safari so usage is up substantially.
>
> > Can someone turn off the throttle?  Causing latency and 13% error
> > rate.
>
> > Thank you.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .

[johnP]

Interesting - earlier they said the magic number was 1000...

[Robert Kluin]

I am starting to get concerned. A few months ago this number was
1000ms, right? Then about a month or two ago it became 850ms;
actually I have even saw the 850 number posted within the last week.
Now it is 700ms?

From my experience, getting or putting even a single entity can use a
substantial portion of 700ms (20% to 40%). If you operate on multiple
entities you'll easily use 1/2 of 700ms. Just the act of _running_ a
query takes around 250ms -- when the datastore is actually functioning
correctly.

This trend is _really_ not good.

Robert

> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

[Harshal Patil]

+1

[Francois MASUREL]

I totally agree with you.

Francois

[Gordon]

bothering, indeed..

[Jeff Schwartz]

+1 and a whole lot more :(

While it is all our goals to produce efficient applications that can be scaled out, the platform itself has to be usable &, might I add, enforce ceilings that don't choke the life out of even the simplest of processes. In that regard I'd be willing to give up a little bit of scalability for somewhat more relaxed quotas.

But the real issue I believe is that of imposing unrealistic quotas. It is one thing to show an example of an efficient application built by Google and another to show how that relates to real world applications that though they employ all the same best practices still cannot function within the allowable quotas.

Resiliency is also a major issue on App Engine, if 99% of our code is protect the app from what can go wrong and that eats up our quota, what is left for doing real work?

It is my desire and I suppose that of many if not even most of the other developers that Google rethink their approach to providing scalability & resiliency to the masses on App Engine.

Jeff

--
--
Jeff

[Matt H]

+1

=(

[Harshal Patil]

I am OK with Google introducing tiered pricing for handle this issue. Don't take these numbers at their face values, but you would get the point I am trying to make here.

Avg. Requests               CPU Charges

< 700ms                         $0.02/hr

< 1500ms                        $0.04/hr

< 2000ms                        $0.06/hr

For all the requests Google provision new servers but if you requests take longer you pay higher. Not sure if it really makes sense, but the idea of totally not allowing any scaling up is not good enough motivation to write ever more complex apps. 

[Flips]

@Harshal
Actually slower requests mostly consume more cpu time and are much
more expensive by default..

On Sep 15, 8:28 pm, Harshal <p.hars...@gmail.com> wrote:
> I am OK with Google introducing tiered pricing for handle this issue. Don't
> take these numbers at their face values, but you would get the point I am
> trying to make here.
>
> Avg. Requests               CPU Charges
>
> < 700ms                         $0.02/hr
> < 1500ms                        $0.04/hr
> < 2000ms                        $0.06/hr
>
> For all the requests Google provision new servers but if you requests take
> longer you pay higher. Not sure if it really makes sense, but the idea of
> totally not allowing any scaling up is not good enough motivation to write
> ever more complex apps.
>

> On Wed, Sep 15, 2010 at 10:51 PM, Jeff Schwartz <jefftschwa...@gmail.com>wrote:
>
>
>
> > +1 and a whole lot more :(
>
> > While it is all our goals to produce efficient applications that can be
> > scaled out, the platform itself has to be usable &, might I add, enforce
> > ceilings that don't choke the life out of even the simplest of processes. In
> > that regard I'd be willing to give up a little bit of scalability for
> > somewhat more relaxed quotas.
>
> > But the real issue I believe is that of imposing unrealistic quotas. It is
> > one thing to show an example of an efficient application built by Google and
> > another to show how that relates to real world applications that though they
> > employ all the same best practices still cannot function within the
> > allowable quotas.
>
> > Resiliency is also a major issue on App Engine, if 99% of our code is
> > protect the app from what can go wrong and that eats up our quota, what is
> > left for doing real work?
>
> > It is my desire and I suppose that of many if not even most of the other
> > developers that Google rethink their approach to providing scalability &
> > resiliency to the masses on App Engine.
>
> > Jeff
>

> >> > >> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

> > --
> > Jeff
>
> >  --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to

[Ikai Lan (Google)]

If it scaled linearly like that, we probably wouldn't have problems with long running requests. Unfortunately, long running requests are bad for the ecosystem because they impose a non-linear cost.

The number is officially 1000ms. We have been saying 800ms because we allow for some variance. If you tuned your requests to be 990ms and had a period of 10ms of latency, you'd be dead in the water. 800ms is a safe enough number that even if you experienced an additional spike of 100ms-150ms for whatever reason (datastore slowness, unusual usage patterns in your application causing Memcache misses, network latency via URLFetch), you can tolerate it and be fairly confident you will be autoscaled. 

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

[bFlood]

does this count for the Task Queue as well? if so, how are we suppose
to run tasks that span a couple of seconds? are you saying that if one
task goes over 1000ms, you're not going to get any new instances? does
this ban on new instances last for a certain time period?

urlfetch - does one bad network hop (over 1000ms, for whatever reason)
cause you not to scale as well (i'm guessing yes)?

On Sep 15, 5:38 pm, "Ikai Lan (Google)" <ikai.l+gro...@google.com>
wrote:

> > > >> google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

> > e...@googlegroups.com>
> > > >> .
> > > >> > > For more options, visit this group athttp://
> > > >> groups.google.com/group/google-appengine?hl=en.
>
> > > >> --
> > > >> You received this message because you are subscribed to the Google
> > Groups
> > > >> "Google App Engine" group.
> > > >> To post to this group, send email to
> > google-a...@googlegroups.com.
> > > >> To unsubscribe from this group, send email to

> > > >> google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib

> > e...@googlegroups.com>
> > > >> .
> > > >> For more options, visit this group at
> > > >>http://groups.google.com/group/google-appengine?hl=en.
>
> > > > --
> > > > --
> > > > Jeff
>
> > > >  --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine" group.
> > > > To post to this group, send email to google-a...@googlegroups.com
> > .

> > > > To unsubscribe from this group, send email to
> > > > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib
> > e...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=en.
>
> > --

[Ikai Lan (Google)]

Apologize, I wasn't clear. The 1000ms limit is only for user facing requests. This does not apply to task queues or cron jobs.

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

[Martin Webb]

I think on this point one should also consider that whilst the mighty google can afford to invest time and money in building apps that run lightning quik. sadly most of us developers dont have the huge man power technology and needed cash to do the same. Google should remember that we are developers and most of us are developing for $nothing$. in hope that our ideas may reap reward. on that basis we have to all consider how much time we spend investing in lightning fast code - against getting the idea out their and get users. One of the key selling points of app engine is its platform and ability to do just that. Something that restrictions on code execution times and added costs will affect.

 

 

Martin Webb

The information contained in this email is confidential and may contain proprietary information. It is meant solely for the intended recipient. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted in reliance on this, is prohibited and may be unlawful. No liability or responsibility is accepted if information or data is, for whatever reason corrupted or does not reach its intended recipient. No warranty is given that this email is free of viruses. The views expressed in this email are, unless otherwise stated, those of the author

 

 

[nickmilon]

I have raised same issue a month ago here :
http://groups.google.com/group/google-appengine-python/browse_thread/thread/6353b6232e8851aa/3f907598c0826ca0?q=#3f907598c0826ca0
but I got no answer.

And here is my version of the story :
http://gaengine.blogspot.com/2010/09/app-engine-scalability-issues.html


happy coding:-)

[Robert Kluin]

Hi Ikai,
I think we all appreciate your response and clarification of this
issue. Could you also clarify one more point for us, the "100ms"
applies about the handler's actual response time and not the cpu_ms,
is that correct? In other words it is the first "ms" number in my
logs.

The vast majority of my requests complete well under 800ms -- even
some doing fairly "intensive" processing -- but the cpu_ms jumps all
over the map (largely) depending on the cpu_api_ms.

Thanks for the clarification.

Robert

[Ikai Lan (Google)]

That should be the case, yes. If it's not, please let us know. The CPU ms can be greater than 1000ms in aggregate since it includes parallelized calls to the datastore.

[Robert Kluin]

That was my understanding, just wanted to verify. Thank you.

On Thu, Sep 16, 2010 at 01:13, Ikai Lan (Google)

[Stephen]

On Sep 15, 2:33 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

> Hi,
>
> We don't throttle apps.

What does 'throttle_code' mean?

http://groups.google.com/group/google-appengine/browse_thread/thread/fd648f4b59281b0b/e7f664260536c721?lnk=raot#e7f664260536c721

[Nick Johnson (Google)]

Hi Martin,

Writing efficient apps doesn't have to be efficient or difficult - it just takes experience. Further, if you don't expect your app to scale to hundreds of requests per second, you may not have to worry about efficiency a great deal - and if you do care about that, then you'd have to worry about efficiency whatever platform you were running on.

We don't have a magic way to make apps scale indefinitely if they don't have a certain degree of efficiency to begin with - hence why you need to be prepared to invest at least a little time into optimisation if you want your app to run on any platform.

-Nick Johnson

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--

[Nick Johnson (Google)]

Hi Stephen,

On Thu, Sep 16, 2010 at 4:31 PM, Stephen <sde...@gmail.com> wrote:

On Sep 15, 2:33 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

> Hi,
>
> We don't throttle apps.

What does 'throttle_code' mean?

As I expanded immediately after the one sentence you clipped out of my reply, we decide to schedule additional instances of your app based on its user-facing latency. The original poster was asking about an explicit throttle on the maximum request rate for his app, which is not something that we have in place.

-Nick Johnson

 

http://groups.google.com/group/google-appengine/browse_thread/thread/fd648f4b59281b0b/e7f664260536c721?lnk=raot#e7f664260536c721

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Stephen]

On Sep 16, 4:33 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> Hi Stephen,

>
> On Thu, Sep 16, 2010 at 4:31 PM, Stephen <sdea...@gmail.com> wrote:
>
> > On Sep 15, 2:33 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> > wrote:
> > > Hi,
>
> > > We don't throttle apps.
>
> > What does 'throttle_code' mean?
>
> As I expanded immediately after the one sentence you clipped out of my
> reply, we decide to schedule additional instances of your app based on its
> user-facing latency.

Yes, I read that.

But what does 'throttle_code' mean when it appears in your logs?

http://groups.google.com/group/google-appengine/browse_thread/thread/fd648f4b59281b0b/e7f664260536c721

Does it mean that due to too many requests taking > 1000ms a request
which otherwise would have started a new instance was throttled? If
so, that's useful info.

[johnP]

"We decide to schedule additional instances of your app based on its
user-facing latency. "

This very material fact is not even mentioned in the article 'Best
practices for writing scalable applications'
http://code.google.com/appengine/articles/scaling/overview.html

johnP

On Sep 16, 9:03 am, Stephen <sdea...@gmail.com> wrote:
> On Sep 16, 4:33 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> wrote:
>
> > Hi Stephen,
>
> > On Thu, Sep 16, 2010 at 4:31 PM, Stephen <sdea...@gmail.com> wrote:
>
> > > On Sep 15, 2:33 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> > > wrote:
> > > > Hi,
>
> > > > We don't throttle apps.
>
> > > What does 'throttle_code' mean?
>
> > As I expanded immediately after the one sentence you clipped out of my
> > reply, we decide to schedule additional instances of your app based on its
> > user-facing latency.
>
> Yes, I read that.
>
> But what does 'throttle_code' mean when it appears in your logs?
>

> http://groups.google.com/group/google-appengine/browse_thread/thread/...

[Martin Webb]

Nick - Thanks for your reply - i agree with that. In honest i started coding in 1978 and building code that runs fast is what i grew up on - in those days it was cycles too, when we built games we had to made the code run in the time it took the TV set to go from the bottom of the screen and back again to the top - i think it was about 50cycles.In those days every single command and sum counted. We'd spend hours speeding up code. If you were super cool you would make your code run in the time it took the raster to leave the right side of the screen and enter the left.That's how we made 8 space invaders make 64. I like the idea of building fast code - its something microsoft stole from the art of programming. Perhaps those days are back again. I just hope my new app when i launch it runs at all - maybe my programing skills will be put to shame when i start getting hundreds of requests per second.
Then again maybe the tricks i learnt as a kid may come in handy again.

Thanks again for your reply - im feeling quite excited now!

Martin Webb

[Ikai Lan (Google)]

I'll add an article about backgrounding and keeping web requests fast to my to-do list:

http://code.google.com/appengine/articles/scaling/overview.html