Need documentation of various limits/quotas regarding concurrent requests

[Markus]

Hi,

first of all, let me say that I think AppEngine is one of the greatest
ideas ever. I love the concepts behind it; even some of the
restrictions it comes with that allow it to be so scalable.

Unfortunately, though, I keep running into random undocumented limits,
that force me to rewrite my code frequently, or to even abandon GAE
for certain projects altogether.

Right now, I am fighting with concurrent requests and have found
several limits that I didn't see documented anywhere and I am not sure
whether I should be seeing them, or whether I am doing something
wrong. So I am hoping one of the Google engineers is available and can
give some background info regarding some of the limits/quotas/strange
behaviors I found with one of my test apps:

- Frontend instances can handle only about 30 requests/s (qps). (I've
seen that number mentioned in posts somewhere, but I can't find it in
the docs.)

- Frontend instances keep getting created by the scheduler "randomly",
sometimes already when I only have 2-3 requests pending, even if I set
the respective Application settings to max 1 idle instance and max 15s
request latency.

- Backend instances can handle only 10 requests/s (qps), independent
of instance class. (I haven't seen this mentioned anywhere.)

- Backend instances can only have 6 requests open simultaneously,
independent of how I set max-concurrent-requests. (Someone mentioned
this in a post in 2009 for a frontend instance, but I can't find any
infos on backends.)

- My backend sends "503 Service unavailable" errors randomly, if there
are too many requests pending. Unfortunately I can't get a number on
when this happens (how many requests is too many). Also, I found this
surprising since I thought that the whole point of backends was to not
have these sorts of limits.

Any light anyone can shed on these questions would be much
appreciated.

I am most interested in the <max-concurrent-requests> setting in
backends.xml. Should this do something? I guess if I had written GAE
(I wish), this would specify the size of the thread-pool for request
servlets? Is there a limit/quota for how high I can set this number?

Are there any differences in these limits between paid and free apps?
How about Python vs. Java?

Do the 9 hours per day of free backend use also apply for paid apps?
(Or more general: are all free quotas still free for paid apps?)


And, I guess, the question that non-Google-engineers can answer to
help me in this quest is:

- Has anyone been able to exceed any of the above mentioned limits? If
so, do you know how?


Thank you very much for your help,

Markus



PS: Is there a chance we can get all quotas/pricing information on a
single page? Currently there are:
- http://code.google.com/appengine/docs/quotas.html
- http://code.google.com/appengine/docs/billing.html
- http://www.google.com/enterprise/cloud/appengine/pricing.html
- http://code.google.com/appengine/docs/java/backends/overview.html
(at the bottom)
- the Dashboard of the app itself
- and potentially many others that I haven't found yet

[Brandon Wirtz]

You aren't understanding the scheduler. There are a number of good posts in
this forum on the topic. Check the archives.

[Markus]

I have been looking through the archives, but maybe with the wrong
keywords. Maybe you have a couple of pointer for me which posts I
should look at?

[Jeff Schnitzer]

I suspect one of the missing pieces of information is that multithreaded Java instances can only handle 10 threads right now (or so I have been told).

Jeff

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Markus]

Ouch! Really??? This is a MAJOR deal-breaker for one of my projects.

But if that's the case, what good is <max-concurrent-requests>? Is it
really only there to limit the number of requests even further?
Wouldn't it be brilliantly sensible and useful to use this setting to
specify the size of the thread pool?

Do you know if this is a soft-limit? I.e. could I request an increase?
With the app I'm working on, I'd love to have many 10s of thousands of
requests simultaneously locked in a wait() on a single backend. This
should consume virtually no CPU and probably not much memory either.
But with 10 threads (i.e. 10 requests) per instance, there is no way
this would work.

In case you guessed it: I am indeed trying to build my own much
improved version of the Channels API.

Thank you for the data-point. This might also explain the 6-request-
limit for my backend, if there are 4 other things going on in parallel
that I didn't initiate.

Markus

> > -http://code.google.com/appengine/docs/java/backends/overview.html

[Robert Kluin]

Why are you doing this rather than using a service such as beaconpush.com?

As for wanting limits thoroughly documented, I could not agree more.
This is some thing that should be done, and I frankly can't understand
why it isn't. I would like to see something like:
http://code.google.com/p/googleappengine/issues/detail?id=5677

Along the same lines, it is very annoying to hit temporary
(undocumented!) quotas, such as datastore operations per minute, then
have no record of exactly what quota was hit. I'd love to see a daily
log of *exactly* every quota an app hits with enough detail for me to
tell Google what I'm hitting:
http://code.google.com/p/googleappengine/issues/detail?id=6296

Robert

[Jeff Schnitzer]

Someone at Google should confirm, but I believe this falls outside the design parameters of GAE.  Remember that your appserver is not directly listening for client requests, rather it goes more like this:

 * Some piece of google infrastructure takes the client request (in full)

 * The request is submitted to to your appserver

 * Your appserver generates the response (in full)

 * This completely buffered response is sent back to the listener

 * The listener sends the response back to the client

I speculate that the channel service is built around long polling (or websockets or whatnot) with specialized listeners, but regular requests are optimized for quick request/response cycles.

Maybe in the future Google will allow backends to block waiting on thousands of requests, but I wouldn't wait for it.  You may wish to move this piece of infrastructure to another part of the cloud.  Also:  You'll likely get much better capacity out of NIO, Node.js, or some other nonblocking system.  I'm sure JVMs are much better about thousands of threads than they used to be, but NIO and thread pools are still the right tools for the job.

Jeff

[Johan Euphrosine]

Also worth noting that the Go runtime while being single threaded can handle multiple http requests concurrently:

- each incoming request is handled by a goroutine

- if the goroutine currently being executed is blocking on I/O, execution yield to another goroutine.

This is similar in practice to what other non-blocking system are achieving with single threaded event loop and callbacks.

Hope that helps.

Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations

[Markus]

I have pretty much come to the same conclusion. For this I think I'll have to go to AWS. I just wanted to make sure that I didn't miss any clever little detail that would suddenly make it all possible and so much more elegant on AppEngine. This has happened to me before several times, especially when it came to transactions, queries, etc. in the datastore.

None-the-less, a limit of 6 concurrent requests on a back just seems ridiculously little, no? Especially if you take into account the fact that backends were specifically designed for long-running requests. Let's say you only use the backend to quickly compile a report for the client by spidering a whole bunch of pages, i.e. long time, but virtually no CPU use, you could only service 6 of these requests simultaneously, unless you want to somehow slice up the tasks and write your own scheduler to interleave their execution...

Backends seem like they aren't fully thought through yet...

[Markus]

Now THAT could actually be a real alternative, then! (Maybe I have to learn Go next?)

Two pieces of information that would be needed to ensure that this would be viable:

• Do you know if Go still has a limit on the number of simultaneous requests, though, even if not determined by the thread-count?
  
• Is there a way to have a Go routine "block" on a Semaphore or something until another Go routine signals it to continue?

Thank you!

[Jeff Schnitzer]

On Wed, Feb 22, 2012 at 11:23 AM, Markus <mca...@gmail.com> wrote:

Backends seem like they aren't fully thought through yet... At least not as much as the rest of GAE.

This has long been my complaint.  Pricing for meaningful quantities of RAM are absurd (10X anywhere else) which means they're useless for in-memory indexes.  Backends aren't reliable enough to be used for multiplayer game state.  Near as I can tell, the only real advantage of a backend is that it can exceed the 10m task deadline, which might make certain kinds of aggregation easier.  Although since in-progress queries expire after about a minute, even this is a PITA.

I have yet to find a use case for backends that isn't dramatically cheaper to run on Elastic Beanstalk using the remote API.

Jeff

[Johan Euphrosine]

On Wed, Feb 22, 2012 at 5:20 PM, Markus <mca...@gmail.com> wrote:

Now THAT could actually be a real alternative, then! (Maybe I have to learn Go next?)

Two pieces of information that would be needed to ensure that this would be viable:

• Do you know if Go still has a limit on the number of simultaneous requests, though, even if not determined by the thread-count?
  

You should ask google-appengine-go@ group, since the runtime is still experimental discussions are held in a language specific forum.

• Is there a way to have a Go routine "block" on a Semaphore or something until another Go routine signals it to continue?

That's what channels are for.

 

Thank you!

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/FYqcVduKjx4J.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Markus]

interesting... beaconpush actually offers pretty much exactly what I'm looking for.

There's just one big problem, that the Channels API (and probably beaconpush as well) have for me: I'm working on a security-focused service, and the last thing that I can afford to do is have my page open random non-https requests to any domains other than the one from the main page. Especially not TALKGADGET.google.com. Even though "beaconpush.com" already sounds a lot less gossipy, I still don't think it would be a good idea. At least I would find such a page highly suspicious if I cared about my data being safe with them. Even including external javascript libraries is something I'd really like to avoid.

[Robert Kluin]

Hey Markus,
We actually have *exactly* the same problem with Channel API.
There's no way for us to use it since it is going to
talkgadget.google.com.

If I remember correctly, beaconpush, or one or more of their
competitors, actually allows you to use your own domain. It will look
a lots less suspicious if it is hitting "service.yourdomain.com."

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.

> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/3SUBVUny9a0J.

[Markus]

Hey Robert,

what a small world! Maybe we should post a feature request to the channels api people.

Do you know if it's possible with any of these other solutions to host their javascript libraries from your server as well? Preferably in a way that you won't have to check every hour whether they updated their server-side code to break compatibility with your version of the client library?

Which solution did you end up using?

Thanks,

Markus

[Markus]

It's really too bad, that AWS doesn't use the same datastore. Or do they maybe have something similar? :)

Markus

[Robert Kluin]

Hey Markus,
I've looked at using the Channel API or third party solutions in
several projects. In one I went with the Channel API to test it out a
little. I've still not made heavy use of it though. I've looked at
the third party solutions a bit, but have not used one in a production
app. I'm sometimes very lazy, and in these cases old-fashioned
polling was sufficient. So, I'm not sure about self-hosting the
requisite javascript libs. I'd love to hear about your thoughts if
you dig into one though.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/41i3jv2Y5b0J.