GAE scalability issues & questions unresolved

[pir...@gmail.com]

I am a newbie to Python but not so to web application development and
scalability concerns. I am looking for some better understanding of
the GAE Web Scalability software and hardware architecture and how
requests are managed.

Here are some concerns and questions from that.

urlfetch.fetch(url) from urlfetch.py states """Fetches the given HTTP
URL, blocking until the result is returned.

1> Doesn't this mean that my app freezes (for instance) while calling
a web service? What is the web application service model for users
hitting my site? If I have a very heavy SOA (100 service calls to
generate a page for instance) will that not be a big problem?

A> Does GAE queue requests?
B> How many processing threads or servers are dedicated to a single
web app (myapp?)
C> How does this blocking impact GAE and what does it mean when I
have 100k users hitting my web site?

2> In the same vein regarding urlfetch.py code. In Socket base class
there's a 60 second default timeout and there does not appear to be a
way to override the socket timeout (looking over the code for hours
and hours.) I have looked from the code for urlfetch.fetch(url) to
httplib.py down to socket.py and see no way to change it so I would
consider this a big miss that needs to be corrected (or perhaps it's
just me that needs correcting.)

Unresolved local developer proxy issues with urlfetch.fetch(url)

3> No way to set a proxy server for the local developer environment
which essentially means that no one that works inside Intel.com can
make use of these tools. Which means exclusion of 100,000 plus
potential Intel persons at work that might want to use GAE for
possible solutions. For that matter if you can't make this method
work from behind a firewall for local development then you eliminate
millions of developers around the world or your essentially saying
that they all need to work from home (heck I kind of like that idea
though.)

Where can I find some good core details regarding the hardware and
software architecture that GAE is running on and to what extent
globally is my application available?

[Brett]

On Apr 13, 5:50 pm, "pir...@gmail.com" <pir...@gmail.com> wrote:
> I am a newbie to Python but not so to web application development and
> scalability concerns. I am looking for some better understanding of
> the GAE Web Scalability software and hardware architecture and how
> requests are managed.
>
> Here are some concerns and questions from that.
>
> urlfetch.fetch(url) from urlfetch.py states """Fetches the given HTTP
> URL, blocking until the result is returned.
>
> 1> Doesn't this mean that my app freezes (for instance) while calling
> a web service? What is the web application service model for users
> hitting my site? If I have a very heavy SOA (100 service calls to
> generate a page for instance) will that not be a big problem?

Yes 100 service calls to generate a page most likely will not work,
depending on how fast the target webservices are.

> A> Does GAE queue requests?
> B> How many processing threads or servers are dedicated to a single
> web app (myapp?)
> C> How does this blocking impact GAE and what does it mean when I
> have 100k users hitting my web site?

As mentioned during Campfire One, we use a very scalable
infrastructure to automatically load-balance your application as it
receives more requests. Some of this is covered here:

http://code.google.com/appengine/docs/whatisgoogleappengine.html

> 2> In the same vein regarding urlfetch.py code. In Socket base class
> there's a 60 second default timeout and there does not appear to be a
> way to override the socket timeout (looking over the code for hours
> and hours.) I have looked from the code for urlfetch.fetch(url) to
> httplib.py down to socket.py and see no way to change it so I would
> consider this a big miss that needs to be corrected (or perhaps it's
> just me that needs correcting.)

To clarify: You'd like to have a deadline parameter for the urlfetch
service, right? Please file an issue:

http://code.google.com/p/googleappengine/issues/list

> 3> No way to set a proxy server for the local developer environment
> which essentially means that no one that works inside Intel.com can
> make use of these tools. Which means exclusion of 100,000 plus
> potential Intel persons at work that might want to use GAE for
> possible solutions. For that matter if you can't make this method
> work from behind a firewall for local development then you eliminate
> millions of developers around the world or your essentially saying
> that they all need to work from home (heck I kind of like that idea
> though.)

You can give others access to the local development environment by
using the flag: --address 'your ip'

By default it's local-only for security.