Advice on deploying a Django App on AppEngine

[Luca de Alfaro]

I am struggling with the decision of where to deploy a site I am building with Django. 

I am very keen on App Engine, and I have been reading about django-nonrel, so that seems to be a very appealing route. 

However, I have a few questions: 

1.  Is it advisable to wait until the 3000-file-limit issue is solved before deciding for AppEngine?  It seems the standard Django 1.3 is very close to the limit, not sure about django-nonrel.  My app has something like 50-100 files right now.  I understand a fix is in progress. 

2.  I read somewhere in the past that Django takes a long time to start on AppEngine.  Is this still true now?  Is this a serious consideration?  I tend to think not, since even other scalings (bringing up more machines on EC2, ...) are not instantaneous.

3.  A typical request causes several hits to the datastore; typically at least some access to a permissions table (to see if the user can perform a given operation), followed by one or more accesses to actually carry out the request.  Will this work relatively well?  I am worried that if the latency to the datastore is high, since Python on AppEngine is single-threaded, then a single AppEngine instance will be able to serve considerably fewer qps than e.g. a RackSpace Cloud virtual machine, where the latency to the db could be lower.  Any insight on how many AppEngine instances are equivalent to, say, a Rackspace Cloud VM? 

Thanks!! -- Luca

[Robert Kluin]

Hi Luca,
Responses inline.

On Wed, Aug 17, 2011 at 13:35, Luca de Alfaro <luca.de...@gmail.com> wrote:
> I am struggling with the decision of where to deploy a site I am building
> with Django.
> I am very keen on App Engine, and I have been reading about django-nonrel,
> so that seems to be a very appealing route.
> However, I have a few questions:
> 1.  Is it advisable to wait until the 3000-file-limit issue is solved before
> deciding for AppEngine?  It seems the standard Django 1.3 is very close to
> the limit, not sure about django-nonrel.  My app has something like 50-100
> files right now.  I understand a fix is in progress.

There is always zip import if you need it while you're waiting for
them to raise the limit.

> 2.  I read somewhere in the past that Django takes a long time to start on
> AppEngine.  Is this still true now?  Is this a serious consideration?  I
> tend to think not, since even other scalings (bringing up more machines on
> EC2, ...) are not instantaneous.

Yes, Django will take longer to load because it is an extremely heavy
framework. Yes, I would personally say it is an important
consideration. App Engine instances get brought up and down
significantly more often than a typical 'VPS' type hosting
environment.

> 3.  A typical request causes several hits to the datastore; typically at
> least some access to a permissions table (to see if the user can perform a
> given operation), followed by one or more accesses to actually carry out the
> request.  Will this work relatively well?  I am worried that if the latency
> to the datastore is high, since Python on AppEngine is single-threaded, then
> a single AppEngine instance will be able to serve considerably fewer qps
> than e.g. a RackSpace Cloud virtual machine, where the latency to the db
> could be lower.  Any insight on how many AppEngine instances are equivalent
> to, say, a Rackspace Cloud VM?

You should use Appstats to optimize your requests anywhere possible.
You can often optimize by implement batch-fetch/put strategies.
However, having a couple datastore fetches / puts in a request should
not generally be an issue. Make sure you are using the
high-replication datastore for a new app.

You simply can not compare App Engine instances to anything else
really. AE instances are much much lighter, but very restricted. I
see well under 1 qps / instance even on apps that serve requests in
around 100ms; under heavier load (1500+qps) I've seen around 2
qps/instance with 100ms average latency. With any luck improvements
to the scheduler will get that up to at least 6 qps / instance, and
hopefully threading will further improve that, but at this point
you'll use far more App Engine instances.

Robert

> Thanks!! -- Luca
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/SFjHAhSnZ40J.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>