django and amazon web services

[Robin Lery]

Hello.

I have a very basic noob question, which I don't know. Please help me understand this concepts. When I use ec2 for my application web server, and if I use the services like autoscale and elastic load balancer, do I always have to sync the database with the new ec2 instances that are created? Or its not required? Your help and guidance will be greatly appreciated.

Thank you!

Robin

[Russell Keith-Magee]

Hi Robin,

In the simplest deployment case, your entire Django (or, for that matter, any other framework) website will run on a single server. That single server will contain the web server, the database, any files that have been uploaded, any any other services your site needs (such as memcache, a mail server, or anything else)

Amazon EC2 is essentially selling you "a server". So, if you spool up a single EC2 instance, you could install all the bits you need to run your site on that single server.

If you're using autoscale and elastic load balancer, what you're getting is the ability to spool up new web servers as traffic increases.

However, when you move to having more than one server, you have a problem. What do you do with your database? No matter how many web servers you have, your application still needs to have a *single* database. 

Django's syncdb command is the step of creating the database schema (i.e., the collection of tables) that your application will run. You should only be running it once for your application -- on the single database. 

If you're setting up load balancing and autoscaling for your web app, you have three options for your database.

Firstly, you can set up a *separate* EC2 instance for your database. The idea here is that you would have multiple small EC2 instances serving web traffic (scaling up and down as required), but a single large EC2 instance running your database. 

Secondly, if you're looking for redundancy, you might set up *multiple* large EC2 instances for your database, and link them in a master-slave configuration. This provides some redundancy (in case one of the database servers goes down), and also provides some load balancing capability. Again, these instances are independent of your web server instances, and aren't affected by any load balancing behaviour.

Thirdly, and probably the best option if you're looking to deploy on Amazon - use Amazon RDS. This is Amazon's autoscaling database-as-a-service. You create a database on RDS, and get charged for how much data you store, and how much you serve. Amazon worries about scaling, backup, and all the rest.

In any of these three cases, you only run syncdb once -- you only ever have *one* User table, and you only need to run syncdb once to get it.

I hope that clarifies things.

Yours,

Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CA%2B4-nGpSoxqr6ff5-D%2B-%2BGvLEADU8RW%3D%3DTSdepk9daWSY6zFNw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Robin Lery]

Very nicely explained! I was actually hoping to use Amazon RDS  with postgresql. Your help was very useful. Thank you! 

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJxq84_miF9N8r1e5gMC-Ee3dq0m-vwkc0%3DDzVv1Dt9%2BWFS7vQ%40mail.gmail.com.

[CLIFFORD ILKAY]

On 03/01/2014 11:21 AM, Robin Lery wrote:
> Very nicely explained! I was actually hoping to use Amazon RDS with
> postgresql. Your help was very useful. Thank you!

There is another issue if your application has to deal with user
uploaded files, such as photos, PDFs, etc. You must consider the
instances in an autoscaling cluster to be volatile because they can come
and go automatically. Consequently, you must have a way of ensuring that
either the filesystems of those instances are synchronized or store user
contributed files on Amazon S3. Fortunately, we have pluggable storage
in Django so it's easy to do the latter.

If you ever have to deploy an application that doesn't have that, as we
did, we ended up creating an NFS cluster in a VPC (Virtual Private
Cloud) and the instances in the autoscaling group mounted the exported
NFS filesystem. The autoscale instances had two network interfaces, one
for the public Internet and another for the internal network in the VPC,
so the NFS cluster was not exposed to the Internet.

--
Regards,

Clifford Ilkay

647-778-8696

Dinamis

<http://dinamis.com>

[Robin Lery]

Thank you Clifford for the heads up and your advice! 

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/53123CF0.9030005%40dinamis.com.