This is an interesting problem, which you can work around to a certain
On Thu, Nov 15, 2012 at 3:49 PM, rh <richard_hubb...
> In the djangocon lightning talk there was mention of this disconnect
> between "works on runserver" but not in production env.
> Not a problem restricted to django. I've been checking out a lot of
> frameworks in different languages and most suffer the same malady.
> i.e. everything works great locally but hauling it all into a production
> env. becomes a project in itself.
> It's hard to document because each env. will be different.
extent. Making predictable and reproducible deployments of code is an
integral part of Software Configuration Management. SCM is a big
topic, so I'm not going to go into crazy details, but here are some
details of how we mitigate these risks at $JOB
1) Use virtualenv and pip
Virtualenv is essential. It allows you to separate the libraries you
use in your deployment from the any other python package. I recommend
using "--no-site-packages" to force virtualenv to ignore any system
site packages, but sometimes it can be necessary.
Pip is the other side of this. Pip is a simple way to specify python
packages to install. You can explicitly reference versions of
packages, or ranges of packages. Eg, "Django>1.4,<1.5" specifies that
the most recent release in the 1.4 cycle is correct. You can also have
a single file that lists all the packages required for your project to
run, usually called 'requirements.txt'. Pip will take that file, and
install all listed packages along with any dependencies.
2) Your project structure is code, and should live in source control.
Your project structure is all the files and folders that make up a
deployment, but aren't actual python code. Eg, if your deployment for
django project 'myproj' looks like this:
│ ├── manage.py
│ └── myproj/
The 'myproj' directory contains the project source code and is usually
version controlled. However, everything listed there is part of the
project deployment, and changes there must be tracked. If your project
requires a 'logs' directory, then a logs directory must be a versioned
resource that you can check out at any point in it's past.
Just as importantly, any changes that you make in the project
structure in development must be replicated to the production
deployments, and therefore must be tracked.
3) Django apps are also python libraries.
A 'django app', regardless of what it does, is also a python library.
Therefore, make sure you give it a setup.py script using distutils
that will allow you to do standard python library packaging operations
on it. Pip will happily install an app from source control, it will
even install in a manner that allows you to edit and commit changes
back to source control.
This allows you to install packages in development in the same way as
4) Write a bootstrap script.
A bootstrap script is a simple shell script that does two things -
sets up a virtualenv environment, and installs a set of packages.
This combination of storing project structure in source control, using
virtualenv and pip, having a bootstrap script and having a
requirements.txt listing your required packages allows you to simply
checkout a project and run it's bootstrap script to end up with an
identical, repeatable project environment.
6) Use the capabilities of your source control to assemble a deployment
This is the tricky one where I tend to lose people! We use subversion
for source control. Subversion allows us to specify 'externals', which
are secondary subversion repositories that we want to be automatically
checked out in specific locations within another repository. You can
also have 'file externals', which is a single file (rather than an
entire repository tree) that is checked out at a specific location.
The benefit of this is that you can use your project structure
repository as a basis for a deployment on a single site, by
duplicating it for each server that you wish to deploy on.
On each one, you would have an external adding the project source code
in the right place. You would have a file external that pulls in a
'requirements.txt' file correct for this project into the right place.
You would have another file external that pulls in the correct
settings.py (or settings_local.py if that is what you prefer).
7) Use South
Use south for database migrations. If you aren't doing this already,
you should be! Schemas rarely stay static, you need good ways of
managing that change.
8) Use Fabric
Fabric is a clever system for writing scripts to do things on remote
servers. Work out the steps required to update your project source
code, libraries, deploy migrations, and codify it into a fabric
fabfile. Never make a mistake in deployment again!
There is more to it than this of course (there is a lot more to each
of these steps even!), but these simple ideas allowed us reliable and
predictable deployments and updates on existing boxes, one touch
deployments on new boxes, and generally make your life maintaining
websites and deploying updates easier and safer.