Deploying Django on Docker
201 views
Skip to first unread message
Anssi Kääriäinen
Jan 8, 2015, 5:45:47 AM1/8/15
to django...@googlegroups.com
For a long time I have been battling with the following problem: how to deploy Django in an easy, maintainable, secure and reliable way for small Django applications. The applications I write are mainly very low traffic, and often they are coded in a couple of days. Unfortunately the hardest part of producing these applications has been deployment. A big part of the problem is that I have to work with RHEL6 based servers, and those tend to have old libraries.
The solution I am currently using is a nice little setup helper I wrote, django-dockerfile: https://github.com/akaariai/django-dockerfile. The idea is that you write a environment file for each server you want to deploy into, and after that everything happens automatically through usage of fab.
The solution works great for me even if there is still a lot to do. For example, usage of the fig package could make the code more robust, and allow for multi-server installations.
The main reason I am writing here on django-users is that I would like to see an easy way to deploy small Django applications. For large applications it seems OK to just build up your own deployment strategy, but for small applications it is too easy to end up doing the deployment in a hackish way where for example deploying the application isn't scripted, and each application's deployment strategy varies slightly.
I am looking for a way to deploy Django with at least these features:
- Allows one-command deployment directly from Git to Docker-enabled server
- Easy to use and get started
- Doesn't use development server, or SQLite as database.
- Does care for security
- Media and static files served properly (note: CDN is overkill for my projects!)
- Allows one to define secrets and other environment variables per installation
- Preferably: logging, backup and restore thought out
The django-dockerfile package has the above features except for logging, backup and restore.
So, if there already exists something that has the above features implemented, then please tell me. If not, I am looking for interested developers to participate in building a tool for the small Django project deployment use case. The django-dockerfile package could be used as basis for the work, but using some other package or just starting from scratch are also options. Just writing a blog post that has information about an example Django deployment with every aspect of the above feature list covered could be enough.
- Anssi
The solution I am currently using is a nice little setup helper I wrote, django-dockerfile: https://github.com/akaariai/django-dockerfile. The idea is that you write a environment file for each server you want to deploy into, and after that everything happens automatically through usage of fab.
The solution works great for me even if there is still a lot to do. For example, usage of the fig package could make the code more robust, and allow for multi-server installations.
The main reason I am writing here on django-users is that I would like to see an easy way to deploy small Django applications. For large applications it seems OK to just build up your own deployment strategy, but for small applications it is too easy to end up doing the deployment in a hackish way where for example deploying the application isn't scripted, and each application's deployment strategy varies slightly.
I am looking for a way to deploy Django with at least these features:
- Allows one-command deployment directly from Git to Docker-enabled server
- Easy to use and get started
- Doesn't use development server, or SQLite as database.
- Does care for security
- Media and static files served properly (note: CDN is overkill for my projects!)
- Allows one to define secrets and other environment variables per installation
- Preferably: logging, backup and restore thought out
The django-dockerfile package has the above features except for logging, backup and restore.
So, if there already exists something that has the above features implemented, then please tell me. If not, I am looking for interested developers to participate in building a tool for the small Django project deployment use case. The django-dockerfile package could be used as basis for the work, but using some other package or just starting from scratch are also options. Just writing a blog post that has information about an example Django deployment with every aspect of the above feature list covered could be enough.
- Anssi
Guilherme Leal
Jan 8, 2015, 5:56:13 AM1/8/15
to django...@googlegroups.com
Honestly, i dont have the time to participate on this project right now, but i would use a tool like this for small projects. FOR SURE.
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/eeb24be3-1f3d-4d4e-8ba0-7a476871c43a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mike Dewhirst
Jan 8, 2015, 6:20:39 AM1/8/15
to django...@googlegroups.com
On 8/01/2015 9:45 PM, Anssi Kääriäinen wrote:
> For a long time I have been battling with the following problem: how to
> deploy Django in an easy, maintainable, secure and reliable way for
> small Django applications. The applications I write are mainly very low
> traffic, and often they are coded in a couple of days. Unfortunately the
> hardest part of producing these applications has been deployment. A big
> part of the problem is that I have to work with RHEL6 based servers, and
> those tend to have old libraries.
>
> The solution I am currently using is a nice little setup helper I wrote,
> django-dockerfile: https://github.com/akaariai/django-dockerfile. The
> idea is that you write a environment file for each server you want to
> deploy into, and after that everything happens automatically through
> usage of fab.
Have you seen Mezzanine fab deployment? Ken Bolton has blogged about it ...
> For a long time I have been battling with the following problem: how to
> deploy Django in an easy, maintainable, secure and reliable way for
> small Django applications. The applications I write are mainly very low
> traffic, and often they are coded in a couple of days. Unfortunately the
> hardest part of producing these applications has been deployment. A big
> part of the problem is that I have to work with RHEL6 based servers, and
> those tend to have old libraries.
>
> The solution I am currently using is a nice little setup helper I wrote,
> django-dockerfile: https://github.com/akaariai/django-dockerfile. The
> idea is that you write a environment file for each server you want to
> deploy into, and after that everything happens automatically through
> usage of fab.
http://bscientific.org/blog/mezzanine-fabric-git-vagrant-joy/
Mike
> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users...@googlegroups.com
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> <mailto:django-users...@googlegroups.com>.
> To post to this group, send email to django...@googlegroups.com
> <mailto:django...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/eeb24be3-1f3d-4d4e-8ba0-7a476871c43a%40googlegroups.com
> <https://groups.google.com/d/msgid/django-users/eeb24be3-1f3d-4d4e-8ba0-7a476871c43a%40googlegroups.com?utm_medium=email&utm_source=footer>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/eeb24be3-1f3d-4d4e-8ba0-7a476871c43a%40googlegroups.com
Jeroen Bakker
Jan 8, 2015, 7:28:25 AM1/8/15
to django...@googlegroups.com
Hi Anssi,
Not really an answer to your question, but just to give you insight to a solution.
We at l1nda are using docker and django with small applications and large applications.
We developed an nginx container and uwsgi container and a devops container. The devops container does all the tricky parts and the nginx and uwsgi containers are basically simple ones.
We have automated the deployment docker (devops) image that internally uses docker-py that will minimize the downtime of the django containers. The script is just a python file of 100 lines that brings application down and up in a redundant way it also starts the staticscollecting etc. As it is pulling the images from another server (at our office), the security hurden is small.
1. Pull new images from our private repository
2. Collect statics, update databases etc.
3. loop per container [we run every application in 4 different containers]
3.1 stop old container
3.2 start new container.
We use a shared FS (NFS) for sharing the resources (uwsgi is allowed to write to the FS and the nginx is only allowed to read from the FS. The machine that contains the nginx image is also hosting the FS.
Jeroen
Not really an answer to your question, but just to give you insight to a solution.
We at l1nda are using docker and django with small applications and large applications.
We developed an nginx container and uwsgi container and a devops container. The devops container does all the tricky parts and the nginx and uwsgi containers are basically simple ones.
We have automated the deployment docker (devops) image that internally uses docker-py that will minimize the downtime of the django containers. The script is just a python file of 100 lines that brings application down and up in a redundant way it also starts the staticscollecting etc. As it is pulling the images from another server (at our office), the security hurden is small.
1. Pull new images from our private repository
2. Collect statics, update databases etc.
3. loop per container [we run every application in 4 different containers]
3.1 stop old container
3.2 start new container.
We use a shared FS (NFS) for sharing the resources (uwsgi is allowed to write to the FS and the nginx is only allowed to read from the FS. The machine that contains the nginx image is also hosting the FS.
Jeroen
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/eeb24be3-1f3d-4d4e-8ba0-7a476871c43a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
L1NDA B.V.
www.facebook.com/L1NDA.NL
Nieuwezijds Voorburgwal 101-III
Nieuwezijds Voorburgwal 101-III
1012 RG, Amsterdam
020-7526690
KvK: 57861765
Anssi Kääriäinen
Jan 9, 2015, 5:49:06 AM1/9/15
to django...@googlegroups.com
On Thursday, January 8, 2015 at 2:28:25 PM UTC+2, Jeroen Bakker wrote:
Hi Anssi,
Not really an answer to your question, but just to give you insight to a solution.
We at l1nda are using docker and django with small applications and large applications.
We developed an nginx container and uwsgi container and a devops container. The devops container does all the tricky parts and the nginx and uwsgi containers are basically simple ones.
We have automated the deployment docker (devops) image that internally uses docker-py that will minimize the downtime of the django containers. The script is just a python file of 100 lines that brings application down and up in a redundant way it also starts the staticscollecting etc. As it is pulling the images from another server (at our office), the security hurden is small.
1. Pull new images from our private repository
2. Collect statics, update databases etc.
3. loop per container [we run every application in 4 different containers]
3.1 stop old container
3.2 start new container.
We use a shared FS (NFS) for sharing the resources (uwsgi is allowed to write to the FS and the nginx is only allowed to read from the FS. The machine that contains the nginx image is also hosting the FS.
This solution seems excellent, but it is too heavy for those who just want to deploy single small application.
What I am looking after is a solution for guys who have just completed Django tutorial and now want to deploy their code. I have a hunch that way too often the solution is to use devserver + sqlite.
- Anssi
Anssi Kääriäinen
Jan 9, 2015, 6:00:59 AM1/9/15
to django...@googlegroups.com
On Thursday, January 8, 2015 at 1:20:39 PM UTC+2, Mike Dewhirst wrote:
Have you seen Mezzanine fab deployment? Ken Bolton has blogged about it ...
http://bscientific.org/blog/mezzanine-fabric-git-vagrant-joy/
Seems interesting. I have to play with it.
The biggest issues for me are that you use settings file directly to have your deployment description. Also, at least by default this solution doesn't work on Docker (though vagrant does seem to have Docker support, so it might be just a configuration issue).
What I did in django-dockerfile, and what I really like about django-dockerfile, is that server specific configuration is separated from settings.py. There is usually a base, secrets and then deployment environment files. The way this works is that base.env could contain the following:
{
"DATABASE_HOST": null,
"DATABASE_PASSWORD": null,
"DATABASE_USER": "someuser
}
Here null values denote that these settings must be overridden by other environment files.
secrets.env could contain:
{
"DATABASE_PASSWORD": "somepassword"
}
and then production.env has:
{
"from": ["base.env", "secrets.env"],
"DATABASE_HOST": "example.com"
}
Now, when using production.env, first variables from base.env are read, then vars from production.env override the DATABASE_PASSWORD variable, and finally the production.env overrides the DATABASE_HOST variable.
The project's settings.py uses environment variables always, so that there is no need for multiple different settings files.
The point is that it is very easy to have production, qa and other needed environments set up. Just create a qa.env and inherit from base.env. The code used for different environments is always the same, including the settings file which makes it easier to see what is actually going on with settings.
- Anssi
Anssi Kääriäinen
Jan 12, 2015, 1:24:39 AM1/12/15
to django...@googlegroups.com
I have given this issue a bit more thought, and it seems using Docker might be a bit too complex for the "just completed tutorial use case". Docker is nice, but if things do not work out the way you want, troubleshooting can get a bit complex.
Instead it might be good to *fully* document how to set up a virtual machine based Django setup. The documentation should include:
- minimal information about how to install virtualbox and Ubuntu 14.04 image. (Maybe vagrant would be a better idea?)
- how to setup Django with Gunicorn (is supervisord a good idea for process management?)
- static files
- media files
- security
- logging
- multiple environments (for example, how to use settings for multiple environments. environment variables are the way to go IMO)
- automatic deployment
- backup, restore, clone production to qa (clone to qa is a really useful feature if your database is small enough for it)
Second step would be to automate as much of the above list as possible. Maybe the mezzanine fab file is enough for that.
The above list also tells a pretty good story about why I pursue a documented way to fully set up Django. When setting up a small Django app, doing all of the above things properly requires just too much effort.
- Anssi
Instead it might be good to *fully* document how to set up a virtual machine based Django setup. The documentation should include:
- minimal information about how to install virtualbox and Ubuntu 14.04 image. (Maybe vagrant would be a better idea?)
- how to setup Django with Gunicorn (is supervisord a good idea for process management?)
- static files
- media files
- security
- logging
- multiple environments (for example, how to use settings for multiple environments. environment variables are the way to go IMO)
- automatic deployment
- backup, restore, clone production to qa (clone to qa is a really useful feature if your database is small enough for it)
Second step would be to automate as much of the above list as possible. Maybe the mezzanine fab file is enough for that.
The above list also tells a pretty good story about why I pursue a documented way to fully set up Django. When setting up a small Django app, doing all of the above things properly requires just too much effort.
- Anssi
Aaron C. de Bruyn
Jan 12, 2015, 12:21:42 PM1/12/15
to django...@googlegroups.com
My docker setup is pretty easy:
wget -qO- https://raw.github.com/progrium/dokku/v0.3.13/bootstrap.sh |
sudo DOKKU_TAG=v0.3.13 bash
cat ~/.ssh/id_rsa.pub | ssh ro...@mynewhost.mydomain.tld "sudo
sshcommand acl-add dokku myproject"
git remote add production do...@mynewhost.mydomain.tld:myproject
git push production master
Dokku is awesome. ;)
wget -qO- https://raw.github.com/progrium/dokku/v0.3.13/bootstrap.sh |
sudo DOKKU_TAG=v0.3.13 bash
cat ~/.ssh/id_rsa.pub | ssh ro...@mynewhost.mydomain.tld "sudo
sshcommand acl-add dokku myproject"
git remote add production do...@mynewhost.mydomain.tld:myproject
git push production master
Dokku is awesome. ;)
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/c6bd553e-f21e-4a4f-ba5a-150e58bac0db%40googlegroups.com.
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
Reply all
Reply to author
Forward
0 new messages