Automatic deployment tool

[Jonathan Slenders]

Hi Everyone,

In the past there have been some discussionh about how to deploy Django web applications through SSH. How to use Fabric or other tools, and whether we should provide or maybe force users to deploy applications according to a certain conventions.

Back then, maybe already more than a year ago, I said that I was working on my own deployment tool. [1] Something that could be used instead of Fabric. It's a tool which could probably help a lot of you, although it can take a while to adopt. The core consists of high quality code. I took me a while before I felt confident enough for releasing this, and it has been refactored more then any project I did before. Things still can be improved, but it's ready to share with you.

Key features are:

• Interactive execution of remote commands. Locally, they will appear in a pseudo terminal (created with openpty), so that even editors like Vim or Emacs works fine when you run them on the remote end. You can easy start an interactive shell on any host as well.
• Reusability of all deployment code is a key point. It's as declarative as possible, but without loosing Python's power to express everything as dynamic as you'd like to. Deployment code is hierarchically structured, with inheritance where possible.
• Parallel execution is easy when enabled, while keeping interaction with these remote processes possible through pseudoterminals. Every parallel task gets his own terminal, either a new xterm or gnome-terminal window, a tmux pane, or whatever you'd like to.
• Logging of your deployments. New loggers are easily pluggable into the system.

So, enjoy!

So, what does it have to do with Django? I have a setup-definition of what we use for Django deployment [2]. However, I suppose that quite a lot of people aren't using uwsgi like us. So, I'd like to know what the most common use cases of Django deployment are. If I can cover most cases, it's very easy for end-users to pick one, override what they don't like, and enjoy the full power of this deployment system.

For instance, to demonstrate the power. If we want to connect to a Django shell_plus of our Mobile Vikings production system, we type in the interactive shell:

    > mobile_vikings django shell_plus

This will call the shell_plus function of our django setup, it will ask on which host the shell needs to be started, and immediately fire an interactive shell_plus of the remote server in your terminal.

[1] https://github.com/jonathanslenders/python-deployer

[2] https://github.com/citylive/citylive-operations/blob/master/deployment/deployer/contrib/services/django.py

I'll publish one of these days on pypi.

All feedback is welcome. For bugs/feature requests on things which arn't Django related, please go to the github.

Cheers,

Jonathan

[Cal Leeming [Simplicity Media Ltd]]

Hi Jonathan,

Just from a very brief point of view.. my eyes started to glaze over whilst looking at the github README, and even more so when I looked at the code.

Even if this was the best thing since sliced bread, the documentation in its current state leaves me with the feeling of "why do I want to use this".

I think what would benefit this project massively is good/easy to read documentation, with a simple overview section explaining common uses, what makes it better than alternatives, etc.. maybe via readthedocs..?

Statements such as "It's as declarative as possible.." sound impressive, but don't really give me much insight into what this is, and why I'd want to use it.

Hope this helps!

Cal

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/k4RS_9Kmn9cJ.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Jonathan Slenders]

Thanks for your feedback, Cal,

You're right about the documentation, some very useful parts aren't even documented at all. (There are comments in the code, but there are some little things you wouldn't know from the readme.)

During the last years I also put most of the effort in experimenting and getting something really useful. It took a while before I got there, and it didn't make sense to document features which were to be refactored every now and then. However, now I feel its quite stable. I should start making better documentation, and a lot of usage examples. I also cannot deny that the learning curve may be a little bit steeper then Fabric in the very beginning, but soon you will see the advantages. If only I were as good in selling a project as I can code. :)

Anyway, I hope this can also improve automatic deployment of Django applications for other people.

Cheers,

Jonathan

[Cal Leeming [Simplicity Media Ltd]]

On Sun, Dec 9, 2012 at 11:40 PM, Jonathan Slenders <jonathan...@gmail.com> wrote:

Thanks for your feedback, Cal,

You're right about the documentation, some very useful parts aren't even documented at all. (There are comments in the code, but there are some little things you wouldn't know from the readme.)

During the last years I also put most of the effort in experimenting and getting something really useful. It took a while before I got there, and it didn't make sense to document features which were to be refactored every now and then. However, now I feel its quite stable. I should start making better documentation, and a lot of usage examples. I also cannot deny that the learning curve may be a little bit steeper then Fabric in the very beginning, but soon you will see the advantages.

 

If only I were as good in selling a project as I can code. :)

I know that feeling!!!!

 

To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/5fPxtSy12ysJ.

[Jonathan Slenders]

Hi all,

Some improvements have been made and documentation is coming.

For the curious, here is a 5min screen recording of me using it on our deployment environment that I'll present this weekend on djangocon-europe. You can watch it here:

https://vimeo.com/66302371


It shows:

- The hierarchical architecture of the deployment tree. (navigation through the nodes.)
- Interacting with remote redis shell and the django shell. (some of our deployment services.)
- source code introspection
- sandboxing of deployments.
- one complete (sandboxed) setup of our distributed application.

What's not shown, but great:
- typing "--connect" will open a bash shell on the remote end of the current node.
- introspection of everything
- a lot more.

Feedback is very welcome, but maybe keep it off django-developers.

https://github.com/jonathanslenders/python-deployer/

Enjoy!

Jonathan