Executing ssh scripts with django

[Marc Aymerich]

Hi guys, 

I'm developing an ISP control panel for my organization. I just finished the web interface based on django admin and now it's time to introduce calls to the "system scripts" in order to make the changes effective on the ISP servers (add users, manage virtualhosts, domains, and all this stuff). To achieve that I was thinking in overriding the save and delete methods from my models ( virtualhost model, domain model..). For example. before saving a new "apache virtualhost" I want to run a proper "create vhost" script through ssh using Paramiko library and if it is successful save the new virtualhost into the database, otherwise send a message to enduser telling that an error has occurred. 

class Virtualhost(models.Model):

    ....

    def save(self, *args, **kwargs):

         if not self.id:

             ssh = Ssh('create_new_virtualhost', self)

             if ssh.errors: 

                  message.add(self.user, 'something wrong was happend')

             else:

                   super(self, Virtualhost).save(*args, **kwargs)

I'm wondering if this approach is the right way for the interaction between django and ISP servers. 

Moreover, I read that is highly recommended to use a message queue like celery in order to avoid a possible 'long wait' until ssh command ends [1]. Would you execute the save function through celery? Is it safe? or maybe if save() is executed asynchronously it can cause some unexpected behavior?

I need some "expert" opinion here :) Would you affront this situation in a similar way or would you take a completely different approach?

Many thanks!!

[1] http://www.quora.com/How-can-I-remotely-execute-a-script-via-SSH-from-a-Django-view

--
Marc

[Knut Ivar Nesheim]

Hi,

I would strongly suggest looking into using Celery or some other form
of message queue.

In general you want to decouple the logic of the web app, such as
validating domain names, writing to the database, with operations that
has real-world important side effects. Once you split these, the
system as a whole is much easier to understand for newcomers, much
easier to reason about and above all, actually possible to test.

The operations that has side effects on your other systems is thus a
separate part of your system. That way you can test them in isolation
and you may even run a failed operation several times, without
requiring work from the user. You could also batch the operations if
necessary.

A separation like this requires some a new concept for communication
with the user. This can be as simple as "You just added a new virtual
host. We will send you a confirmation when the host is active."

As a side note, I would strongly recommend writing your operations in
such a way that you may run the same operation more than once and
achieve the same result. Otherwise you will run into nastiness. And if
you want to update a virtual host, just run the "add_virtualhost"
operation again, but this time with updated data.

You could also look into using a configuration management system, like
Chef. You could then have your system create Chef recipes.

Regards
Knut

> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>

[Marc Aymerich]

Hi Knut, many thanks for your recommendations, I'll follow them!. All you've comment has a lot of sense :) 

I've never heard before about configuration management systems. Now I just read something about it on wikipedia and seems to be useful when you have a lot of similar servers. In my organization we have only one web server, one DB server and one primary mail server. Does the use of CM make sense on my scenario somehow?

Thanks!

marc.

--
Marc

[Andrej]

> I'm developing an ISP control panel for my organization.

a bit off topic on Django and ISP. Look at the NOC Project
http://www.nocproject.org/

> NOC is an Operation Support System (OSS) for the Telco,
> Service provider and Enterprise Network Operation Centers (NOC).
> Written in Python language and utilizing the power
> of Django framework and PostgreSQL RDBMS .

[Knut Ivar Nesheim]

On Mon, Nov 1, 2010 at 3:17 PM, Marc Aymerich <glic...@gmail.com> wrote:
> I've never heard before about configuration management systems. Now I just
> read something about it on wikipedia and seems to be useful when you have a
> lot of similar servers. In my organization we have only one web server, one
> DB server and one primary mail server. Does the use of CM make sense on
> my scenario somehow?

The great benefit about these systems is that you can go from a
freshly installed machine, to a machine with all the necessary
packages, services and configuration all in place, without any manual
work other than starting the configuration management system.

There is a big initial investment to start using these systems, and
everything you do to your machines you need to do through the tool.
For your scale this might not be a good investment.

Another benefit of systems like Chef is that the configurations is
plain text, so you can use your normal development tools, like version
control, peer review, etc. It becomes very easy to reason about the
proposed change, when you have a diff at hand.

Regards
Knut

[Sævar Öfjörð]

> Would you execute the save function through celery? Is it safe? or maybe if
> save() is executed asynchronously it can cause some unexpected behavior?

You could use signals to trigger execution of celery tasks, e.g. the
post_save signal:
http://docs.djangoproject.com/en/dev/ref/signals/#django.db.models.signals.post_save

- Sævar