Best way to run a shell script, using input value from Django?

[Chris Robinson]

Hello, 

I'm going to attempt to generalize my question to make this easier to answer. 

Lets say I have a simple terminal based shell script (e.g. multiply_by_two) that takes a number argument, then multiplies that number by 2, and returns the result into a result.txt file. 

I would enter something like this:

multiply_by_two -n 6

The -n flag asks what number I would like to multiply by 2. The result of this would be a result.txt, containing the number 12 inside. 

What I would like to do is develop a simple Djano application that contains a text field allowing me to input the number 6, then click "Submit."

This job will start on the server by running my custom multiply_by_two application with my input parameter (6), but when the job is finished and the result.txt is available, the browser will automatically download the file. 

To make this a tad bit more complex, lets say that the job takes 2 minutes to run. What would be the best way to monitor the job? Maybe I accidentally close the window.

Not looking for anyone to solve this, I'm just new to Django and want to know if someone can give me any pointers on where to start. Are there any apps existing that will help me not need to write everything from scratch, especially if 'monitoring' is needed? Would Celery be ideal for this?

Thanks for any input!

Regards,

Chris

[Alex Heyden]

So, a lot of this doesn't have much to do with Django, but I'm not sure how familiar you are with web server best practices in general, so apologies if some of this is remedial for you.

If something takes two minutes to finish, you don't want it being handled on the request handler itself. Split the job off into a separate worker, then return something to the user. Ideally, that something will allow the user to continue polling for job updates.

Sounds simple, but the details can be obnoxious. First, you have to figure out how to track the job. Personally, I prefer to not have to watch a worker directly. I'd rather watch the results come through. For example, if your script takes two minutes because it's updating a few thousand items somewhere, I'd rather watch the items change by OS mtime or auto_now on a model or whatever makes sense in your context. That means less complexity in my workers and less complexity in my database. You might not be so lucky. If one user can kick off a dozen or so of these jobs, and they only do one thing each, you're probably going to have to track Celery workers individually by sending to the user a job identifier and storing them in your database with a foreign key out to your User model. Regardless of how you decide to do it, the job of the initial request handler is to take care of this piece of the puzzle: how am I going to track this, kick off the job, send the tracking information to the user.

The script itself is probably pretty easy. A Celery worker using subprocess from the standard library will be sufficient, with maybe a little bit watching stdout to figure out how far along the job is if you need to. Have the worker write its progress to the database row associated with the job, or use celery-progress if you'd rather use someone else's implementation of that idea.

In your UI, you'll want some sort of AJAX loop so the user can watch the job go. This is usually pretty simple too: just an endpoint that queries the job's status as recorded in the database and returns it without fanfare.

You probably don't want to automatically delete the database entry for the job once its done, because you don't know if the user was there to see the end of it. Save that for a periodic task or a response to the user acknowledging the end of the job run.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/7da4b068-ac58-4c26-a0bc-ed4925f8a4f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.