problems running subprocess inside django view

[Mark Lancaster]

I'm having problems running subprocess inside a django view using:

result = subprocess.Popen([<A SCRIPT> , <SOME PARAMS>],
stdout=subprocess.PIPE).communicate()[0]

exactly the same method works perfectly inside a regular python
script.

Should I be using a different method to initiate a script?

Thanks

[Eugene Gavrish]

You are not alone with this problem. But I haven't got any decision.
Maybe its django-specific with Popen??

I divided my task in two part: first cron-driven disk file generation.
Second - from django-view this file parsing.
It satisfied my goals, but question with Popen is still opened...

[Brian Schott]

As the same user? Are you running with manage.py runserver? Generally it is a bad idea to call subprocess from a web process context. A blocking call you don't expect might time out the client and/or hang your server.

Check out Django-celery. Create a tasks.py and have your view call a task.

Sent from my iPhone

> --
> You received this message because you are subscribed to the Google Groups "Django users" group.
> To post to this group, send email to django...@googlegroups.com.
> To unsubscribe from this group, send email to django-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
>

[Russell Keith-Magee]

You shouldn't be trying to initiate a script from within a view.

The request-response cycle needs to be short lived. The responsiveness
of the user's experience is directly tied to how long it takes for
your server to complete executing a view; if you're invoking
subprocesses (especially long lived subprocesses) as part of a view,
then you aren't going to b

"Oh, but my subprocess *will* be short lived!" you say? Well, you've
still got a problem -- because your user can hit cancel at any time
during a request, closing a connection, which can cause all sorts of
interesting problems with managing dangling subprocesses.

If you need to execute something long lived, you should break it down
into parts:

1) A view that creates a "job". This doesn't need to be any more than
writing a single entry in a database table.
2) A view that can be used to poll the status of the job.
3) A background task -- completely outside the request-response cycle
-- that executes jobs, and reports the status back to the database
table.

This ensures that the expensive process is handled out of the
request-response cycle, yielding a more responsive interface, and
giving you better scalability, too (since your ability to handle
background processes is decoupled from your ability to handle user
requests for background processes).

There are many ways to achieve the background task -- celery is one
popular option, but you can organise an quick and nasty
proof-of-concept using cron.

Yours,
Russ Magee %-)