rails - providing user feedback during a long running process

[Mark Woods]

As far as I can tell, there is no way to flush the output buffer while
processing a rails request. With this in mind, I'm now scratching my
head wondering how to provide feedback to a user during a long running
request (import scripts).

The only thing I've come up with so far is to break the import process
into multiple steps, call each step using an ajax request from a page
and use rjs templates in responses to add feedback messages (importing
customer 'Mark Woods' etc.) into the calling page. Sounds like an awful
lot of trouble compared to flushing an output buffer in coldfusion, php
etc. Anyone got any other suggestions?

Mark

[TonyB]

Hi Mark,

> The only thing I've come up with so far is to break the import process
> into multiple steps, call each step using an ajax request from a page
> and use rjs templates in responses to add feedback messages (importing
> customer 'Mark Woods' etc.) into the calling page. Sounds like an awful
> lot of trouble compared to flushing an output buffer in coldfusion, php
> etc. Anyone got any other suggestions?

Have you looked at BackgroundRB?

There's a recipe in Mike Clark's "Advanced Rails Recipes" book that
uses BackgroundRB to do something similar to what you need.

Regards,

Tony.

[Dave Rice]

Previously I used backgroundrb for this but then switched to using backgroundjob. bdrb was brittle in a high usage environment. However bdrb now has new maintainers so might be becoming production ready...

Any long running actions should be taken out of the request cycle.

* The rails controller action instead of performing really complex stuff, starts a process to do the work

* The process updates the progress of the action in a centralised place. Database or object store

* A javascript callback on the page can query to get the progress of the job and update the page accordingly

There is also a new framework in progress called Vertebra. Really got my eye on that one.

http://backgroundrb.rubyforge.org/

http://codeforpeople.rubyforge.org/svn/bj/trunk/README

http://brainspl.at/articles/2008/06/02/introducing-vertebra

---

David Rice

+44 (0)78 708 12996

http://davidjrice.co.uk

[Denis Hennessy]

It's generally a bad idea to do too much work (say more than a second
or two) in the request thread. Although it'll seem to work OK with 1
or 2 users, it won't scale much beyond that. The best option is to
hand it off to a worker thread/process and allow the client to monitor
the status with ajax calls.

Right now, I'd say backgroundrb is probably the best library to use
(although there are certainly plenty to pick from). Spawn
(http://rubyforge.org/projects/spawn) might also be worth a look.

/dh

[Mark Woods]

> It's generally a bad idea to do too much work (say more than a second
> or two) in the request thread. Although it'll seem to work OK with 1
> or 2 users, it won't scale much beyond that.

Obviously, in general long running requests are a bad idea, but these
are import scripts that are manually run once a month in a system that
currently has 6 users and isn't likely to ever have more than 20 (and
I'm thinking a good few years down the line there). The system also runs
on a dedicated server in the client's office. A long running process
wouldn't be a big deal here if the user was getting constant feedback.

However, backgrounding the job and having the web interface monitor it
is a nice approach and I'll look into that. Thanks for all the replies.

Mark

[macarthy]

Hi Mark,

Are you running mongrels ? You can't stream from rails in Mongrel
because Rails and Mongrel do buffering
Also most (HAML ERB) renderers return completed pages rather than
increments.
Maybe you should break your import out of rails ?

I know of solutions, but they require out of rails coding...

Justin

[macarthy]

OK I have 5 minutes...

You can stream output in a non blocking manner by writing a custom
mongrel handler.

something like

In lib/long_time_coming.rb

require 'rubygems'
require 'mongrel'

class LongTimeComing < Mongrel::HttpHandler
def process(request, response)
response.status = 200
response.send_status(nil)

response.header['Content-Type'] = "text/plain"
response.send_header


while working
response.write("working")
end
response.write("done")

response.done
end
end

uri "/stream", :handler => LongTimeComing.new



Use the -S switch with mongrel_rails....

mongrel_rails start -S lib/long_time_coming.rb


Simple enough..

Justin

[Mark Woods]

Hey Justin,

> You can stream output in a non blocking manner by writing a custom
> mongrel handler.

We are using mongrel, but hadn't planned to tie the app to it. Using a
custom mongrel handler is an appealing option though. The background
process is a nice idea, but at this stage we need something quick and
easy (we're building a functioning prototype at the moment and we don't
know exactly how things will pan out in the end). We'd even thought
about just throwing in a php script to handle the import for the moment,
but keeping it in ruby would be preferable.

Thanks

Mark