I want to make a webapp that allows users to queue bit of work (converting video for example). They are also able to see the queue at any time and check the progress (queued|processing|done) and are able to REORDER items in the queue. The queue can be stopped (ie. paused) and resumed in between conversions of course.
Im sure there's ppl out there who've done something similar. I can do this in java no problem (synchronized lists etc) but not sure how to do this in ruby as im sort of a newbie.
I have a working bit of code but im sure im missing something (memory leak, synchorizing issues etc). Can post more code (very small) if someone's willing to help.
> I want to make a webapp that allows users to queue bit of work > (converting video for example). They are also able to see the queue at > any time and check the progress (queued|processing|done) and are able to > REORDER items in the queue. > The queue can be stopped (ie. paused) and resumed in between conversions > of course.
> Im sure there's ppl out there who've done something similar. I can do > this in java no problem (synchronized lists etc) but not sure how to do > this in ruby as im sort of a newbie.
> I have a working bit of code but im sure im missing something (memory > leak, synchorizing issues etc). Can post more code (very small) if > someone's willing to help.
> Thanks!
Oh and i forgot to mention that the queue can be persisted if u want - i dont care at this stage. But im essentially keeping the queue in a flat file on disk to hopefully get some 'cheap locking'. I'd like to do this without a DB so to speak.
OK i altered my code to use Sequel.sqlite. So now i have my "list" in memory. So now the queue implementation is the issue. thx -- Posted via http://www.ruby-forum.com/.
On Thu, Nov 12, 2009 at 3:46 PM, Gurpal 2000 <g...@gurpal.co.uk> wrote: > OK i altered my code to use Sequel.sqlite. So now i have my "list" in > memory. So now the queue implementation is the issue. thx
Are you wanting to farm out work among a distributed network of agents? Nanite + RabbitMQ comes to mind:
not really farm out, i'll check this out but mq sounds over the top for what's essentially a small home project (for now) -- Posted via http://www.ruby-forum.com/.
On Thu, Nov 12, 2009 at 10:45 PM, Gurpal 2000 <g...@gurpal.co.uk> wrote: > Gurpal 2000 wrote: > > Hi
> > I want to make a webapp that allows users to queue bit of work > > (converting video for example). They are also able to see the queue at > > any time and check the progress (queued|processing|done) and are able to > > REORDER items in the queue. > > The queue can be stopped (ie. paused) and resumed in between conversions > > of course.
> > I'm using sintatra and the LoopingThread example from here
> > Im sure there's ppl out there who've done something similar. I can do > > this in java no problem (synchronized lists etc) but not sure how to do > > this in ruby as im sort of a newbie.
> > I have a working bit of code but im sure im missing something (memory > > leak, synchorizing issues etc). Can post more code (very small) if > > someone's willing to help.
> > Thanks!
> Oh and i forgot to mention that the queue can be persisted if u want - i > dont care at this stage. But im essentially keeping the queue in a flat > file on disk to hopefully get some 'cheap locking'. I'd like to do this > without a DB so to speak.
There are numerous solutions for this kind of task: BackgroundJob, DelayedJob, Workling etc. The list is quite extensive.
My personal opinion is that spinning off threads to do this work is not the way to go about it. To my mind this is an activity that should be spun off in a separate process.
While you want to avoid using the DB for this, I think you should reconsider: DB backed scheduling gets your locking and monitoring via transactions/persistence and it doesn't have to be difficult either. I would look for a background scheduling tool that implicitly uses your DB so that you don't get your hands dirty.
OK i managed to use Sequel DB. But the backgroundjob api etc seems overkill. This LoopingThread works below, but i noticed 100% usgae on ruby 1.8.6. Is this normal?
class LoopingThread < Thread def initialize @stopped = false @paused = false super do before_loop until @stopped yield Thread.stop if @paused end after_loop end end
I would guess your culprit for the spike to 100% CPU usage may be your infinite loop:
> until @stopped > yield > Thread.stop if @paused > end
Look at using sleep() if you want to decrease CPU usage. Even sleeping for one second should decrease your CPU usage dramatically in my experience(maybe with a rare spike).
> I would guess your culprit for the spike to 100% CPU usage may be your > infinite loop:
>> until @stopped >> yield >> Thread.stop if @paused >> end
> Look at using sleep() if you want to decrease CPU usage. Even sleeping > for one second should decrease your CPU usage dramatically in my > experience(maybe with a rare spike).
Sure, i got this code from the Oreilly Ruby Cookbook. Is it bad code then? Again this doesn't happen when i use similar code in Java... -- Posted via http://www.ruby-forum.com/.
> Sure, i got this code from the Oreilly Ruby Cookbook. Is it bad code > then?
The book seems to be a little outdated - for example, i rarely see people defining setters themselves nowadays, especially when you can use attr_writer :pause . I'm not sure why they didn't pick up on what could be an infinite loop, or at least a loop that can repeat itself many, many times.
> Again this doesn't happen when i use similar code in Java...
I don't know why that is in Java - in C, Ruby, an infinite loop is going to consume a lot of CPU power unless you take precautions.
|
