Multiple Workers

[Matt Nichols]

Hi All,
How do I go about running multiple workers when I have remote
participants. It seems the only way to register participants is via an
engine instance. Is it acceptable to have multiple engines as well? I
may be missing something. Here's an example of what I'm trying to
do...


--------------------------------------------------------------------------------------------------------
Run an engine instance...
--------------------------------------------------------------------------------------------------------

@worker = Ruote::Worker.new(Ruote::FsStorage.new('/ruote_work'))
@engine = Ruote::Engine.new(@worker, false)

# Setup listeners
RuoteAMQP::LaunchitemListener.new(@engine,'the_launch_queue')
RuoteAMQP::WorkitemListener.new(@engine,'the_workflow_engine_queue')

# Register participants
@engine.register_participant(:tagging,
RuoteAMQP::Participant.new( :default_queue =>
'remote_participant_queue' ))

# Run
@worker.run


--------------------------------------------------------------------------------------------------------
In another ruby process run just a worker...
--------------------------------------------------------------------------------------------------------

@worker = Ruote::Worker.new(Ruote::FsStorage.new('/ruote_work'))
@worker.run

# This will fail because the worker has no knowledge of the :tagging
participant.



--------------------------------------------------------------------------------------------------------
Can I just do this?
--------------------------------------------------------------------------------------------------------

@worker = Ruote::Worker.new(Ruote::FsStorage.new('/ruote_work'))
@engine = Ruote::Engine.new(@worker, false)

#
# Omit the listener setup
#

# Register participants
@engine.register_participant(:tagging,
RuoteAMQP::Participant.new( :default_queue =>
'remote_participant_queue' ))

# Run
@worker.run



Thanks!

[John Mettraux]

On Thu, Jun 17, 2010 at 02:25:36AM -0700, Matt Nichols wrote:
>
> How do I go about running multiple workers when I have remote
> participants. It seems the only way to register participants is via an
> engine instance. Is it acceptable to have multiple engines as well? I
> may be missing something. Here's an example of what I'm trying to
> do...
>

> (...)

>
> # Register participants
> @engine.register_participant(:tagging,
> RuoteAMQP::Participant.new( :default_queue =>
> 'remote_participant_queue' ))
>
> # Run
> @worker.run
>

> # In another ruby process run just a worker...

>
> @worker = Ruote::Worker.new(Ruote::FsStorage.new('/ruote_work'))
> @worker.run
>
> # This will fail because the worker has no knowledge of the :tagging
> participant.

Hello Matt,

you're right, since the participant :tagging is instantiated at registration time, it's only known by the ruby process where the registration happened.

To have a participant which is known to all workers :

---8<---
@engine.register_participant(
:tagging,
RuoteAMQP::Participant,
'default_queue => 'remote_participant_queue')
--->8---

> -----------------------------------------------------------------------------

> Can I just do this?
> -----------------------------------------------------------------------------
>

> @worker = Ruote::Worker.new(Ruote::FsStorage.new('/ruote_work'))
> @engine = Ruote::Engine.new(@worker, false)
>
> #
> # Omit the listener setup
> #
>
> # Register participants
> @engine.register_participant(:tagging,
> RuoteAMQP::Participant.new( :default_queue =>
> 'remote_participant_queue' ))
>
> # Run
> @worker.run

It could work as well, but the "stateless" registration described above is the recommended way.

Best regards,

--
John Mettraux - http://jmettraux.wordpress.com

[Torsten Schönebaum]

John Mettraux wrote:
> On Thu, Jun 17, 2010 at 02:25:36AM -0700, Matt Nichols wrote:
> >
> > How do I go about running multiple workers when I have remote
> > participants. It seems the only way to register participants is via an
> > engine instance. Is it acceptable to have multiple engines as well? I
> > may be missing something. Here's an example of what I'm trying to
> > do...
> >
> > (...)
> >
> > # Register participants
> > @engine.register_participant(:tagging,
> > RuoteAMQP::Participant.new( :default_queue =>
> > 'remote_participant_queue' ))
> >
> > # Run
> > @worker.run
> >
> > # In another ruby process run just a worker...
> >
> > @worker = Ruote::Worker.new(Ruote::FsStorage.new('/ruote_work'))
> > @worker.run
> >
> > # This will fail because the worker has no knowledge of the :tagging
> > participant.
>

> you're right, since the participant :tagging is instantiated at
> registration time, it's only known by the ruby process where the
> registration happened.
>
> To have a participant which is known to all workers :
>
> ---8<---
> @engine.register_participant(
> :tagging,
> RuoteAMQP::Participant,
> 'default_queue => 'remote_participant_queue')
> --->8---

You should note that there seems to be a bug in ruote-amqp which makes
using that approach impossible: The RuoteAMQP::Participant just forgets
the configuration hash.

More info:
http://groups.google.com/group/openwferu-users/browse_thread/thread/b629605d62ce192c
(and of course in the follow-ups).

Hopefully, there will be a fix soon.

Yours,
Torsten

[John Mettraux]

Hello,

here is a "fixed" version (specs in the green) :

http://github.com/jmettraux/ruote-amqp

I will request a pull from Kenneth.

Feel free to build a gem from this version or to point your Gemfile to it.