DRb Server way to enable Ruote support multiprocess Rails web app depoyment
Rui Ma
I finished a prototype to run ruote engine as a DRb server for multiprocess deployments.
Ruote-rest is the current way to use ruote engine in a multiprocess environment. But developers need to write code to take care of the underlying communication between models and ruote-rest server.
A DRb ruote server provides an alternative way to deploy ruote in a multiprocess environment. The primary benefit of using it is that developers can still use ruote as the way of ruote-web2 client code, without adding/chaning client code for multiprocess depolyment.
Another concern is that using TCP/UNIX socket might be faster than initiating internal HTTP requests in terms of the ruote server/client communications.
I am not sure either ruote-rest or drb server is thread-safe. Maybe the only way to provide ruote with a thread-safe multiprocess environment is to implement workitem level lock.
I put together DRb server/client code here: http://github.com/ruima-nyc/ruote-web2
running it:
cd ruote-web2
ruby script/ruote_server.rb
ruby script/server
Rui Ma
http://rui.ma
John Mettraux
>
> I finished a prototype to run ruote engine as a DRb server for multiprocess
> deployments.
Hello Rui Ma,
you're fast ! Looks very neat.
> Ruote-rest is the current way to use ruote engine in a multiprocess
> environment. But developers need to write code to take care of the
> underlying communication between models and ruote-rest server.
>
> A DRb ruote server provides an alternative way to deploy ruote in a
> multiprocess environment. The primary benefit of using it is that developers
> can still use ruote as the way of ruote-web2 client code, without
> adding/chaning client code for multiprocess depolyment.
>
> Another concern is that using TCP/UNIX socket might be faster than
> initiating internal HTTP requests in terms of the ruote server/client
> communications.
>
> I am not sure either ruote-rest or drb server is thread-safe. Maybe the
> only way to provide ruote with a thread-safe multiprocess environment is to
> implement workitem level lock.
Since I'm working on ruote 2.0 and making sure multiprocess
deployments do not interfere with ruote, I worked on locking access to
scheduler queues (so that timeouts and such scheduled events get
triggered on time and only once for the whole pack of engines) and to
tracker queues (so that a listen expression (not very often used, I
know) is triggered once for the whole pack of engines).
For workitems themselves, thread-safe measures brought by
active-record itself should suffice.
> I put together DRb server/client code here:
> http://github.com/ruima-nyc/ruote-web2
>
> running it:
>
> cd ruote-web2
> ruby script/ruote_server.rb
> ruby script/server
>
> Rui Ma
> http://rui.ma
Well done,
John
Enrico Bianco
On Aug 15, 3:12 am, John Mettraux <jmettr...@openwfe.org> wrote:
I'm currently trying to integrate this with my application, which has
custom-written participants that make changes to models in the
application. The participants are defined and registered with the
engine when the Rails application starts up (not when the DRb ruote
instance does). I see the following error when one of these custom
participants is invoked:
undefined method `new' for #<DRb::DRbUnknown:0x114b7b0>
I know that this particular participant never instantiates any new
objects, so I think this might be happening when the participant
itself is being instantiated and passed the workitem by ruote.
Should participants be defined and registered when the engine starts
up instead of the app? How would those participants then access the
application models?
Thanks,
Enrico
John Mettraux
>
> I'm currently trying to integrate this with my application, which has
> custom-written participants that make changes to models in the
> application. The participants are defined and registered with the
> engine when the Rails application starts up (not when the DRb ruote
> instance does). I see the following error when one of these custom
> participants is invoked:
>
> undefined method `new' for #<DRb::DRbUnknown:0x114b7b0>
Hello Enrico,
I have no experience with drb, but let me ask for a few
clarifications, maybe that'll explain Rui Ma for a further reply.
Do you have a detailed stack trace for that "undefined method" issue ?
Maybe you could show the piece of code where the error occurs.
Do I guess correctly when I say that the error occurs on the remote
(engine) side ?
> I know that this particular participant never instantiates any new
> objects, so I think this might be happening when the participant
> itself is being instantiated and passed the workitem by ruote.
Participants are instantiated at registration time.
> Should participants be defined and registered when the engine starts
> up instead of the app? How would those participants then access the
> application models?
I guess it's bit hard to push the work to a "remote" engine and then
expect the participants to interact with "local" models. Thorny issue
indeed.
Best regards,
--
John Mettraux - http://jmettraux.wordpress.com
Enrico Bianco
On Aug 28, 12:30 pm, John Mettraux <jmettr...@openwfe.org> wrote:
>
> > I'm currently trying to integrate this with my application, which has
> > custom-written participants that make changes to models in the
> > application. The participants are defined and registered with the
> > engine when the Rails application starts up (not when the DRb ruote
> > instance does). I see the following error when one of these custom
> > participants is invoked:
>
> > undefined method `new' for #<DRb::DRbUnknown:0x114b7b0>
>
> Hello Enrico,
>
> I have no experience with drb, but let me ask for a few
> clarifications, maybe that'll explain Rui Ma for a further reply.
>
> Do you have a detailed stack trace for that "undefined method" issue ?
> Maybe you could show the piece of code where the error occurs.
>
> Do I guess correctly when I say that the error occurs on the remote
> (engine) side ?
ProcessError log entry. But I think your guess is correct.
>
> > I know that this particular participant never instantiates any new
> > objects, so I think this might be happening when the participant
> > itself is being instantiated and passed the workitem by ruote.
>
> Participants are instantiated at registration time.
during the process? *puzzled*
>
> > Should participants be defined and registered when the engine starts
> > up instead of the app? How would those participants then access the
> > application models?
>
> I guess it's bit hard to push the work to a "remote" engine and then
> expect the participants to interact with "local" models. Thorny issue
> indeed.
rest server after all. But since I integrated ruote to my application
using ruote_plugin to begin with, the amount of refactoring that would
be required to separate them again is reason enough to seriously
consider alternatives.
But in good news, I did get the participants to work, in the end. I
shuffled code around so that the DRb process was defining and
registering them instead of the application. Somehow that "Just
Works." Now I'm just finding that any expressions that use the
scheduler are not getting fired.
Thanks for all the help!
- Enrico
John Mettraux
> On Aug 28, 12:30 pm, John Mettraux <jmettr...@openwfe.org> wrote:
>>
>> Do I guess correctly when I say that the error occurs on the remote
>> (engine) side ?
>
> Unfortunately, I can't get a full stack-trace. That was inside the
> ProcessError log entry. But I think your guess is correct.
Hello Enrico,
the "inside the ProcessError log" is a capital piece of information ;)
>> > I know that this particular participant never instantiates any new
>> > objects, so I think this might be happening when the participant
>> > itself is being instantiated and passed the workitem by ruote.
>>
>> Participants are instantiated at registration time.
>
> Strange, because if that's the case then why did the error occur
> during the process? *puzzled*
Maybe the "participant was trying to instantiate something" and not
"something was trying to instantiate the participant". You gave us no
stacktrace, no real info about the target of the "new" method ;)
> But in good news, I did get the participants to work, in the end. I
> shuffled code around so that the DRb process was defining and
> registering them instead of the application. Somehow that "Just
> Works." Now I'm just finding that any expressions that use the
> scheduler are not getting fired.
Ouch. Kind of a edgy setup that you have.
Well done !
Rui Ma
Your client code (Ruote code in your rails app) should not need to be changed. They should work exactly the way they are in the single-process server setup.
undefined method `new' for #<DRb::DRbUnknown:0x114b7b0> means some of your objects at the remote engine server didn't get instantiated successfully. Similar to a nil.new error. One thing you can do is to run your app in debug mode. Calling the ruote engine code at the breakpoint might give you some more info.
Please notice that there is a config/ruote_environment. Make sure you include all the plugins/classes you need in tha file as well.
How would those participants then access the application models?
hard to tell why any expressions that use the scheduler didn't fired. More details are helpful.
-Rui
Enrico Bianco
On Aug 29, 2:09 am, Rui Ma <rui.m...@gmail.com> wrote:
> Hello Enrico,
>
> Your client code (Ruote code in your rails app) should not need to be
> changed. They should work exactly the way they are in the single-process
> server setup.
>
> undefined method `new' for #<DRb::DRbUnknown:0x114b7b0> means some of your
> objects at the remote engine server didn't get instantiated successfully.
> Similar to a nil.new error. One thing you can do is to run your app in debug
> mode. Calling the ruote engine code at the breakpoint might give you some
> more info.
>
> Please notice that there is a config/ruote_environment. Make sure you
> include all the plugins/classes you need in tha file as well.
to bring a whole bunch of config.gem statements into there.
>
> *How would those participants then access the application models?*
> A "remote" engine doesn't interact with "local" models. In my case,
> ActiveRecord models' ids are carried by workitem fields. Particiapnts
> re-instantiate models on the remote engine side and process them.
serialize particularly well.
>
> hard to tell why any expressions that use the scheduler didn't fired. More
> details are helpful.
could dig around some more. The problem is that nothing is being put
into the ProcessError log for these expressions. I'll check over the
process history and see if anything shows up there, too.
Thanks,
Enrico
>
> -Rui
>
> On Fri, Aug 28, 2009 at 9:25 PM, John Mettraux <jmettr...@openwfe.org>wrote:
Enrico Bianco
Turns out that the scheduled events aren't firing. I just couldn't
find them being fired in the logs. Actually, I needed to move the
ActionMailer SMTP settings into config/ruote_environment.rb so that
custom participants using ActionMailer would be able to send e-mail
correctly.
Basically, it seems like most of the problems that I've had with this
setup have been with something being defined in one process and not
the other. With a little more shuffling, I'm confident that I should
have everything working the way it did before I started using DRb.
So, for what it's worth: DRb FTW! =D
- Enrico
Enrico Bianco
That should read "the scheduled events ARE firing."
*blush*
- Enrico