how to stop route worker after current step completes

[ypz]

Hi, All:

I have a need to restart route workers in the middle of processing a multi-step workflow. I would like to stop a worker after it finishes current step so that next step in workflow can be processed either by other worker instances or by newly started worker instance (if only one worker instance is configured to run).

However,  I don't know how to determine if the worker is currently processing a message/running a step.  From what I see in source, worker has two known states:

"stopped"  : worker will not processing any new messages

"running:   : worker will process new messages from storage

Neither of above two states tells me if the worker is currently busy/processing a message.  Is there a way to determine if my worker is doing a step or do we need to introduce a new worker state ?

Thanks,

ypz

[John Mettraux]

On Tue, Apr 15, 2014 at 01:02:30PM -0700, ypz wrote:
>
> Neither of above two states tells me if the worker is currently
> busy/processing a message. Is there a way to determine if my worker is
> doing a step or do we need to introduce a new worker state ?

Hello,

by "doing a step", do you mean "has handed a workitem out to a participant" ?

John

[Yiping Zhang]

Yes.

My understand is that the worker thread, after handing a workitem out to a participant, will not process next step in workflow until that said participant has finished processing its workitem.

--
--
you received this message because you are subscribed to the "ruote users" group.
to post : send email to openwfe...@googlegroups.com
to unsubscribe : send email to openwferu-use...@googlegroups.com
more options : http://groups.google.com/group/openwferu-users?hl=en
---
You received this message because you are subscribed to the Google Groups "ruote" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwferu-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Mettraux]

> On Tue, Apr 15, 2014 at 1:54 PM, John Mettraux <jmet...@gmail.com> wrote:
> >
> > On Tue, Apr 15, 2014 at 01:02:30PM -0700, ypz wrote:
> > >
> > > Neither of above two states tells me if the worker is currently
> > > busy/processing a message. Is there a way to determine if my worker is
> > > doing a step or do we need to introduce a new worker state ?
> >
> > Hello,
> >
> > by "doing a step", do you mean "has handed a workitem out to a
> > participant" ?

On Tue, Apr 15, 2014 at 03:09:02PM -0700, Yiping Zhang wrote:
>
> Yes.
>
> My understand is that the worker thread, after handing a workitem out to a
> participant, will not process next step in workflow until that said
> participant has finished processing its workitem.

```ruby
concurrence do
step_a
step_b
end
```

With that workflow, the worker will likely proceed step_b right after having
proceeded step_a.

Your understanding is not right.


Best regards,

John

[Yiping Zhang]

Assuming participant's do_not_thread() returns false,  worker instance would run above concurrent sample workflow in two new threads, correct ? In this situation, going back to my original question, since I want to restart worker instance,  I still need to know if those two threads, handling step_a and step_b respectively, have completed their respective work or not

[John Mettraux]

On Tue, Apr 15, 2014 at 03:44:55PM -0700, Yiping Zhang wrote:
>
> Assuming participant's do_not_thread() returns false, worker instance
> would run above concurrent sample workflow in two new threads, correct ? In
> this situation, going back to my original question, since I want to restart
> worker instance, I still need to know if those two threads, handling
> step_a and step_b respectively, have completed their respective work or not

If do_not_thread returns false, the participant "consume" operation will not
happen in a new thread. It'll happen in the current, worker, thread. IIRC.

If you tell #shutdown to a worker, it will return has soon as it has finished
processing the current message. If it isn't processing a message it will
return immediately. Notice the #join at the end of #shutdown.

Best regards.

John

[Yiping Zhang]

On Tue, Apr 15, 2014 at 4:16 PM, John Mettraux <jmet...@gmail.com> wrote:

On Tue, Apr 15, 2014 at 03:44:55PM -0700, Yiping Zhang wrote:
>
> Assuming participant's do_not_thread() returns false,  worker instance
> would run above concurrent sample workflow in two new threads, correct ? In
> this situation, going back to my original question, since I want to restart
> worker instance,  I still need to know if those two threads, handling
> step_a and step_b respectively, have completed their respective work or not

If do_not_thread returns false, the participant "consume" operation will not
happen in a new thread. It'll happen in the current, worker, thread. IIRC.

 

Aren't you getting the effect of do_not_thread() reversed here?

 

If you tell #shutdown to a worker, it will return has soon as it has finished
processing the current message. If it isn't processing a message it will
return immediately. Notice the #join at the end of #shutdown.

This is true only if participant's do_not_thread() returns true. 

 

This is actually how my code currently work : my participant has do_not_thread() return true, and when I need to restart a worker instance (process!),  I call worker,shutdown and check worker.state in a while true loop.  Once worker.state is set to  "stopped", my worker process exits.

This all works well until  I also need to skip a workflow step with cancel_expression (see my email thread yesterday: 

https://groups.google.com/forum/?fromgroups#!topic/openwferu-users/h8vAAFkmlu8 ), 

After issuing cancel_expression, I noticed that the worker process is NOT freed up and the participant associated with the canceled step is still running.   Since I have only one worker process configured to run, then this means  there would be no more worker process to handle next step (the step after canceled step) in the workflow.

This is the reason why I am revisiting my setting of do_not_thread() in my participant and the need to figure out if  worker's @run_thread (not main thread) is busy or not  so that I can exit the main worker thread.

Thanks,

 

Best regards.

[John Mettraux]

2014-04-16 9:26 GMT+09:00 Yiping Zhang <yipin...@gmail.com>:
>
>
>
> On Tue, Apr 15, 2014 at 4:16 PM, John Mettraux <jmet...@gmail.com> wrote:
>>
>>
>> On Tue, Apr 15, 2014 at 03:44:55PM -0700, Yiping Zhang wrote:
>> >
>> > Assuming participant's do_not_thread() returns false, worker instance
>> > would run above concurrent sample workflow in two new threads, correct ?
>> > In
>> > this situation, going back to my original question, since I want to
>> > restart
>> > worker instance, I still need to know if those two threads, handling
>> > step_a and step_b respectively, have completed their respective work or
>> > not
>>
>> If do_not_thread returns false, the participant "consume" operation will
>> not
>> happen in a new thread. It'll happen in the current, worker, thread. IIRC.
>>
>
> Aren't you getting the effect of do_not_thread() reversed here?

No.

>> If you tell #shutdown to a worker, it will return has soon as it has
>> finished
>> processing the current message. If it isn't processing a message it will
>> return immediately. Notice the #join at the end of #shutdown.
>>
> This is true only if participant's do_not_thread() returns true.
>
> This is actually how my code currently work : my participant has
> do_not_thread() return true, and when I need to restart a worker instance
> (process!), I call worker,shutdown and check worker.state in a while true
> loop. Once worker.state is set to "stopped", my worker process exits.

You're replicating the behaviour of the ruote worker.

> This all works well until I also need to skip a workflow step with
> cancel_expression (see my email thread yesterday:
> https://groups.google.com/forum/?fromgroups#!topic/openwferu-users/h8vAAFkmlu8
> ),
>
> After issuing cancel_expression, I noticed that the worker process is NOT
> freed up and the participant associated with the canceled step is still
> running. Since I have only one worker process configured to run, then this
> means there would be no more worker process to handle next step (the step
> after canceled step) in the workflow.
>
> This is the reason why I am revisiting my setting of do_not_thread() in my
> participant and the need to figure out if worker's @run_thread (not main
> thread) is busy or not so that I can exit the main worker thread.

You're not telling it but it seems you have long running work
happening in your participant implementation. Well, that's my guess.

I think you need the worker to keep track of the [participant]
dispatch threads. Sorry, it does not do it, it just fires them.

Best regards.

John