How to execute parallel Multi-Instance Call Activity's

[john.rich...@gmail.com]

We have a process that references an external multi-instance Call Activity. In the main process, we have defined a collection for it to run off of that is populated from a database query. This is all running fine...the only issue is that I have specified isSequential to be false, thinking that the instances of the Call Activity would run in parallel, but this isn't the case. It does them serially. We're running GlassFish and I have entered the necessary entries in domain.xml from the Camunda docs, it appears that there should be at least 3 threads available. Could somebody help me out?

From the .bpmn file for the main process:
...
<bpmn2:serviceTask id="PopulateNodeList" activiti:class="PopulateNodeList" name="Populate Node List">
<bpmn2:incoming>SequenceFlow_20</bpmn2:incoming>
<bpmn2:outgoing>SequenceFlow_2</bpmn2:outgoing>
</bpmn2:serviceTask>
...
<bpmn2:callActivity id="DoWorkOnNode" name="Do Work on Node" calledElement="DoWorkOnNode">
<bpmn2:extensionElements>
<activiti:in source="node" target="node"/>
</bpmn2:extensionElements>
<bpmn2:incoming>SequenceFlow_2</bpmn2:incoming>
<bpmn2:outgoing>SequenceFlow_22</bpmn2:outgoing>
<bpmn2:multiInstanceLoopCharacteristics activiti:collection="${nodeList}" activiti:elementVariable="node"/>
</bpmn2:callActivity>

From glassfish domain.xml:
...
<thread-pool max-thread-pool-size="6"
name="platform-jobexecutor-tp"
min-thread-pool-size="3"
max-queue-size="10">
</thread-pool>

[Bernd Rücker (camunda)]

Hi John.

Please note that we use the client Thread we get and do not introduce an
own multi-threading within the engine (as this would introduce possible
resource problems on the server and not be Java EE compliant). Some
background reading is here:
https://app.camunda.com/confluence/display/foxUserGuide/Thread+and+Transac
tion+Management

That means if you have no wait states (e.g. ServiceTasks) everything is
executed within own transaction from one thread - that it is why it is
executed in sequence. If you need it parallel you have to use a so called
"async" on the ServiceTask (see article above). But please note that in
that case you have to take care about exclusive jobs as well - as the
engine normally only executes one thing at a process instance at a time to
not step on each other toes. Unfortunately the docs are not yet moved - so
you find information about that here:
http://www.activiti.org/userguide/index.html#exclusiveJobs.

Does that help?

Cheers
Bernd

-----Urspr�ngliche Nachricht-----
Von: camunda-...@googlegroups.com
[mailto:camunda-...@googlegroups.com] Im Auftrag von
john.rich...@gmail.com
Gesendet: Samstag, 6. Juli 2013 21:57
An: camunda-...@googlegroups.com
Betreff: How to execute parallel Multi-Instance Call Activity's

--
You received this message because you are subscribed to the Google Groups
"camunda BPM users & process application developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to camunda-bpm-us...@googlegroups.com.
To post to this group, send email to camunda-...@googlegroups.com.

[john.rich...@gmail.com]

Hi Bernd,

I've added 'async=true' on the Multi-instance Call Activity and it appears to be trying to execute them in parallel. The issue I'm having now is it's saying it can't deserialize the object from the collection, caused by ClassNotFoundException. However, when async was set to false and it was running them serially, it found and used the object just fine. We're using GlassFish 3.1.2.2 and I'm deploying the camunda jobexecutor rar, the camunda glassfish ear and our own application ear. Along with other EJB and utility jars, our ear also contains a war file with the workflow stuff in it. Thanks,

John

[Daniel Meyer]

Hi John,

This should work. In order to help you solve this problem, it would be helpful to see a stacktrace and get more precise information about the exact deployment structure of the EAR file. Here are some questions which could help finding a solution: 

I gather you're deploying a multi-module EAR? 

Are you using a custom ProcessApplication class or are you using the camunda-ejb-client library? 

How is the library deployed? 

The class mentioned in the ClassNotFoundException, is it visible from the classloader of either the camunda-ejb-client or, if you are using a custom @ProcessApplication class, from the classloader of the module which bundles the PA class?

Cheers,

Daniel Meyer

[john.rich...@gmail.com]

Hi Daniel,

I was able to get past the ClassNotFoundException by adding some jars to my lib folder; however, once I ran it through again, it still looked like it ran things serially.
In my config I have the following:

<bpmn2:callActivity id="DoWorkOnNode" activiti:async="true" activiti:exclusive="false" name="Do Work On Node" calledElement="DoWorkOnNode">

<bpmn2:extensionElements>
<activiti:in source="node" target="node"/>
</bpmn2:extensionElements>
<bpmn2:incoming>SequenceFlow_2</bpmn2:incoming>

<bpmn2:outgoing>SequenceFlow_3</bpmn2:outgoing>

<bpmn2:multiInstanceLoopCharacteristics activiti:collection="${nodeList}" activiti:elementVariable="node">

<bpmn2:loopCardinality xsi:type="bpmn2:tFormalExpression">3</bpmn2:loopCardinality>
</bpmn2:multiInstanceLoopCharacteristics>
</bpmn2:callActivity>

When we didn't set a value for loopCardinality, it just looped through 'nodeList' over and over running on each item serially until we killed glassfish. Right now, we're testing with a 'nodeList' with 3 items in it so we set loopCardinality to 3. That will execute DoWorkOnNode for each item in the list once and exit, but it doesn't do them in parallel. I guess there's two issues here...how do we set loopCardinality on the fly (programatically) depending on the size of 'nodeList'? Do we just need to set another process variable? That won't be too hard to try. And, the second issue...why isn't it executing them in parallel when we have async and exclusive set?

[Daniel Meyer]

Hi John Richard,

 

> Why isn't it executing them in parallel when we have async and exclusive set?

 

The asynchronous continuation happens before the multi instance call activity is executed. This means that execution will stop before the call activity is reached and then resumed. Subsequently A SINGLE thread will execute the multi instance activity behavior and create the sub-executions for the individual instances. These are executed serially, by this very thread.

 

If you want to make the execution of the individual called process instances truly concurrent, you need  to add an asynchronous continuation on the first activity in the called sub process. This way all instances will be created by the first thread and then continued concurrently. By multiple new threads:

 

 

 

Be aware that your process instances will potentially encounter optimistic locking exceptions when the finished executions of the multi instances should be joined in concurrent, interleaving transactions. So make sure that the last transaction in the subprocess does not have any non-transactional side effects.

(You could also consider using the retry interceptor

https://github.com/camunda/camunda-bpm-platform/blob/master/engine/src/main/java/org/camunda/bpm/engine/impl/interceptor/RetryInterceptor.java?source=cc )

 

As a general thought: think about what you want to optimize for:

Intra-Process Instance Concurrency <> Inter Process Instance Concurrency

local vs. global throughput. Should a single instance finish fast or do you want to do may instances per time-unit?

 

Cheers,

Daniel Meyer

Project Lead

www.camunda.org/community/team.html

 

 

[john.rich...@gmail.com]

Daniel,

Maybe you can give me some guidance on the best way to go about this. We have our main process which, at a certain point, queries the database, forms a list which is saved as a Camunda process variable, then the multi-instance Call Activity (we'll call it X) is executed once against each of the entries in the list. The Call Activity is actually broken down into a couple other Call Activity's (let's call them Y & Z) which themselves have a handful of Service Task's. The total execution time of each X is going to be anywhere from 1-5 minutes. So, for example, if we have 4 nodes and let's say they run in 1 minute, 3 minutes, 2 minutes and 4 minutes, that's 10 minutes total if they run serially. If we can execute them in parallel, they're done in 4 minutes. Also, we want to be able to join when they're all done and run a couple more Service Task's in the main process.

By the way, thanks for the detail. We're going to try to make the inner Service Task's async and see if that achieves what we're looking for.

John

[john.rich...@gmail.com]

Update...so making the service task inside the call activity asynchronous worked. Things are working very smoothly now. My next question has to do with an anomaly situation. If the Call Activity doesn't behave like we expect it to, it may take longer than 5 minutes to complete. In this case, the Camunda engine executes the job retry strategy. However, now we have 2 instances of the Call Activity executing, which is not what we want. As an immediate fix, I would like to extend the job timeout to 10 minutes so the first instance will have time to finish. I have tried to set the retry strategy to R2/PT10M on the same Service Task that we made asynchronous, but it still retries at 5 minutes. I also tried setting the retry strategy on the Call Activity itself in the outer process, but that doesn't seem to affect the timeout, either. Based on the Camunda docs, it seems like it should be very straightforward, but it's not working for me. Any words of wisdom?

[Bernd Rücker (camunda)]

Hi John.

5 minutes is the default LOCK timeout for jobs. You have to reconfigure
this (from my head: JobExecutor.maxLockTime) in order to have threads/tx
longer than this timeout (which is by the way not always a good idea -
keep in mind that you have other timeouts as well - e.g. Transactions).

Cheers
Bernd
Evangelist & Consultant
www.camunda.org/community/team.html

-----Urspr�ngliche Nachricht-----
Von: camunda-...@googlegroups.com
[mailto:camunda-...@googlegroups.com] Im Auftrag von
john.rich...@gmail.com

Gesendet: Freitag, 2. August 2013 20:18
An: camunda-...@googlegroups.com
Cc: john.rich...@gmail.com
Betreff: Re: How to execute parallel Multi-Instance Call Activity's

[Daniel Meyer]

Hi John,

 

If I understand you correctly, you want to use a custom configuration for the retry time cycles.  

 

In that case you have to add the Job Retry Plugin to the configuration of the process engine.

Example of how to do it programmatically:

https://github.com/camunda/camunda-bpm-platform/blob/master/engine/src/main/java/org/camunda/bpm/container/impl/jmx/deployment/StartProcessEngineStep.java?source=c#L120

 

Example of how to do it using Spring:

https://github.com/camunda/camunda-bpm-platform/blob/master/engine/src/test/resources/activiti.cfg.xml#L24

 

The plugin is automatically activated in case you are using the pre-packaged distribution.

 

We are currently working on improving the docs on Job Executor for the next release. Have a sneak preview here:

http://stage.docs.camunda.org/guides/user-guide/#job-executor

(stage.docs...=current, unreleased master)

 

Cheers,

Daniel Meyer

 

-----Ursprüngliche Nachricht-----
Von: camunda-...@googlegroups.com [mailto:camunda-...@googlegroups.com] Im Auftrag von Bernd Rücker (camunda)
Gesendet: Montag, 5. August 2013 08:14
An: john.rich...@gmail.com; camunda-...@googlegroups.com
Betreff: AW: How to execute parallel Multi-Instance Call Activity's

 

Hi John.

 

5 minutes is the default LOCK timeout for jobs. You have to reconfigure this (from my head: JobExecutor.maxLockTime) in order to have threads/tx longer than this timeout (which is by the way not always a good idea - keep in mind that you have other timeouts as well - e.g. Transactions).

 

Cheers

Bernd

Evangelist & Consultant

www.camunda.org/community/team.html

 

-----Ursprüngliche Nachricht-----