Camunda alternative, please help!

[derek...@gmail.com]

We recently were awarded work on a legacy system that included a very poorly implemented jBPM solution. So we performed a trade study and presented it to our customer. Camunda was our # 1 recommendation. Unfortunately they decided they want us to develop our own in-house Java solution. I think this is a very poor choice. But management has decided that is what we must do. Argh!!

So I was wondering if you were in a similar predicament what you might do? I have no idea where to even begin with this in terms of possible approaches/patterns to take.

I found one dated article for a rather simple solution here: http://www.javaworld.com/article/2071865/web-app-frameworks/use-spring-to-create-a-simple-workflow-engine.html#resources

Our system however has more complexities than a simple workflow however including high concurrency, forks, joins, long lived processes and human interaction. So I am at a loss.

Any ideas where to begin looking or what approaches you might take if in a similar situation would be greatly appreciated.

Thank you!

[Jakob Freund]

Honestly I think your customer made a big mistake here. Plus it sounds odd
in a way: If you are supposed to create sth. In a Java Stack, you will
unavoidably make us of existing libraries, for persistence or whatever. I
mean, even Spring is a Java library exactly like Camunda (if you're looking
at the pure engine, just add a .jar and that's about it).

Also, writing your own process engine is just ridiculous. The article you
refer to in the forum post is almost 10 years old - for good reason. It's
sth. You just wouldn't do anymore, today.

So frankly speaking, my own approach would be to say:

"OK customer, I understood you do not want us to make use of a BPM platform
product but rather create our own. We will do that. Here are the ingredients
we will use: Spring, Camunda, … "

I mean, if they balk at that, you could emphasize that you will not use the
"Camunda BPM platform", but only the process engine, a very light-weight
component that you can easily understand and maintain, just like all the
other Java libs you're going to use anyway.

Good luck!

Jakob

-----Ursprüngliche Nachricht-----
Von: camunda-...@googlegroups.com
[mailto:camunda-...@googlegroups.com] Im Auftrag von
derek...@gmail.com
Gesendet: Samstag, 11. Oktober 2014 07:34
An: camunda-...@googlegroups.com
Betreff: [camunda-bpm-users] Camunda alternative, please help!

--
You received this message because you are subscribed to the Google Groups
"camunda BPM users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to camunda-bpm-us...@googlegroups.com.
To post to this group, send email to camunda-...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/camunda-bpm-users/7f91d2c1-0f57-49d7-b43f-d3b3511c5acf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Meyer]

Hi Derek,

The tricks of the trade:

The core of your workflow engine needs to address three things:

1. Structure: defines the Graph Model for representing the Structure of processes. Here you have
1. Activities with parent/child relation-ship (cf. Activity in camunda)
2. Connections / Transitions / Sequence Flows (whatever you want to call them)
3. Provide possibility to attach additional "properties" to an activity, such as declarations of events, io/mappings etc... 
   
   
2. State: represent the state of an Instance of the Graph Model. 
1. Graph State: Structurally this is almost always a tree. For simplicity you may want to make it isomorphic to the structure spanned by the parent-child relationships in your graph model. (cf. Executions in camunda)
2. Variables: ideally you have hierarchical variable scopes with similar visibility properties (cf. VariableScope  in camunda)
3. Instances of your additional "properties" such as Event Subscriptions etc...
   
   
3. Behavior: defines how the Graph Model is executed. It has proven useful to separate the following two aspects
1. What happens between activities: aspects of the behavior responsible for traversing the graph and making changes to the graph state / instance. (cf. AtomicOperations in camunda)
2. What happens inside activities: most process languages provide different "types" of activities: BPMN has user tasks, service tasks, script tasks etc... (cf. Activity Behaviors in camunda).

The next issue is defining a persistence model and a matching consistency and threading model. 

• You need to maintain data consistency of your persistent state (ie. the physical representation of your "state" as defined above in a persistent data store). In general, workflow engines have to face three consistency challenges:
• Racing incoming signals: multiple threads want to continue the same activity instance => only one can win. In BPMN you have this with catching Message Events, User tasks, ... 
• Synchronization: need to decide whether to continue (cf. joining parallel & inclusive gateway, parallel multi instance, compensation ...)
• Cancel Activity Instance: one activity instance is interrupted / cancelled by an other one (cf. Boundary event, Terminate End Event ... in Bpmn)
  
  
• Key question here: do you need intra-process instance concurrency (multiple threads active within a single process instance at the same time)? Think about
• selecting a persistence model which allows for concurrent updates (in camunda each Execution is an individual row in the database which can be updated separately)
• selecting a locking strategy (pessimistic vs. optimistic) for ensuring consistency among multiple persistent entities. (in camunda we use Optimistic Locking)

• Selecting a database / persistence store: the consistency model needs to be layered on top of the consistency guarantees / properties provided by the underlying technology.

Next, think about integration with an environment and / or programming model:

In a Java EE / Spring context you traditionally have a programming model centered around Request / Response, Transactions and layered architectures. You want to think about how your persistence, consistency and threading model integrates here.

Then there is additional things like

• API design,
• Asynchronous Background Work, scheduling etc...
• Managing deployments, versionning, ...
• Testability,
• Database Schema Management and migration,
• 
  
• ............ (add many more dots here :)

Important comment:

It is very hard to get this right. 

And even if you do, it will not solve your problems :) 

Usually when people decide to write their own workflow engines after having made bad experiences with an existing one, they did not have problems related to the core topics addressed above. 

If you used Jbpm 3, your workflow engine had a good enough graph modal for representing structure, state representation model was good, behavior implementation was also good, it had an adequate threading and consistency model and it integrated with Spring and JTA. I bet that your problems were not related to this. Usually the problems are somewhere else, usually related to

• lacking experience with the workflow engine,
• versionning and upgrades,
• poor application architecture,
• operations & monitoring,
• bugs in user code,
• ...

All of these things are not related to the core of the process engine as such. So how would writing a new core help you solve these problems? At best, your core will be as good as the jBPM core. Or maybe a bit better, who knows? Point is: it does not matter, because after having spent a lot of effort into designing a good core, you will still not have solutions to your actual problems.

Or to put it differently: if your pain is related to lack of experience of how to use a workflow engine and designing a well functioning and scalable application on top of it, then building a workflow engine will not help you :) 

Maybe this gives you a little bit more ammunition for convincing your customers...

Good luck and Regards,

Daniel

[derek...@gmail.com]

Thank you very much for the good advice Jakob and Daniel. If there is a chance I could convince them to at least use the Camunda Engine as Jakob suggests vs. writing our own from scratch how usable is it by itself with Spring integration for example?

If they give me the opportunity to prove that it works within a reasonable amount of time and effort maybe they would accept this approach. I have to present it in such a way that they don't think it will take a 'BPM expert' do design and maintain. That is one of there great apprehensions given our compressed timelines to deliver a working system.

Thanks again for you sage advice and knowledge within this BPM arena.

Derek

[Daniel Meyer]

Hi Derek,

You could start by working through the Spring Getting stated Example: 

http://docs.camunda.org/latest/guides/getting-started-guides/spring-framework/

It demonstrates how to setup an application with embedded process engine and deploy it to a vanilla apache tomcat server.

You could combine  this by deploying the camunda standalone webapp (including cockpit and tasklist) and configure them to connect to the same database.

http://camunda.org/download/

Cheers,

Daniel