Start subprocess programatically - class cast exception

[androi...@gmail.com]

Hi,

background: i love searching for usage of business data fields by java IDE. So i want to avoid using process variable names in bpmn itself, instead i want to access them only from java.

problem: I got stuck on calling of subprocess from java.

I started with:
public void invokeSubprocess(DelegateExecution execution) throws Exception {
ProcessDefinitionImpl processDefinition = Context.getProcessEngineConfiguration().getDeploymentCache().
findDeployedLatestProcessDefinitionByKey("longLastingAsyncActivity");
PvmProcessInstance instance = ((ActivityExecution)execution).createSubProcessInstance(processDefinition, execution.getProcessBusinessKey());
instance.start();

That worked find untill subprocess finished. Then it failed in PvmAtomicOperationProcessEnd.eventNotificationsCompleted
on line
subProcessActivityBehavior = (SubProcessActivityBehavior) activity.getActivityBehavior();

as my activity behavior is of type ServiceTaskExpressionActivityBehavior.


Is there other way to start subprocess (so that main process waits and than i also see in cockpit that mainprocess has a subprocess)?
I can imagine workarounds using events etc, but then i don't see it in cockpit.

easiest would be to as if-instanceOf check into PvmAtomicOperationProcessEnd.


thank you

best regards
jano

[thorben....@camunda.com]

Hi Jano,

Could you please clarify your use case a little further? I don't understand how your background description relates to the problem you describe. Furthermore, I ask myself why you don't model a call activity in your BPMN process?

Cheers,
Thorben

[j...@atlas.cz]

Hi,

at company i work for, we have often quite big fully automated (no user tasks) processes with quite a lot of data. Often process is started with XML (e.g. payload of webservice) and then during execution calculations for necessary provisioning steps are done. These results are stored as smaller xmls or individual fields - all as process variables. Sometimes we have maybe even 50-100 of process variables (single variable is easier to change by operations team, then more complex xml structure).
During evolution of processes, we are often solving questions like:
- can i reorder these activities?
- can i change meaning of these variable?
etc.
To answer them, we have to find all usages (reads and writes) of relevant fields and process variables.

Currently we are using some WF (work flow ~= process engine) tool where process is defined in (or can be exported into) some kind of text/xml format, where it's possible to search using fulltext search. You can imagine how much time it takes to find something, especially if it's longer chain (e.g. from input XML to process variable, then mapped to some activity, then to subprocess input xml, then again to some process variable etc.)
Camunda seems to be something similar, as BPMN is xml.

In WF tool we managed situation by implementing all activities in way, that there is no process variable usage in process definition itself (except defining parameter itself - but this step is not necessary with camunda - which is btw. both good and bad), but instead java implementation of underlaying activities is reading/writing process variables, while referencing them with constants defined on single place per process (in fact it's more soffisticated, so that it's type safe, includes some validation etc). This sacrificed potential reuse (as activity was now bound to specific names of process variables) of activity implementation, but it's not an issue, as we don't have much reuse (and in fact, reusable parts were moved one layer deeper, where layer called by WF is then only mapping process variables to reusable layer).
Hence all process variables are read and written only in java, including conditions on gateways/transitions => no more need to fulltext search in XML. Process definition contains only calls to activities, but as we have dedicated package for each process definition, it's very clear without need for fulltext search.
Actually we were able to fully separate concrete activity implementation from WF engine we are using and our existing code should be usable without changes also in camunda, we only have to change framework which is glue between WF engine and implementation of activities.

I was successul to apply similar approach for service tasks with camunda, also for spawning individual processes, but so far I don't have success with subprocess called from java. Public api only supports creating of independent process instance, but in most of situations, we would like to wait for subprocess to finish, before advancing in current execution and definitelly in all cases we require to see subprocess in cockpit, so it's easy to go into subprocess (specially when something went wrong).
Hence i was digging into implementation of CallActivityBehavior to simulate it from java, but it failed on error i described before.

Do you have a hint which way to go? Or if this is right direction, how to solve it?

I started to think also about solution, where we go through bmpn file and generate java classes for that, that would represent variables usage and mappings in BPMN file, so that we can use java search.
But it is still not prefered way. Actually, while doing simple testing processes, i found myself struggling while defining process as after few steps i wasn't sure what process variable name i used, so i had to click to other activity and back quite often only to copy variable name. With java i just reference class per process type and intellisense offers all possibilities + because of compilation i can be sure i didn't make typo in variable name.

There are also other topics, where this could help us. E.g. we have requirement to see in cockpit for each activity, which variables are used for reading and writing. This will help operations team. IF always mapped in bmnm file, we could parse this. If always called from java, we could either parse, or use reflection, or even determine in runtime what is used. With last approach, we can in addition detect, which variables are declared to be used, but are not invoked for certain time, hence it's signal to inspect code and potentially remove input/output variable mapping.

[thorben....@camunda.com]

Hi Jano,

Thanks for your very detailed description. As far as I understand, you basically replace the behavior of every activity by your own activity behavior such that variables can be set directly in Java code. Now you have extended ServiceTaskExpressionActivityBehavior. Creating a subprocess instance fails with the mentioned class cast exception.

My recommendation would be to implement the interface SubProcessActivityBehavior in your custom activity behavior. It provides two callbacks that are called when the created process instance is about to complete and after it has completed. Especially the latter is important because you must also ensure to continue process execution in the parent process instance after completion. Confer the implementation of the "completed" method in the class CallableElementActivityBehavior [1] (being the super class of CallActivityBehavior). There the method "leave" is called. You should also call it after completion or else the process instance gets stuck.

I hope that helps.

Cheers,
Thorben

[1] https://github.com/camunda/camunda-bpm-platform/blob/master/engine/src/main/java/org/camunda/bpm/engine/impl/bpmn/behavior/CallableElementActivityBehavior.java

[j...@atlas.cz]

Hello Thorben,

I'm not replacing any behavior till now and i didn't plan to do so. I'm using spring integration and i'm calling spring beans methods like #{myProcessBean.myActivityName()} without any arguments (alternative would be to pass current execution as parameter).

My myProcessBean uses bean MyProcessVariables, which contains getter setters for all global and local variables we want to use. Implementation of these methods
Context.getBpmnExecutionContext().getExecution() to fetch current execution.
On execution we call public api methods to read/write variables.

Important for us is that all access to variables is via getters/setters defined on MyProcessVariables.

This all works fine, unless i want to create subprocess as there is no public API for that. I tried to copy code from CallActivityBehavior, so i can starting new subprocess programatically and hence i can use setters of MySubprocessVariables to set initial variables of subprocess. It all worked fine, until subprocess finished, as it's expecting behavior to be CallActivityBehavior, but it's ServiceTaskExpressionActivityBehavior.

Thanks for hint. I'll try to subclass ServiceTaskExpressionActivityBehavior and implement missing SubProcessActivityBehavior and then somehow find an extension point, how to use it, so i don't have to change anything in process definisions etc.
Maybe i can just override BpmnParse.execute and postprocess created structure.
Do you have other ideas how to configure new behavior to be used? I would see it nice, if there are simple behavior creation methods in BpmnParse, which i can simply override to return different instance.

i'll post my results

thank you a lot
best regards
jano

[thorben....@camunda.com]

Hi Jano,

Thanks again for the clarification.

Here's my feedback:
1. Please don't model a service task when you actually need a call activity; That's also the reason why there is no public API for creating sub process instances programmatically. BPMN has the construct of the call activity for these cases that make the difference to a regular service visible.
2. Now with the engine's default implementation of a call activity, I'm not sure if you can easily achieve your goal. You could subclass/copy/reimplement CallActivityBehavior in the way you like and replace the behavior instances for your process definitions by implementing a Bpmn ParseListener, see [1] for an example. The interface has a callback "parseCallActivity" that is called whenever a call activity has been parsed. In your implementation, you could replace the activity behavior there. Btw, there is no need to subclass ServiceTaskExpressionActivityBehavior then.

Cheers,
Thorben

[1] https://github.com/camunda/camunda-bpm-examples/tree/master/process-engine-plugin/bpmn-parse-listener

[j...@atlas.cz]

Hi,

so far i'm testing following parse listener:

    @Override
    public void parseCallActivity(Element callActivityElement, ScopeImpl scope, ActivityImpl activity) {
        if (activity.getActivityBehavior() instanceof CallActivityBehavior) {
            CallActivityBehaviorWithMapping activityBehavior = new CallActivityBehaviorWithMapping();
            activityBehavior.setCallableElement(((CallActivityBehavior)activity.getActivityBehavior()).getCallableElement());
            activity.setActivityBehavior(activityBehavior);
        }
    }    

with following behavior

public class CallActivityBehaviorWithMapping extends CallActivityBehavior {

    protected void startInstance(ActivityExecution execution, VariableMap variables, String businessKey) {
        CallSubprocessInfo subprocessInfo = invokeJava(execution);

        ProcessDefinitionImpl definition = getProcessDefinitionToCall(execution, subprocessInfo);
        PvmProcessInstance processInstance = execution.createSubProcessInstance(definition, businessKey);
        processInstance.start(subprocessInfo.getVariables());
    }

    protected CallSubprocessInfo invokeJava(ActivityExecution execution) {
        Object tmp = callableElement.getDefinitionKeyValueProvider().getValue(execution);
        if (tmp == null || false == tmp instanceof CallSubprocessInfo) {
            throw new RuntimeException("Expected return type "+CallSubprocessInfo.class.getName()+" for call activity...");
        }
        CallSubprocessInfo p = (CallSubprocessInfo) tmp;
        return p;
    }
   
    public ProcessDefinitionImpl getProcessDefinitionToCall(VariableScope execution, CallSubprocessInfo p) {
        String processDefinitionKey = p.getProcessDefinitionKey();
        DeploymentCache deploymentCache = CallableElementUtil.getDeploymentCache();
        return deploymentCache.findDeployedLatestProcessDefinitionByKey(processDefinitionKey);
    }
}

and it seems to work. Now i have to enhance it to support all we need.

    <bpmn2:callActivity id="invokeSubprocess" camunda:async="true" name="invokeSubprocess" calledElement="#{springTask.invokeSubprocess2()}">
      <bpmn2:extensionElements>
        <camunda:in businessKey="#{execution.processBusinessKey}"/>
      </bpmn2:extensionElements>
      <bpmn2:incoming>SequenceFlow_13</bpmn2:incoming>
      <bpmn2:outgoing>SequenceFlow_3</bpmn2:outgoing>
    </bpmn2:callActivity>

Maybe i have to put expression elsewhere from calledElement as it seems not to be valid. But it has to be on field, which is available in modeler, so we don't have to extend that one.

best regards
jano

[j...@atlas.cz]

i extended my solution of CallActivityBehavior subclass with call of java (via juel expression) where in java i do mapping from finished subprocess variables to resumed main process variables.
During that I hit a behavior,, which is suspicious:
When calling  Context.getBpmnExecutionContext().getExecution() in methods 'completed' and 'completing' of SubProcessActivityBehavior, i get back execution of child process. Is this really ok?
It was not a big issue, as in completing method there are arguments on method, which are enough to map parameters, i'm just wondering, as i build expect to get already execution of main process, as methods are on activity behavior belonging to main process.

[thorben....@camunda.com]

Hi Jano,

Thanks for reporting this. I was able to identify a bug with expression resolving, see [1].

Cheers,
Thorben

[1] https://app.camunda.com/jira/browse/CAM-3962

[j...@atlas.cz]

Hi,

before this bug is resolved, could you give me a hint, how to wrap code to execute EL expression in context of parent process application, when returning from subprocess call?

I have subclass of CallActivityBehavior, where is also following method:

    @Override
    public void completing(VariableScope execution, VariableScope subInstance) throws Exception {
        if (responseMappingExpression != null) {
            subInstanceVariableScope.set(subInstance);
            mainInstanceVariableScope.set(execution);
            try {
                // resolve expression by executing it.
                Context.getProcessEngineConfiguration().getExpressionManager().createExpression(responseMappingExpression).getValue(execution);
            } finally {
                subInstanceVariableScope.remove();
                mainInstanceVariableScope.remove();
            }
        }
    }

subInstanceVariableScope and mainInstanceVariableScope are threadlocal, which are read by framework code, which is executed from method, which should be called while expression is resolved.

My problem is that now we refactored sample main processes and sample subprocess to target architecture -> we split it to two separate applications and it stopped working as it's trying to resolve expression in wrong application.

Maybe for fixing the bug 3962, there is probably needed much more, as there is need to be able to resolve access both - variables of subprocess + variables of parent process + beans of parent process .... which could get tricky, if there are coflicts on names.
But for my quick solution i only need access to spring beans of parent applications ... no variables at all.

[thorben....@camunda.com]

Hi Jano,

Have a look at the method CommandContext#performOperation [1]. The implementation shows how to make a context switch to the class loader of a process application. Feel free to ask if you need assistance with that.

Cheers,
Thorben

[1] https://github.com/camunda/camunda-bpm-platform/blob/master/engine/src/main/java/org/camunda/bpm/engine/impl/interceptor/CommandContext.java#L114

[j...@atlas.cz]

Hello Thorben,

thank you. This worked fine.

btw. Related question came to my mind, i want to test it next days:
We planned to use heterogenous cluster. Wouldn'tCAM-3962be even bigger in such setup? When subprocess is finished, mapping of variables to parent process would have to be done on other tomcat as only that one contains correct application deployed.

In fact during debuging i realized AbstractBpmnActivityBehavior.propagateError() is also calling subProcessActivityBehavior.completing(superExecution, processInstance); and doing mapping of parameters (even if it's in turn rolled back).

actually it's more general issue .. how can platform do a context switch to other application, when other application is not deployed on current tomcat node?

Are there some restrictions in heterogenous clusters about process-subprocess deployments or similar ones?

[thorben....@camunda.com]

Hi Jano,

how can platform do a context switch to other application, when other application is not deployed on current tomcat node?

The short answer is: It cannot.

When exchanging variables between calling and called process, there may be expressions (and therefore beans) involved from both processes and their respective process applications. This procedure is currently not separatable and doing so is probably not a straight forward task.

If you have such a case, using tight boundaries of asynchronous continuations and delaying bean access after the next asynchronous continuation might be a way to go. In particular, the call activity could be asyncAfter and instead of accessing beans in the call activity's output parameter expression, you can pass an absolute value and invoke the bean after the asyncAfter continuation.

Cheers,
Thorben