BPMN Process Engine Output of Executed Process ID

[mp4...@att.com]

I need to understand when, by default, the Camunda process engine outputs the Process ID and related content. Example:

{

"links": [

{
"method": "GET",
"href": "http://mybpmnserver.domain.com:8880/camunda/api/engine/engine/default/process-instance/bfb6828a-de60-12f5-d4a2-0050569d1aa4",
"rel": "self"
}
],
"id": "bfb6828a-de60-12f5-d4a2-0050569d1aa4",
"definitionId": "testWorkflow:8:f53a4371d-ef3c-22c5-b720-d33efffd1a467",
"businessKey": null,
"caseInstanceId": null,
"ended": true,
"suspended": false

}

Assume no asynchronous processing. In a standard JBoss container, it appears that this is output when the process is complete. In my particular case, multiple call activities invoke other processes, all of which complete successfully. So, if the entire workflow takes 16 seconds, then I get the response at that time.

However, we noticed that if we set the "Asynchro...Before" property on the first step, we get immediate output. However, the workflow goes no further. This is undoubtedly due to the fact that we don't know how to use this feature yet. It does show that it is possible to output the response at the start.

What we want is for the response with the Process ID to be sent at the start of the process every time and then the process can move on. That way the invoking external client has the Process ID it needs to keep track of the process and it can terminate itself should it wish to do so.

How do we make Camunda output the response at the start of the process and then continue processing?

Thanks.

[webcyberrob]

Hi,

I think a key concept to understand is the async continuation. Have a look at [1] as its a good description of how client threads and job executor threads work within the engine.

In the case of no async continuations, the entire process will run in the context of the client's thread. In addition, the process state won't be flushed to the database until the process is complete.

If the use an explicit async continuation (async before/after) or an implicit wait state (timer, receive task etc), then the process state will be flushed to the database at the point of the continuation.

Once the process state has been flushed, you will be bale to use the get process-instance call to get details. If you re finding that the asynch continuation is not progressing in a timely manner, it could be that the job executor is not running, or its in a sleep state for a period of time. All of this is configurable.

Anyway, a brief overview to get you started,

regards

Rob

[1] https://docs.camunda.org/manual/7.4/user-guide/process-engine/transactions-in-processes/#why-asynchronous-continuations

[mp4...@att.com]

Thank you for the reply. Asynchronous behavior isn't really the issue here. I mentioned it only to provide one method whereby the process ID is returned to invoking client immediately. The failure of the workflow to proceed beyond that step is a consequence of my ignorance.

What I really want to know is how to explicitly return the process ID information to the invoking client (i.e a REST client) when the workflow first starts such that it will continue to completion, no matter how long that takes.

I've asked a similar question about communication back to the invoking client application on a separate post and was told something about the use of callbacks that I didn't completely understand (again, forgive my ignorance).

There is clearly a method or statements within Camunda that are responsible for sending the message to whatever started the workflow because it happens every time. I just want to know what they are so I can mimic them somehow at a point earlier that completion of the process.

Thanks again for taking the time to reply.

[webcyberrob]

Hi,
I guess that depends on your client integration style... If you look at the rest api, there are numerous start process instance mechanisms. These will return the instance I'd. Hence you now have choice. You could bundle your own Api which may perform a start and retrieve process info. You could do this by wrapping and extending the rest api, or you could crest spring web service calls and wrap the Java spirit...

I hope this helps a little...

[mp4...@att.com]

Thank you for the reply, but unfortunately that does not help. Perhaps I need to be more clear on what I'm looking for.

Somewhere within the Camunda code base is a statement or statements that produce the JSON encoded "response" to a process start request as shown in the original post.

I want to know where those statements are (i.e. the specific Java file that contains them) and preferably the line number(s) of the statement(s). If can see what Camunda is doing to send this message back to the client, I'm hoping I can figure a way to reproduce that behavior "on demand".

[Sebastian Menski]

Hi,

the code which produces this response is the ProcessDefinitionResourceImpl [1] in the REST API. It starts a process instance using the RuntimeService [2] and returns this instance [3]

as ProcessInstanceDto [4]. The transformation to JSON is done by jackson.

Cheers,

Sebastian

[1] https://github.com/camunda/camunda-bpm-platform/blob/master/engine-rest/engine-rest/src/main/java/org/camunda/bpm/engine/rest/sub/repository/impl/ProcessDefinitionResourceImpl.java#L92

[2] https://github.com/camunda/camunda-bpm-platform/blob/master/engine-rest/engine-rest/src/main/java/org/camunda/bpm/engine/rest/sub/repository/impl/ProcessDefinitionResourceImpl.java#L133

[3] https://github.com/camunda/camunda-bpm-platform/blob/master/engine-rest/engine-rest/src/main/java/org/camunda/bpm/engine/rest/sub/repository/impl/ProcessDefinitionResourceImpl.java#L115

[4] https://github.com/camunda/camunda-bpm-platform/blob/master/engine-rest/engine-rest/src/main/java/org/camunda/bpm/engine/rest/dto/runtime/ProcessInstanceDto.java

[PEOPLES, MICHAEL P]

Thank you for the response. 

 

I have reviewed that file and the lines referenced below.  What does “DTO” mean? I see it all over the code and in the logs. 

 

Now I need help to understand which statement actually sends the response back to the calling client.  I’m assuming there is a point in the code where the message is actually sent. 

 

What I’m hoping is that by understanding how that statement works, I can use it to populate either a message similar to what is sent when the process is complete or a message (properly encoded) of my own that would provide the relevant information to the client.

 

Thank you.

 

Michael

[Sebastian Menski]

Hi,

DTO stands for data transfer object and is a concept [1] use to encapsulate data which should be transferred, for example to a remote process. The actual response is not generated by the Camunda REST API as

it uses the JAX-RS [2] standard. The response is than produced by the used JAX-RS implementation.

Nevertheless I think you have to understand the main concept of synchronous and asynchronous process execution. 

There is basically one entry point to start a process:

runtimeService.startProcessInstanceByKey
runtimeService.startProcessInstanceById

If you want to start a process you have to call one of these methods. These methods will block as long as the process is not finished or has reached a wait state. Which also means that no information

about this process will be saved inside the database until it is finished or reached a wait state. As everything is done in the thread which called these methods.

So I don't think your use case of getting process information while it is synchronously is running is not possible using the Camunda APIs.

Cheers,

Sebastian

[1] https://en.wikipedia.org/wiki/Data_transfer_object

[2] https://en.wikipedia.org/wiki/Java_API_for_RESTful_Web_Services

[PEOPLES, MICHAEL P]

Just so I understand this, if we want the process ID at the “start” of a process, then we must use the “Asynchronous Before” or “Asynchronous After” properties on the step or task where we want the process ID returned to the client. 

 

This implies that we will also need to learn how to configure our processes to utilize these properties such that they continue to run after the process ID is sent.

 

Michael Peoples (mp4783)

Global Customer Service Dev Ops

Office: +1 614-886-0923

Mobile: +1 614-886-0923

 

Principal Applications Developer

mp4...@att.com

--
You received this message because you are subscribed to a topic in the Google Groups "camunda BPM users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/camunda-bpm-users/-bg2bn86zOo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to camunda-bpm-us...@googlegroups.com.
To post to this group, send email to camunda-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/camunda-bpm-users/af1b738a-60f6-4fed-adaa-491272418822%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[webcyberrob]

Hi Michael,

Yes,  you've got it!

Asynch before/after is really easy. Lest assume you configure asynch after on the start event of your process. You just have to set the flag using the Eclipse IDE etc. Then your done. All thats going on behind the scenes is rather than the process running in your 'client thread', you re handing control to the background job executor.

regards

Rob

[Sebastian Menski]

Hi Michael,

if you use asynchronous continuation you don't have to configure you process to "continue". The concept is that if a process reaches a asynchronous continuation

the engine creates a job to continue the process and saves the state to the database. A separate thread, called the job executor, than periodically checks the

database for new jobs to execute. If a job is found it is hand over to a thread pool which will execute it. So your process should continue if you didn't disabled the

job executor. It might take some time depending on the number of jobs or if the job executor found no jobs for a period of time it will decrease the database

polling frequency. You can read more about the job executor and its main concepts in the documentation [1].

Cheers,

Sebastian

[1] https://docs.camunda.org/manual/7.4/user-guide/process-engine/the-job-executor/

[PEOPLES, MICHAEL P]

I hate to ask this, but I’m obviously still not understanding what needs to be done to keep the process moving along.  I did read the documentation and used the ‘camunda:asyncBefore="true"’ attribute (checking the box puts in “camunda:async=true”, so this was a manual update) in the service task definition.  I thought this would allow the process to continue to completion, but it just stops and waits.  Here’s the code from this simple workflow:

 

<?xml version="1.0" encoding="UTF-8"?>

<bpmn2:definitions xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:bpmn2="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:camunda="http://camunda.org/schema/1.0/bpmn" xmlns:dc="http://www.omg.org/spec/DD/20100524/DC" xmlns:di="http://www.omg.org/spec/DD/20100524/DI" xsi:schemaLocation="http://www.omg.org/spec/BPMN/20100524/MODEL BPMN20.xsd" id="_7-gSIF09EeWE96cFqGLI8w" exporter="camunda modeler" exporterVersion="2.7.0" targetNamespace="http://bpmn.io/schema/bpmn">

  <bpmn2:process id="log-message-wf" name="LogMessageWorkFlow" isExecutable="true">

    <bpmn2:startEvent id="StartEcho">

      <bpmn2:outgoing>SequenceFlow_2</bpmn2:outgoing>

    </bpmn2:startEvent>

    <bpmn2:serviceTask id="ServiceTask_1" camunda:class="com.att.gcsdevops.ajsc_camunda_archetype_40_mp4783.workflow.LogMessageDelegate" camunda:asyncBefore="true" name="Perform Echo Assignment">

      <bpmn2:incoming>SequenceFlow_2</bpmn2:incoming>

      <bpmn2:outgoing>SequenceFlow_3</bpmn2:outgoing>

    </bpmn2:serviceTask>

    <bpmn2:sequenceFlow id="SequenceFlow_2" name="" sourceRef="StartEcho" targetRef="ServiceTask_1"/>

    <bpmn2:endEvent id="EndEvent_1">

      <bpmn2:incoming>SequenceFlow_3</bpmn2:incoming>

    </bpmn2:endEvent>

    <bpmn2:sequenceFlow id="SequenceFlow_3" name="" sourceRef="ServiceTask_1" targetRef="EndEvent_1"/>

  </bpmn2:process>

  <bpmndi:BPMNDiagram id="BPMNDiagram_1">

    <bpmndi:BPMNPlane id="BPMNPlane_1" bpmnElement="log-message-wf">

      <bpmndi:BPMNShape id="_BPMNShape_StartEvent_3" bpmnElement="StartEcho">

        <dc:Bounds height="36.0" width="36.0" x="191.0" y="253.0"/>

      </bpmndi:BPMNShape>

      <bpmndi:BPMNShape id="_BPMNShape_ServiceTask_4" bpmnElement="ServiceTask_1">

        <dc:Bounds height="80.0" width="100.0" x="277.0" y="231.0"/>

      </bpmndi:BPMNShape>

      <bpmndi:BPMNEdge id="BPMNEdge_SequenceFlow_2" bpmnElement="SequenceFlow_2" sourceElement="_BPMNShape_StartEvent_3" targetElement="_BPMNShape_ServiceTask_4">

        <di:waypoint xsi:type="dc:Point" x="227.0" y="271.0"/>

        <di:waypoint xsi:type="dc:Point" x="277.0" y="271.0"/>

      </bpmndi:BPMNEdge>

      <bpmndi:BPMNShape id="_BPMNShape_EndEvent_103" bpmnElement="EndEvent_1">

        <dc:Bounds height="36.0" width="36.0" x="427.0" y="253.0"/>

      </bpmndi:BPMNShape>

      <bpmndi:BPMNEdge id="BPMNEdge_SequenceFlow_3" bpmnElement="SequenceFlow_3" sourceElement="_BPMNShape_ServiceTask_4" targetElement="_BPMNShape_EndEvent_103">

        <di:waypoint xsi:type="dc:Point" x="377.0" y="271.0"/>

        <di:waypoint xsi:type="dc:Point" x="427.0" y="271.0"/>

      </bpmndi:BPMNEdge>

    </bpmndi:BPMNPlane>

  </bpmndi:BPMNDiagram>

</bpmn2:definitions>

 

 

At present, we don’t want to use asynchronous behavior for the purposes you detail in the documentation (i.e. setting breakpoints).  We want to use it to get Camunda to send the Process ID back as quickly as possible to the client and the we always want the process to continue to completion immediately after sending the Process ID.  In other words, the process should keep going because there are no other triggers or events that are required for it to run to completion.

 

I’m not sure if this behavior is a consequence of our odd Jetty container configuration or something else.

 

Michael Peoples (mp4783)

Global Customer Service Dev Ops

Office: +1 614-886-0923

Mobile: +1 614-886-0923

 

Principal Applications Developer

mp4...@att.com

 

From: camunda-...@googlegroups.com [mailto:camunda-...@googlegroups.com] On Behalf Of Sebastian Menski

Sent: Tuesday, February 23, 2016 4:26 AM
To: camunda BPM users

--

You received this message because you are subscribed to a topic in the Google Groups "camunda BPM users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/camunda-bpm-users/-bg2bn86zOo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to camunda-bpm-us...@googlegroups.com.
To post to this group, send email to camunda-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/camunda-bpm-users/35c2d7ed-e33f-4399-be12-3b37a2a79d6a%40googlegroups.com.

[webcyberrob]

Hi Michael,

the behaviour I would expect given your process model is:

Initiating thread wiil run the process up to, but not including Service task 1. AT this point, the process state will be flushed to the DB and the process instance ID will be available (if you called the Rest or Java API). In addition, a job will be created in the job table with a null due date indicating work for the job executor.

The behaviour of the job executor is to periodically poll the job table. If there is work to be done, it reserves the job and allocates it to a thread from the executor thread pool. If there is no work to be done, the job executor may sleep for a configurable period. Hence if the job executor is sleeping, there can be a small amount of latency for the process instance to continue.

It sounds like you may not have a job executor configured or started. Im not familiar with a Jetty setup so you may have to help us help you...

When you say its just stops and waits, where do you see this? Do you mean you see the process ready to run in the Cockpit view?

In cockpit, are the 'traffic lights' for the process instance green?

Are you running a shared or embedded engine? If its shared, how is the job executor configured? If embedded, again how configured and how started?

As an aside, there seems to be a little bit of wanting your cake, but eat it too. What I mean is you have to embrace either synchronous or asynchronous execution semantics. You can't have both. Hence with synchronous execution, your client will block until the process completes. At this point, the process instance id and its completed state are available to the calling context. If you want to return the process instance id to the calling client aggressively, then for the client to use this effectively, it cant be blocking and thus you are embracing asynchronous processing. Hence not sure if your requirement is for synchronous execution, which means you will get the process ID when its finished, or you are tolerant of asynch processing which will require a properly configured job executor...

regards

Rob

[PEOPLES, MICHAEL P]

I’ve tested what you describe below in the standard JBoss distro and it works as you said.  The process outputs the Process ID when it reaches the “Asynchronous…” point and then continues to completion.

 

However, our implementation of the embedded engine in Jetty doesn’t seem to work that way.  Could there be something misconfigured in the engine?  Is there a global “do not continue asynchronously” setting that got set?  Have we just borked up the engine somehow?

 

Michael Peoples (mp4783)

Global Customer Service Dev Ops

Office: +1 614-886-0923

Mobile: +1 614-886-0923

 

Principal Applications Developer

mp4...@att.com

 

From: camunda-...@googlegroups.com [mailto:camunda-...@googlegroups.com] On Behalf Of Sebastian Menski

Sent: Tuesday, February 23, 2016 4:26 AM
To: camunda BPM users

--

You received this message because you are subscribed to a topic in the Google Groups "camunda BPM users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/camunda-bpm-users/-bg2bn86zOo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to camunda-bpm-us...@googlegroups.com.
To post to this group, send email to camunda-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/camunda-bpm-users/35c2d7ed-e33f-4399-be12-3b37a2a79d6a%40googlegroups.com.

[PEOPLES, MICHAEL P]

It works as you described in earlier emails when run in a JBoss container.  This problem appears to be isolated to our own slightly unusual implementation of the embedded engine in a Jetty container.  I’ve ask Sebastian about it and our own folks may have a look also.

 

Thank you for your generous responses.  I hope to be able to help others in the future as my own experience grows.

 

Michael Peoples (mp4783)

Global Customer Service Dev Ops

Office: +1 614-886-0923

Mobile: +1 614-886-0923

 

Principal Applications Developer

mp4...@att.com

 

To view this discussion on the web visit https://groups.google.com/d/msgid/camunda-bpm-users/ed7bcc42-909e-46ef-b69e-c6dcf1ee85bf%40googlegroups.com.

[PEOPLES, MICHAEL P]

Please disregard the message below.  Our Jetty container Camunda instances are allowing the process to run to completion, it’s just a lot slower than the JBoss instance.

 

Thanks.

 

Michael Peoples (mp4783)

Global Customer Service Dev Ops

Office: +1 614-886-0923

Mobile: +1 614-886-0923

 

Principal Applications Developer

mp4...@att.com

 

From: PEOPLES, MICHAEL P
Sent: Wednesday, February 24, 2016 8:08 AM
To: 'camunda-...@googlegroups.com'
Subject: RE: [camunda-bpm-users] Re: BPMN Process Engine Output of Executed Process ID

 

I’ve tested what you describe below in the standard JBoss distro and it works as you said.  The process outputs the Process ID when it reaches the “Asynchronous…” point and then continues to completion.

 

However, our implementation of the embedded engine in Jetty doesn’t seem to work that way.  Could there be something misconfigured in the engine?  Is there a global “do not continue asynchronously” setting that got set?  Have we just borked up the engine somehow?

 

Michael Peoples (mp4783)

Global Customer Service Dev Ops

Office: +1 614-886-0923

Mobile: +1 614-886-0923

 

Principal Applications Developer

mp4...@att.com

 

From: camunda-...@googlegroups.com [mailto:camunda-...@googlegroups.com] On Behalf Of Sebastian Menski

Sent: Tuesday, February 23, 2016 4:26 AM
To: camunda BPM users

--

You received this message because you are subscribed to a topic in the Google Groups "camunda BPM users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/camunda-bpm-users/-bg2bn86zOo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to camunda-bpm-us...@googlegroups.com.
To post to this group, send email to camunda-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/camunda-bpm-users/35c2d7ed-e33f-4399-be12-3b37a2a79d6a%40googlegroups.com.