Microservice orchestration with JBPM

[Matej Lazar]

I’m looking for using JBPM as a standalone Microservice orchestrator, deployed on OpenShift.

The process definition should be simple, it should contain only an ItemHandler per service.

ItemHandler will be used to call remote service via REST. Some service operations are long-running, which would make sense to have an ItemHandler that waits for a REST callback to get the completion notification and the result.

“RestCallbackWorkItemHandler” have to support cancel, time-out and in/out variable mapping.

What is the best way to implement RestCallbackWorkItemHandler, which waits for a REST call-back? How to register a callback endpoint?

Another thing I would like to get from the JBPM server is a list of all running process instances with an option to filter the instances based on variables or Colleralation Keys. This way I don’t have to keep the references in my application and can query the JBPM server when a user want to see the list of running processes.

What is the best way to put things together for an OpenShift deployment ?

I thinking to start from the container image you provide and if required extend it with additional components (eg. RestWithCallbackWorkItemHandler, getProcessesRESTEndpoint).

Thanks,

Matej.

[Maciej Swiderski]

Matej,

On 31 Jul 2018, at 18:49, Matej Lazar <matej...@gmail.com> wrote:

I’m looking for using JBPM as a standalone Microservice orchestrator, deployed on OpenShift.

The process definition should be simple, it should contain only an ItemHandler per service.

ItemHandler will be used to call remote service via REST. Some service operations are long-running, which would make sense to have an ItemHandler that waits for a REST callback to get the completion notification and the result.

“RestCallbackWorkItemHandler” have to support cancel, time-out and in/out variable mapping.

What is the best way to implement RestCallbackWorkItemHandler, which waits for a REST call-back? How to register a callback endpoint?

Actually there is not that much that you need to do - jBPM comes with REST work item handler that is capable of sending REST requests and transform both inputs and outputs. Although it does send the request synchronously all the times and waits for the response. So this could be your first place to look - or if you prefer to use other approach (as it uses http client - so you could prefer to use jaxrs client api) then to build your own handler is not a big issue anyway. There you could control if the execution should be sync or async or shall it wait for external signal or proceed directly.

So the external signal thing is very important - you can send the request and then do not complete work item handler - so the process engine will wait in that node waiting for “reply” - the reply is given by completing the work item from external ssytem by REST call. 

With that you can send a callback URL as part of the REST call and then it will simply send back response (with optional data as map)

Another thing I would like to get from the JBPM server is a list of all running process instances with an option to filter the instances based on variables or Colleralation Keys. This way I don’t have to keep the references in my application and can query the JBPM server when a user want to see the list of running processes.

correlation keys are supported out of the box, so you can assign unique correlationkey when you start a process instance.

What is the best way to put things together for an OpenShift deployment ?

I thinking to start from the container image you provide and if required extend it with additional components (eg. RestWithCallbackWorkItemHandler, getProcessesRESTEndpoint).

that sounds like a best approach - we are in progress of preparing OpenShift images.

Feel free to reach out in case you would like to do the work item implementation, either here or on irc

Maciej

Thanks,

Matej.

--
You received this message because you are subscribed to the Google Groups "jBPM Usage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jbpm-usage+...@googlegroups.com.
To post to this group, send email to jbpm-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-usage/2641d83a-271a-4991-b2c0-9b2abc65e9e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matej Lazar]

On Tuesday, July 31, 2018 at 9:24:22 PM UTC+2, Maciej Swiderski wrote:

Matej,

On 31 Jul 2018, at 18:49, Matej Lazar <matej...@gmail.com> wrote:

I’m looking for using JBPM as a standalone Microservice orchestrator, deployed on OpenShift.

The process definition should be simple, it should contain only an ItemHandler per service.

ItemHandler will be used to call remote service via REST. Some service operations are long-running, which would make sense to have an ItemHandler that waits for a REST callback to get the completion notification and the result.

“RestCallbackWorkItemHandler” have to support cancel, time-out and in/out variable mapping.

What is the best way to implement RestCallbackWorkItemHandler, which waits for a REST call-back? How to register a callback endpoint?

Actually there is not that much that you need to do - jBPM comes with REST work item handler that is capable of sending REST requests and transform both inputs and outputs. Although it does send the request synchronously all the times and waits for the response. So this could be your first place to look - or if you prefer to use other approach (as it uses http client - so you could prefer to use jaxrs client api) then to build your own handler is not a big issue anyway. There you could control if the execution should be sync or async or shall it wait for external signal or proceed directly.

So the external signal thing is very important - you can send the request and then do not complete work item handler - so the process engine will wait in that node waiting for “reply” - the reply is given by completing the work item from external ssytem by REST call. 

With that you can send a callback URL as part of the REST call and then it will simply send back response (with optional data as map)

That's great, of course there already is a REST endpoint to notify completion I don't need my own.

Is it possible to POST a data as single json when completing the task ?

What about a cancel, I would like to send a cancel signal to the process instance and the currently active task should catch the event and invoke cancel on a remote service.

Can I get a KieSession in the work item handler and register an event listener ?

And a time-out, which needs to invoke remote cancel, I probably have to start my own timer in the work item handler?

 

Another thing I would like to get from the JBPM server is a list of all running process instances with an option to filter the instances based on variables or Colleralation Keys. This way I don’t have to keep the references in my application and can query the JBPM server when a user want to see the list of running processes.

correlation keys are supported out of the box, so you can assign unique correlationkey when you start a process instance.

Cool. Are this searches supported using Java client or direct REST calls only?

ps.: I can't find the REST API doc for 7.8.0

 

What is the best way to put things together for an OpenShift deployment ?

I thinking to start from the container image you provide and if required extend it with additional components (eg. RestWithCallbackWorkItemHandler, getProcessesRESTEndpoint).

that sounds like a best approach - we are in progress of preparing OpenShift images.

Feel free to reach out in case you would like to do the work item implementation, either here or on irc

Thanks,

Matej.

[Maciej Swiderski]

On 1 Aug 2018, at 11:56, Matej Lazar <matej...@gmail.com> wrote:

On Tuesday, July 31, 2018 at 9:24:22 PM UTC+2, Maciej Swiderski wrote:

Matej,

On 31 Jul 2018, at 18:49, Matej Lazar <matej...@gmail.com> wrote:

I’m looking for using JBPM as a standalone Microservice orchestrator, deployed on OpenShift.

The process definition should be simple, it should contain only an ItemHandler per service.

ItemHandler will be used to call remote service via REST. Some service operations are long-running, which would make sense to have an ItemHandler that waits for a REST callback to get the completion notification and the result.

“RestCallbackWorkItemHandler” have to support cancel, time-out and in/out variable mapping.

What is the best way to implement RestCallbackWorkItemHandler, which waits for a REST call-back? How to register a callback endpoint?

Actually there is not that much that you need to do - jBPM comes with REST work item handler that is capable of sending REST requests and transform both inputs and outputs. Although it does send the request synchronously all the times and waits for the response. So this could be your first place to look - or if you prefer to use other approach (as it uses http client - so you could prefer to use jaxrs client api) then to build your own handler is not a big issue anyway. There you could control if the execution should be sync or async or shall it wait for external signal or proceed directly.

So the external signal thing is very important - you can send the request and then do not complete work item handler - so the process engine will wait in that node waiting for “reply” - the reply is given by completing the work item from external ssytem by REST call. 

With that you can send a callback URL as part of the REST call and then it will simply send back response (with optional data as map)

That's great, of course there already is a REST endpoint to notify completion I don't need my own.

Is it possible to POST a data as single json when completing the task ?

yes, when you complete work item you can provide a map of data to be put into the task as outputs

What about a cancel, I would like to send a cancel signal to the process instance and the currently active task should catch the event and invoke cancel on a remote service.

Can I get a KieSession in the work item handler and register an event listener ?

you can use boundary signal event on the task itself and thus have alternative flow to call another service to compensate

And a time-out, which needs to invoke remote cancel, I probably have to start my own timer in the work item handler?

similar here, use timer boundary event to cancel the activity and invoke compensation flow

 

Another thing I would like to get from the JBPM server is a list of all running process instances with an option to filter the instances based on variables or Colleralation Keys. This way I don’t have to keep the references in my application and can query the JBPM server when a user want to see the list of running processes.

correlation keys are supported out of the box, so you can assign unique correlationkey when you start a process instance.

Cool. Are this searches supported using Java client or direct REST calls only?

both

ps.: I can't find the REST API doc for 7.8.0

REST api is document on running KIE Server - http://localhost:8080/kie-server/docs

Maciej

 

What is the best way to put things together for an OpenShift deployment ?

I thinking to start from the container image you provide and if required extend it with additional components (eg. RestWithCallbackWorkItemHandler, getProcessesRESTEndpoint).

that sounds like a best approach - we are in progress of preparing OpenShift images.

Feel free to reach out in case you would like to do the work item implementation, either here or on irc

Thanks,

Matej.

 

Maciej

Thanks,

Matej.

--
You received this message because you are subscribed to the Google Groups "jBPM Usage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jbpm-usage+...@googlegroups.com.
To post to this group, send email to jbpm-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-usage/2641d83a-271a-4991-b2c0-9b2abc65e9e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jBPM Usage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jbpm-usage+...@googlegroups.com.
To post to this group, send email to jbpm-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-usage/e8b3002c-43fa-4479-8313-fc5af491f55d%40googlegroups.com.

[Matej Lazar]

On Wednesday, August 1, 2018 at 9:11:19 PM UTC+2, Maciej Swiderski wrote:

On 1 Aug 2018, at 11:56, Matej Lazar <matej...@gmail.com> wrote:

On Tuesday, July 31, 2018 at 9:24:22 PM UTC+2, Maciej Swiderski wrote:

Matej,

On 31 Jul 2018, at 18:49, Matej Lazar <matej...@gmail.com> wrote:

I’m looking for using JBPM as a standalone Microservice orchestrator, deployed on OpenShift.

The process definition should be simple, it should contain only an ItemHandler per service.

ItemHandler will be used to call remote service via REST. Some service operations are long-running, which would make sense to have an ItemHandler that waits for a REST callback to get the completion notification and the result.

“RestCallbackWorkItemHandler” have to support cancel, time-out and in/out variable mapping.

What is the best way to implement RestCallbackWorkItemHandler, which waits for a REST call-back? How to register a callback endpoint?

Actually there is not that much that you need to do - jBPM comes with REST work item handler that is capable of sending REST requests and transform both inputs and outputs. Although it does send the request synchronously all the times and waits for the response. So this could be your first place to look - or if you prefer to use other approach (as it uses http client - so you could prefer to use jaxrs client api) then to build your own handler is not a big issue anyway. There you could control if the execution should be sync or async or shall it wait for external signal or proceed directly.

So the external signal thing is very important - you can send the request and then do not complete work item handler - so the process engine will wait in that node waiting for “reply” - the reply is given by completing the work item from external ssytem by REST call. 

With that you can send a callback URL as part of the REST call and then it will simply send back response (with optional data as map)

That's great, of course there already is a REST endpoint to notify completion I don't need my own.

Is it possible to POST a data as single json when completing the task ?

yes, when you complete work item you can provide a map of data to be put into the task as outputs

What about a cancel, I would like to send a cancel signal to the process instance and the currently active task should catch the event and invoke cancel on a remote service.

Can I get a KieSession in the work item handler and register an event listener ?

you can use boundary signal event on the task itself and thus have alternative flow to call another service to compensate

And a time-out, which needs to invoke remote cancel, I probably have to start my own timer in the work item handler?

similar here, use timer boundary event to cancel the activity and invoke compensation flow

Defining cancel and time-out on the process level it makes a process definition too verbose. Since cancel and time-out are common to all the tasks I would like to hide it in the work item handler.

Currently we use process level routing and the idea is it to simplify it. I'm thinking of more declarative approach based on task variables, where you can specify which tasks needs to run also in case of cancel/timeout/failed.

The implementation I have in mind is that the process engine will call all the tasks every time and there will be a "gate" in the work item handler which will decide to execute the task business operation or skip it.

To do that I think I need to subscribe to signal events.

I also need access to the results of previous tasks, to pass them to the next one.

Our use case: the build process

1. pre build operation

2. prepare reposiotry

3. prepare environment

4. run the build

5. collect the results

6. destroy env

7. close repository

8. store the results

Execution:

8 - run always

7, 8 - run if 2 completed

6, 7, 8 - run if 3 completed

5, 6, 7, 8 - run if 4 started

Variables:

8 - needs most of the results

7 - needs a result of 2

3 - needs a result of 2

[Matej Lazar]

I managed to do the variable passing flow. Now I'm on cancel and time-out.

For cancel I need to catch a signal event in the workitem handler. 

Is it possible to do this without a boundary signal event in the process definition?

[Maciej Swiderski]

Have a look at this jira JBPM-7679 and linked article 

Maciej

--
You received this message because you are subscribed to the Google Groups "jBPM Usage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jbpm-usage+...@googlegroups.com.
To post to this group, send email to jbpm-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-usage/a8c44a8a-0ee8-42ba-b830-65ca52ce2960%40googlegroups.com.

[Matej Lazar]

Thanks for the links.

How can I get a list of currently active service tasks in the process?

I'm trying to use taskService.getTasksByStatusByProcessInstanceId but it does not return any.

Matej.

[Maciej Swiderski]

Service tasks are considered sync by default so only way to get them is to retrieve them as NodeInstances and filter by node type

Maciej

To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-usage/0b999e47-eac7-4586-b280-881ee7fddf79%40googlegroups.com.