High Availability: are Helix and ZooKeeper Necessary?

[Lorenzo Ciaravola]

Hi,

I am using an Oracle database, and I have set up my two jBPM Workbench instances to use a shared NFS for the asset and maven repositories:

<property name='org.uberfire.nio.git.dir' value='/data'/>

<property name='org.guvnor.m2repo.dir' value='/data'/>

From what I can tell, everything is working fine. Thus, I don't understand why/if I really need to set up all the jBPM clustering using Helix and ZooKeeper, which seems a bit complicated to me. 

Could you please clarify this for me? Can I avoid setting those up?

Thank you

[Alexandre Porcelli]

You need if you want HA cluster for your authoring (git repositories).
Regards,
___________________
Alexandre Porcelli
F: +1 (978) 844-8695
http://porcelli.me

> --
> You received this message because you are subscribed to the Google Groups
> "jBPM Setup" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jbpm-setup+...@googlegroups.com.
> To post to this group, send email to jbpm-...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jbpm-setup/631d756f-e092-49b0-92ce-4a74d2003477%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Lorenzo Ciaravola]

Thank you for the reply, but still I am not clear why that's needed.

By setting the property org.uberfire.nio.git.dir, I get the git repository created on the NFS volume, thus accessible by all workbench instances. So why do I still need those if I am already able to achieve what I need? Or am I missing something, some issue I may encounter later on that I haven't considered?

[Abhijit Humbe]

I personally dont suggest to use NFS to store .niogit directory. Creating git repo over NFS can cause performance issue. I have seen issue where creating rule/process takes around 5 mins if we use NFS to store git repo. 

Abhijit Humbe

To unsubscribe from this group and stop receiving emails from it, send an email to jbpm-setup+unsubscribe@googlegroups.com.

To post to this group, send email to jbpm-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-setup/eb2bb4ff-1f83-4512-944d-bb5d84a65563%40googlegroups.com.

[Alexandre Porcelli]

NFS isn't supported on 6.x and 7.x for multiple reasons. Actually, if
you're using NFS very likely to end up corrupting your repositories.

For authoring clustered HA you have no option other than use Helix+ZK.

Regards,
___________________
Alexandre Porcelli
F: +1 (978) 844-8695
http://porcelli.me

>> https://groups.google.com/d/msgid/jbpm-setup/eb2bb4ff-1f83-4512-944d-bb5d84a65563%40googlegroups.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "jBPM Setup" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jbpm-setup+...@googlegroups.com.
> To post to this group, send email to jbpm-...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jbpm-setup/CAA%3DHmHrAQoyR4V-63XbVDN1Hs4t%2Bq0CJaxfkCnEZrnY4c%3DwDZQ%40mail.gmail.com.

[Lorenzo Ciaravola]

Ok, now I got it. And it also makes sense the performance issues highlighted by Abhijit.

May I have a further question?

What if I plan to achieve HA only in production but not in development? And say I am not interested in authoring clustered HA? When it comes to deployment, can I just use NFS as remote maven repository location for the deployment assets? Therefore, avoid to use Helix+ZK in production since I don't have authoring in Production. Will this be fine or also in this case it is better to avoid NFS? 

[Alexandre Porcelli]

On Wed, Jul 26, 2017 at 9:33 PM, Lorenzo Ciaravola
<harle...@gmail.com> wrote:
> Ok, now I got it. And it also makes sense the performance issues highlighted
> by Abhijit.
>
> May I have a further question?
>
> What if I plan to achieve HA only in production but not in development? And
> say I am not interested in authoring clustered HA? When it comes to
> deployment, can I just use NFS as remote maven repository location for the
> deployment assets? Therefore, avoid to use Helix+ZK in production since I
> don't have authoring in Production. Will this be fine or also in this case
> it is better to avoid NFS?
>

Sure you can do it! Lots of people do... and you can also make sure
you won't lost a commit in your repos... by using the git hook
feature... and, on every commit, push the changes to somewhere else
too.

> https://groups.google.com/d/msgid/jbpm-setup/8224ee59-eebc-4030-b50b-a0d1fb9e7e90%40googlegroups.com.

[Abhijit Humbe]

Helix+ZK is used to cluster git repo not maven repo so you dont need it in production, as you will not be authoring rules/process there. Yes, you can use NFS to host maven repository, I dont see any issue in using NFS for maven repo

Abhijit Humbe

To unsubscribe from this group and stop receiving emails from it, send an email to jbpm-setup+unsubscribe@googlegroups.com.

To post to this group, send email to jbpm-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-setup/8224ee59-eebc-4030-b50b-a0d1fb9e7e90%40googlegroups.com.

[Lorenzo]

Thank you so much guys. It all starts making more sense now. But follow up questions keep coming in my mind, so hope you don't mind...

About the Maven repo:

1) What if I decide not to use NFS, but to use the local maven repo on the two workbench instances. Then, I make sure that when rolling out in prod, I deploy the same asset on both instances separately. Of course more manual work but easier set up for us. Would that be a valid option?

2) What other options I have instead of NFS for deployment assets? What do you guys usually do?

Lorenzo Ciaravola
+1-347-619-7631

[Abhijit Humbe]

Abhijit Humbe

On Thu, Jul 27, 2017 at 7:28 AM, Lorenzo <harle...@gmail.com> wrote:

Thank you so much guys. It all starts making more sense now. But follow up questions keep coming in my mind, so hope you don't mind...

About the Maven repo:

1) What if I decide not to use NFS, but to use the local maven repo on the two workbench instances. Then, I make sure that when rolling out in prod, I deploy the same asset on both instances separately. Of course more manual work but easier set up for us. Would that be a valid option?

You can go with this approach as well. Inplace of manually copying artifacts you can use 'rsync' command if you are using RHEL/linux os.

2) What other options I have instead of NFS for deployment assets? What do you guys usually do?

Host central maven repository and multiple servers can get artifacts from this repo 

[Lorenzo Ciaravola]

Hi Abhijit,

Hosting a central maven repo sounds cool. Would you mind to tell me more how to achieve this? I guess I need to prepare a settings.xml and put it somewhere for the Workbench to use the new remote repo? Where would that be?

Thank you so much

Abhijit Humbe

Lorenzo Ciaravola
+1-347-619-7631

Abhijit Humbe

[Stathis Rouvas]

Hi Lorenzo,

I would suggest using a ready-made Artifactory, try the one provided by bitnami for example or your favourite source.

The following sample settings.xml could be used as a starting point.
Modify as needed.

-Stathis

[Lorenzo Ciaravola]

Thank you, Stathis

But what I don't understand (for my lack of knowledge) is once I set up a maven repository in settings.xml, how do I instruct my KIE Workbench to read from that repo?

[Stathis Rouvas]

Hi,

BPMS.6 onwards will follow standard maven conventions, so configuring settings.xml will instruct BPMS to resolve artefacts against the configured repository.
Check documentation about KIEScanner for more details on that.

Have fun!
-Stathis

[Abhijit Humbe]

Yes you have to specify central maven repository location in settings.xml and pass this settings.xml while starting server instance using 'kie.maven.settings.custom' option

Abhijit Humbe

To unsubscribe from this group and stop receiving emails from it, send an email to jbpm-setup+unsubscribe@googlegroups.com.

To post to this group, send email to jbpm-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jbpm-setup/c2608c40-9982-49b0-af8f-c0e1b0c9b0f1%40googlegroups.com.

[Lorenzo Ciaravola]

Thank you guys, but I am afraid I am still doing something wrong because I cannot see any artifacts in the jBPM console.

1. I am starting the server with kie.maven.settings.custom pointing to C:/jbpm-6.4-H2/settings.xml

2. In settings.xml, I have set up a local repository for testing purpose (attached)

3. In my local maven repo I copied a kjar and its pom that I got from my .m2 folder

4. I start jBPM console but don't see anything in Artifact Repository or Process Deployments.

What am I doing wrong?

Abhijit Humbe

[Alexandre Porcelli]

You won't see anything... we're not a artifact respository manager...
what we display in workbench are artifacts that you build or manually
uploaded into it.
But, if you configured everything properly, Workbench should resolve
properly your dependencies... and, depending on your config, make your
workbench builds available too.

Regards,
___________________
Alexandre Porcelli
F: +1 (978) 844-8695
http://porcelli.me

On Fri, Jul 28, 2017 at 11:59 AM, Lorenzo Ciaravola

> https://groups.google.com/d/msgid/jbpm-setup/7e03bbaa-523c-4521-bb31-644048c61c83%40googlegroups.com.

[Lorenzo Ciaravola]

Oh, I see. I understand that now. But correct me if I am wrong:

Obviously then, I won't be able to start a business process instance from the Workbench (since the kjar I put in my Maven repo doesn't appear among the Process Deployments/Process Definitions). But perhaps if I try to start it using the Remote API, it should work?

[Stathis Rouvas]

You'll have to deploy it first to the BPMS.
You can use the REST API to do this.

[Lorenzo Ciaravola]

But I understood from Alexandre's post that builds will be available if I have configured everything properly, no need to manually deploy to workbench. In fact, I cannot do that because my two instances are behind a load balancer and I cannot access them individually to perform the deployment on each of them.