Drools workbench Stability concerns and production deployment infrastructure for an enterprise

[Sankaran PV]

Hi,

We are evaluating Drools workbench 6.2 as a rule management application for our business users.   As part of our business requirements, we need the ability to allow business analysts to author and manage their own rules tables that can then be integrated into our larger systems.   Drools Workbench and Engine appears to meet the needs and expectations of our business and we would like to offer this as a proposed solution in our current architecture.   We have encountered a few issues that we are hoping we can resolve so that our recommendation is complete.   Unfortunately, we have been unable to determine root cause and may need some assistance.    We are hoping that you can guide us to the cause and solution to the issues we are encountering.   Any help would be greatly appreciated as we want Drools to be a viable recommendation and would become a critical system in our overall enterprise architecture. 

Our primary usage scenario for our investigation involves converting existing rules tables from Oracle 11g and migrating them to Drools.   These are mainly decision tables, so there isn’t anything unusual about them.   As part of this migration, we convert the existing rules into spreadsheets, add Drools metadata and upload into the workbench.   After the spreadsheet has been uploaded, we convert these to guided decision tables so that we can manage, package and deploy.    Our goal is to leverage Drools Execution Server for this.    These would then be available to our systems via the REST endpoints exposed. 

As for the environment, we installed the workbench into Tomcat 7 running in a cloud machine. (Openstack).  Through the course of the evaluation we have encountered what appear to be stability issues with the interactions between Drools Workbench and the repository.   At this time, we are unsure whether or not this is an issues with Drools, or our environment and are hoping that you would be able to guide us so that we can determine a course of action.   If we can resolve these issues, there is a great likely hood of our recommendation being adopted by our technical and business partners. 

I’ve attempted to summarize the issues we are seeing below and am grateful for any assistance you might be able to offer in helping us further investigate.   If there are any specific technical details you might need, we would be able to offer those to help. 

• 1.       Projects and assets get disappeared from the project explorer and sometimes we get a message as 

Repository Structure was not initialized for current repository.

GroupId for the parent pom.xml. 

ArtifactId for the parent pom.xml. 

Version number for the parent pom.xml. 

We had to delete the .niogit directory and restart the application. We will lose all our projects since we are recreating the repository. Similar to the issue reported in here

• 2.       The second issue, is related to application logging. We have configured DEBUG level in logging, but whenever we get some exception, the log messages are too technical (Null pointer exception, Index out of bounds exception,  RPC endpoint threw exception etc..). We are unable to find out what exactly the issue which resulted in these exceptions. Majority of these issues happened while manipulating with Decision table spreadsheets and converting them to Guided Decision tables and deploying them. I am sure we would have made some errors in the spreadsheet, but instead of getting a pointer to what is wrong in the spreadsheet, we get these exceptions in logs. 

As stated before, we are unsure if our configuration is causing the issue, our process or if this is a bug.   Would you be able to help us make a determination? 

Specifically, we are looking at the preferred production deployment architecture

• 1.       What are our options in terms of GIT repository? We are bit concerned with the repository being local to the workbench server and if anything happens to repository (issue #1), we would lose all the work. So how do we make sure we are robust in repository management? Pushing the changes from local GIT to remote GIT periodically is the solution? Or anything else?
  
• 2.       What is the preferred way of clustering? Do we need to have a separate Zookeeper infrastructure?
  
• 3.       With respect to the execution server, do we run the execution server on the same App server as of workbench?
  

 If you have standard enterprise level production deployment architecture of the workbench suite, please share it and it will be helpful.

 We would like to know if you seen similar issues from other users? If yes what are the things in roadmap to fix these issues

 Sorry for a lengthy email, but we appreciate if you could help us getting these things right.

 Thanks & Regards

Sankaran Bhattathiripad

 

[Michael Anstis]

I'll answer in part:

1. Projects and assets get disappeared from the project explorer and sometimes we get a message

This is indeed peculiar. I don't suppose there's (a) anything untoward in the server logs?, (b) anything untoward in the browsers logs, (c) you're able to give steps to reproduce (other than "now and again it does XYZ.. ")? 

2. [snip] We are unable to find out what exactly the issue which resulted in these exceptions. Majority of these issues happened while manipulating with Decision table spreadsheets and converting them to Guided Decision tables and deploying them. I am sure we would have made some errors in the spreadsheet, but instead of getting a pointer to what is wrong in the spreadsheet, we get these exceptions in logs

These are no doubt runtime errors manifesting one or more bugs in (probably) the XLS->Guided Decision Table conversion.

You're correct that a User should not be presented with these types of error. Can you raise GUVNOR JIRA's with the necessary files etc to reproduce the issues?

Thanks,

Mike

--
You received this message because you are subscribed to the Google Groups "Drools Setup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drools-setup...@googlegroups.com.
To post to this group, send email to drools...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/drools-setup/b74b79e2-e3d3-44a6-9c37-22aff974891a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sankaran PV]

Thank you for the reply. Please find some more comments inline.

I appreciate your help.

On Friday, 22 May 2015 13:22:43 UTC-5, Michael Anstis wrote:

I'll answer in part:

1. Projects and assets get disappeared from the project explorer and sometimes we get a message

This is indeed peculiar. I don't suppose there's (a) anything untoward in the server logs?, (b) anything untoward in the browsers logs, (c) you're able to give steps to reproduce (other than "now and again it does XYZ.. ")? 

[Sankaran] Unfortunately we even could not reproduce it . This happened around 3 times during our evaluation period of 2 weeks. What I remember at one instance was when we try to delete the Data Model ( we made some mistakes and wanted to correct them) from the repository view, and the same data model is referenced in the Guided Decision table, thats when it came up. It did not come up immediately, but it happened when we logged out and logged back in to the application. There was no error in the logs. I did not notice any in Browser logs either.

2. [snip] We are unable to find out what exactly the issue which resulted in these exceptions. Majority of these issues happened while manipulating with Decision table spreadsheets and converting them to Guided Decision tables and deploying them. I am sure we would have made some errors in the spreadsheet, but instead of getting a pointer to what is wrong in the spreadsheet, we get these exceptions in logs

These are no doubt run-time errors manifesting one or more bugs in (probably) the XLS->Guided Decision Table conversion.

You're correct that a User should not be presented with these types of error. Can you raise GUVNOR JIRA's with the necessary files etc to reproduce the issues?

[Sankaran] Attached a spreadsheet, which gave some issues when we do the following

We imported the spreadsheet, I could see the drl source in the editor, but when tried to convert to guided decision table, I received an error saying Index Out of Bounds. I am sure it will be some issue with the spreadsheet, but what I am concerned more is that the error messages are hard to interpret. 

Please let me know your comments on the second part of my question too( Production deployment infra )

[Michael Anstis]

Hi,

I've had a quick look at your XLS file.

The Third Party library we use to parse XLS file (Apache's POI) is detecting 777 rows in the file (whereas on inspection there appear to be 10). I deleted rows 11 to 777 and reimported and it worked OK.

That said we need to better handle this scenario: perhaps a row consisting of completely empty cells (value.trim().isEmpty()==true) would be a reasonable check (however I bet they'll be people with XLS with intentionally empty rows!)

I'll look into more of a fix next week.

With kind regards,

Mike

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-setup/8441782a-5b39-4009-94b0-df54c5ebb498%40googlegroups.com.

[Sankaran PV]

Thanks a lot. That really helps.

[Michael Anstis]

You're welcome.. just looking a little more the CONDITION columns are recognized as being "empty" and no values are extracted; whereas the ACTION columns contain empty Strings on row 11 onwards.

Still.. this should be handled by our conversion routines. I'll do a fix next week.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-setup/71a2aaa4-1b34-4bb6-9f51-1f0b58dfef08%40googlegroups.com.

[Michael Anstis]

See https://issues.jboss.org/browse/GUVNOR-2188

Just waiting for the fix to be merged (and then it'll be available in the next 6.3.0-SNAPSHOT).

[Sankaran PV]

Thank you so much Michael. I would appreciate if you could send me a a preferred deployment architecture of the Workbench/execution server suite.

You received this message because you are subscribed to a topic in the Google Groups "Drools Setup" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drools-setup/INFINdBKWrE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drools-setup...@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-setup/CAAG9P0tPkcV6k69Sziy%3DeO9%3D6Hb5kcCzhCiWkbNruKGN9e65ZQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Regards,
Sankaran

[Edson Tirelli]

What exactly are you looking for?

Edson

> https://groups.google.com/d/msgid/drools-setup/CA%2B%2BAEVJunP0Y%2BaF2CZ4TGuUBDSuOY%2Bn8mF3FAGdu95bdFTA%2BZQ%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.



--

Edson Tirelli
Principal Software Engineer
Red Hat Business Systems and Intelligence Group

[Sankaran PV]

Hi Edson,

Specifically, we are looking for the preferred production deployment architecture

• 1.       What are our options in terms of GIT repository? We are bit concerned with the repository being local to the workbench server and if anything happens to repository (issue #1), we would lose all the work. So how do we make sure we are robust in repository management? Pushing the changes from local GIT to remote GIT periodically is the solution? Or anything else?
  
• 2.       What is the preferred way of clustering? Do we need to have a separate Zookeeper infrastructure?
  
• 3.       With respect to the execution server, do we run the execution server on the same App server as of workbench?
  

 If you have standard enterprise level production deployment architecture of the workbench suite, please share it and it will be helpful.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-setup/CAD7AJncw4CtZovgg0VtMe4viLT2z23Rh6zP1u5zRSYxq2uXVuA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Regards,
Sankaran

[Edson Tirelli]

My .02c:

1. GIT, by itself, has several different options for high
availability, including mirroring. Besides that, Porcelli can confirm,
but I believe the workbench abstracts the GIT filesystem through the
VFS, so if you configure a workbench cluster, it will also cluster the
git repository.

2. I will let Porcelli answer.

3. The rules execution server is designed to be lightweight, with the
goal of easily running it on VMs/containers. While you can run the
execution server on the same app server as the workbench, my
suggestion would be to run it remotely. Anything that fits your
infrastructure, including just a separate app server, or a separate
docker image, or a full blown VM.

Edson

> https://groups.google.com/d/msgid/drools-setup/CA%2B%2BAEVLxMy8DPcAhDAqw7Jf4f5prphwALQjBcdVxDtCjF_tFQA%40mail.gmail.com.

[Mark Proctor]

> On 28 May 2015, at 01:34, Edson Tirelli <ed.ti...@gmail.com> wrote:
>
> My .02c:
>
> 1. GIT, by itself, has several different options for high
> availability, including mirroring. Besides that, Porcelli can confirm,
> but I believe the workbench abstracts the GIT filesystem through the
> VFS, so if you configure a workbench cluster, it will also cluster the
> git repository.
>
> 2. I will let Porcelli answer.
>

Here is the HA setup and installation docs:
http://docs.jboss.org/drools/release/6.2.0.Final/drools-docs/html_single/#wb.WorkbenchHighAvailability

> To view this discussion on the web visit https://groups.google.com/d/msgid/drools-setup/CAD7AJnfrdBPgjMh_yqSPXg1Mf9wgv_yEqfZN1Tjtt68eiZQYAQ%40mail.gmail.com.

[David Holtzhouser]

Hi Edson,

To your point #3, I have tried several times to install the following docker container on AWS:
https://hub.docker.com/r/jboss/kie-server-showcase/

The container seems to be running, but when I try and login with the users noted in the, Users and Roles, section of the dockerhub link above, I get a 'forbidden'.

Users and roles

This image provides a default user kie-server using password kie-server and with the role kie-server.

Thanks,

Dave

[Roger Martinez]

Hi, these docker images are not intended to be used on cloud environmennts such as AWS o Openshift.

These inages do no met the requirements for those environments, as each one can have different requirements/constraints. You can use the drools-workbench image as base for your custom one, and then apply your concrete environment constraints.

[Roger Martinez]

Or in your case, use the kie-server as the base for your custom inage for aws

[Anunay Sinha]

We followed the steps as documented here. But the lack the brevity on how can I get zookeeper and helix started. We looked into their respective documentation but could not get much success.

Is there a particular way to eun zookeeper ans helix.