Drools can't find kjars referenced by the projects POM file

[José Filipe Neis]

Note: I added this same question in StackOverflow last week but got no answer/comment for a few days, so I'm trying here.

Drools 6.5 / Spring Boot 1.3.3

I'm having a hard time trying to run a Spring Boot application with Drools projects dependencies. The project is a Spring Boot application and references 2 different Drools projects. Everything runs fine on Eclipse, but when I try to run using java -jar, Drools can't find 1 of the 2 related projects.

I will describe the steps the application is doing and my guesses about the problem source to make it easier to follow.

Step 1: during Spring CDI, I call KieServices to retrieve 1 of the defined KieBases:

KieContainer kcontainer = KieServices.Factory.get().getKieClasspathContainer();
return kcontainer.getKieBase("kbase-common-rules");

The result of this classpath search is shown in the logs:

ClasspathKieProject:131 : Found kmodule: jar:file:/home/credisfera-services/target/credisfera-services-3.3.0-SNAPSHOT.jar!/lib/credisfera-cep-1.0.1.jar!/META-INF/kmodule.xml
KieRepositoryImpl:113 : KieModule was added: ZipKieModule[releaseId=br.com.credisfera:credisfera-services:3.3.0-SNAPSHOT,file=/home/credisfera-services/target/credisfera-services-3.3.0-SNAPSHOT.jar]
ClasspathKieProject:131 : Found kmodule: jar:file:/home/credisfera-services/target/credisfera-services-3.3.0-SNAPSHOT.jar!/lib/common-rules-2.9.0.jar!/META-INF/kmodule.xml
KieRepositoryImpl:113 : KieModule was added: ZipKieModule[releaseId=br.com.credisfera:credisfera-services:3.3.0-SNAPSHOT,file=/home/credisfera-services/target/credisfera-services-3.3.0-SNAPSHOT.jar]

Kie is able to create this KieBase correctly, although I believe something is strange in this log. Note that Kie "finds" the modules inside dependencies of the project (credisfera-cep-1.0.1 and common-rules-2.9.0), but then uses the project own JAR (credisfera-services-3.3.0) as the Kie Module (ZipKieModule messages). Its different than when I run using Eclipse, as inside Eclipse the individual dependencies JARs are added, never the project own JAR.

Step 2: the application tries to create a KieBase from the first "found" JAR:

KieContainer kcontainer = KieServices.Factory.get().getKieClasspathContainer();
return kcontainer.getKieBase("kbase-cep");

This time, Kie logs show nothing (as the Factory is a Singleton), but the application fails:

Factory method 'ksessionCep' threw exception; nested exception is java.lang.RuntimeException: The requested KieBase "kbase-cep" does not exist

As you can imagine, ksessionCep exists. So I try a different approach:

   final String groupId = "br.com.credisfera";
   final String artifactId = "credisfera-cep";
   final String version = "1.0.1";
   final String kieBaseName = "kbase-cep";

   final KieServices kieServices = KieServices.Factory.get();
   final ReleaseId releaseId = kieServices.newReleaseId(groupId, artifactId, version);
   final KieContainer kieContainer = kieServices.newKieContainer(releaseId);
   final KieBase kieBase = (kieBaseName != null && !kieBaseName.isEmpty()) ? kieContainer.getKieBase(kieBaseName)
                   : kieContainer.getKieBase();

   return kieContainer.newKieSession("ksession-cep");

This time, everything works fine. I conclude: the project is OK to Kie (at least for Maven scanner), but it can't find it using the classpath scanner.

I have 2 guesses right now:

1. It's a Kie bug. Kie shouldn't be inserting the Spring Boot JAR twice as a KieModule, and as it is it's not able to create more than one KieContainer internally;
   
2. Maybe there's any classpath configuration that would allow Kie to add the referenced projects themselves as KieModules instead of the parent JAR?
   

I would prefer to not use explicit versions in the source code and rely completely on Maven to keep managing dependencies, so I'm not considering my second approach as a solution so far.

[Mario Fusco]

Hi,

can you please send a sample project reproducing the problem? In this way it will be easier for us to further investigate it.

Thanks,

Mario

--
You received this message because you are subscribed to the Google Groups "Drools Usage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drools-usage+unsubscribe@googlegroups.com.
To post to this group, send email to drools...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/853cded7-8de0-4fe6-bc8f-94e73b905f65%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[José Filipe Neis]

Sure Mario. I'll take some hours and post a reproducible example until the end of the week.

I'm using Maven scanner as an workaround by now.

To unsubscribe from this group and stop receiving emails from it, send an email to drools-usage...@googlegroups.com.

[KimJohn Quinn]

Jose,

I seem to recall having issues too with SpringBoot and Kjars originally.  Have your tried:

1. Playing around with the "exploded" option in the spring-plugin for the DRL files (if you do it that way)?
2. Changing the format from the Spring Jar to the normal Jar/Zip format?

KJQ

[José Filipe Neis]

Hi Kim!

I don't know if I correctly understood both of your suggestions, so let me explain what I have tried based on my understanding.

Suggestion 1: my DRL files are in another JAR and I don't consider moving them my Spring Boot project (that's what you mean by "exploded"?). Both projects are under Workbench Maven repository, moving them inside Spring project would avoid simultaneous edit using Eclipse or Workbench. Besides that, and I'm using KieServices directly instead of <kie:import /> Spring integration. I was using <kie:import />, but it has the very same behavior that I described when calling KieServices.Factory.get().getKieClasspathContainer() for the first time, except it runs earlier in application startup (but the same problema that I described occurs).

Suggestion 2: the host application (Spring Boot) is packed as a runnable JAR and both Drools projects are also packed as JARs (KJARs, actually, as they have the kmodule.xml files). The Spring Boot host application runs as a standalone application using embedded Tomcat and we use a lot of Boot configuration options, so I don't see how to change its packaging format. Am I missing anything here?

Thank you for your help!

Filipe

[KimJohn Quinn]

Jose,

I had some problems when we first integrated Drools into SpringBoot and it had to do, at the time, with the way SpringBoot handled the classloading.  We actually opted of of using the Spring/Drools integration and manually created our own autoconfiguration (similar I bet to you using kservices directly) to load up the Drools engine.  

In the end, we progressed down three paths to our final approach we use now:

1. SpringBoot application/jar --> Rules under resources and directly compiled/loaded at startup.
2. SpringBoot application/jar --> Rules in separate module (kjar) and used as a dependency inside the app jar.
3. SpringBoot application/jar --> Rules in separate module (kjar) and using Maven discovery to pull it in.

Let me know if one of these scenarios is what you are doing because I will dig into our Git and try and find out what we did, scenarios #1 and #2 ran into problems with SpringBoot and the classloading.

What I seem to recall, and this is a while back, is that the Spring executable Jar, defaulted by the plugin, is a somewhat Spring-specific configuration different from the normal Jar/Zip format.  

This is controlled by the <layout/> tag something like this:

<plugin>
    <groupId>

org.springframework.boot

</groupId>
    <artifactId>

spring-boot-maven-plugin

</artifactId>
    <configuration>
    <layout>

ZIP

</layout>
    </configuration>
        <executions>
            <execution>
                <phase>

package

</phase>
            <goals>
                <goal>

repackage

</goal>
            </goals>
          </execution>
       </executions>
</plugin>

I "think" at one point we also used the "<requiresUnpack>" of the Spring boot plugin originally as well.

We were eventually able to get everything running smoothly and I know now we do not use either of those approaches anymore.

KJQ

--
You received this message because you are subscribed to a topic in the Google Groups "Drools Usage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drools-usage/odQjmBess5A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drools-usage+unsubscribe@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/dcb6a7df-a637-4873-bc0f-72e32a7617ea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

KimJohn Quinn
k...@logicdrop.com

Logicdrop
22620 Woodward Avenue - Suite D 
Ferndale, MI 48220
O 888.229.2817
M 248.882.0728
www.logicdrop.com

[José Filipe Neis]

Kim,

Going through each of the paths you listed:

1. SpringBoot application/jar --> Rules under resources and directly compiled/loaded at startup.

I didn't try this option but I believe it would work. What I don't like of this approach is a) we wouldn't be able to edit the project using Eclipse OR Workbench and b) couples the rules completely with the host application.

2. SpringBoot application/jar --> Rules in separate module (kjar) and used as a dependency inside the app jar.

This is the option that I tried and originated this thread. It runs fine in Eclipse (as Eclipse resolves the classpath in a different way) but crashes when running Spring Boot with java -jar. Yes, I agree with you, it's related with classpath loading. My guess is that there's a way to KieServices resolve it, but maybe it's not implemented yet. :)

3. SpringBoot application/jar --> Rules in separate module (kjar) and using Maven discovery to pull it in.

That's how I was able to make things work. The bad part of it is that we need to specify the kjars version in the POM (as we use types defined inside the kjars projects as Facts) and in the method that loads the KieBase/KieContainer. Besides that, we need to install new version of kjars in every server Maven repository so KieServices Maven loader is able to find it.

Can you tell how are you dealing with this today? That you commented that you have been through all this a while ago, but I didn't understand if you are still using Spring Boot/Drools and solved it or if you have no more this scenario.

Again, thank you for your help and thorough answers!

To unsubscribe from this group and all its topics, send an email to drools-usage...@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/dcb6a7df-a637-4873-bc0f-72e32a7617ea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

KimJohn Quinn
k...@logicdrop.com

Logicdrop
22620 Woodward Avenue - Suite D 
Ferndale, MI 48220
O 888.229.2817
M 248.882.0728
www.logicdrop.com

--

You received this message because you are subscribed to a topic in the Google Groups "Drools Usage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drools-usage/odQjmBess5A/unsubscribe.

To unsubscribe from this group and all its topics, send an email to drools-usage...@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/CAF1TLv4zJJHwPqegcAXd493gZpMnvcec625_BKEOc3d4uEecDQ%40mail.gmail.com.

[KimJohn Quinn]

Jose,

Yes we still are using SpringBoot coupled with Drools.  Let me dig into our Git and see if I can pull some code samples down for you - may not be exact but might give you some hints.  Either way, we have done some pretty extensive work with integrating Drools and SpringBoot not using the Spring API but instead creating our own auto configurations

I'll try to get it over today but at the latest tomorrow...

Thanks.

KJQ

To unsubscribe from this group and all its topics, send an email to drools-usage+unsubscribe@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/dcb6a7df-a637-4873-bc0f-72e32a7617ea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

KimJohn Quinn
k...@logicdrop.com

Logicdrop
22620 Woodward Avenue - Suite D 
Ferndale, MI 48220
O 888.229.2817
M 248.882.0728
www.logicdrop.com

--
You received this message because you are subscribed to a topic in the Google Groups "Drools Usage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drools-usage/odQjmBess5A/unsubscribe.

To unsubscribe from this group and all its topics, send an email to drools-usage+unsubscribe@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/CAF1TLv4zJJHwPqegcAXd493gZpMnvcec625_BKEOc3d4uEecDQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Drools Usage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drools-usage/odQjmBess5A/unsubscribe.

To unsubscribe from this group and all its topics, send an email to drools-usage+unsubscribe@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/CAGaRV4RfjDSOQQ0jmMVBo9UaOpNEtgEpZypboq2%3DkAWcjpXmLQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[KimJohn Quinn]

Jose,

I needed to upgrade to 7.1.0.Final and embed my kjar directly into my SpringBoot application anyhow (for some testing) and I was able to get it to run executing the Jar.

In my Drools Spring @Configuration I changed our Maven lookup

from:

final KieContainer kContainer = this.kServices.newKieContainer(containerKey, releaseId); 

to:

 final KieContainer kContainer = this.kServices.newKieClasspathContainer();

And my resulting output  (munged a little to take out any client stuff) from running the packaged SpringBoot app using either mvn spring-boot:run or java -jar from the CLI is:

[INFO ] org.drools.compiler.kie.builder.impl.ClasspathKieProject - Found kmodule: jar:file:/project/server/bin/my-server-2.3.0.0-SNAPSHOT.jar!/BOOT-INF/lib/my-rules-2.3.0.0-SNAPSHOT.jar!/META-INF/kmodule.xml
[INFO ] org.drools.compiler.kie.builder.impl.KieRepositoryImpl - KieModule was added: ZipKieModule[releaseId=com.my-server:2.3.0.0-SNAPSHOT,file=/project/server/bin/my-server-2.3.0.0-SNAPSHOT.jar]
[DEBUG] drools.services.DroolsContainerFactory - Created container rules (com.my-server:my-rules:2.3.0.0-SNAPSHOT)

I proceeded to test the rules and they executed fine for that created container...

 

 

[José Filipe Neis]

Hey Kim!

I could swear I answered your post a week ago, but looks like I haven't! :)

So I tried upgrading to 7.1 and trying some scenarios. What I found so far:

1. When 6.5 can't find a KieBase/KieSession, it throws an exception, while 7.1 simply returns null. Irrelevant, but I went from a KieServices exception to a NullPointerException;
2. If I have only 1 KJar, KieServices works as expected (as in your case!). In the moment that I add a second KJar dependency, KieServices can only solve references to the last scanned KJar - so it reinforces my suspicion of a bug when there is > 1 KJar;

Despite that (I'll create a sample application and post), looks like something changed (or a bug was reintroduced) involving custom accumulate functions, as the same function that works perfectly with Drools 6.5 can't be found by Drools 7.1. But that's another matter and I will treat that in another post.

If you have any other tips, please feel free to continue the conversation, but I believe the next step is creating a sample project.

Thank you!

Filipe

[José Filipe Neis]

I created a sample setup that reproduces the problem using last Spring Boot version (1.5.5) and Drools/KieServices 6.5. The source code is attached (it's a ZIP file, changed the extension due to Groups policy).

It's a simple setup using Spring/Drools default projects. Just changed kmodule(s).xml to reflect our setup (as maybe they can help to understand the problem). Steps to reproduce the problem:

1. Download/unzip the project
2. cd drools-multiple-kieproject
3. mvn clean package
4. cd host-project/target
5. java -jar demo-0.0.1-SNAPSHOT.jar
6. Console will log: [main] ERROR Unknown KieSession name: ksession-project1
   

Now, situations where this application will run fine:

1. If you run it inside Eclipse, as the classpath will be resolved in a different way and the Kmodule will be resolved from the project directory, not the BOOT-INF/lib/[project].jar
2. If you run the same piece of code that is in HostProjectApplication class inside a test, because, again, the classpath will resolve the project directory and things go just fine.

So far I'm really convinced that it has something to do with classpath/SpringBoot über jar, but also confident that there's somehow related to KieServices classes as, when dealing with just 1 KJar dependency, everything works fine.

I wish I could tag Mario here, but hope he (or someone with deep KieServices understanding) will take a look on this the next couple of days.

Tks!

[KimJohn Quinn]

Jose,

As an interim solution this will work with the SpringBoot executable jar.  I tried it using your example and both sessions are recognized:

<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>

                    <requiresUnpack>
                    <dependency>
<groupId>com.sample</groupId>
<artifactId>kproject1</artifactId>
</dependency>

       <dependency>
           <groupId>com.sample</groupId>
           <artifactId>kproject2</artifactId>
       </dependency>
                    </requiresUnpack>

                </configuration>
<executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
</plugin>

--

You received this message because you are subscribed to a topic in the Google Groups "Drools Usage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drools-usage/odQjmBess5A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to drools-usage+unsubscribe@googlegroups.com.
To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/a3116aa6-4914-4772-be79-6295659201a1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[José Filipe Neis]

Kim, works like a charm. Thank you.

I keep thinking it's a bug in KieServices as it should be able to handle Spring über jar without unpacking libs, but with this workaround the priority is really low.

Anyway: even unpacking we are still not able to use Drools/Spring integration with @KBase and @KSession annotations. I'll create another thread as it's a different topic.

Thanks again for all your help!

To unsubscribe from this group and all its topics, send an email to drools-usage...@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/a3116aa6-4914-4772-be79-6295659201a1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

KimJohn Quinn
k...@logicdrop.com

Logicdrop
22620 Woodward Avenue - Suite D 
Ferndale, MI 48220
O 888.229.2817
M 248.882.0728
www.logicdrop.com

--

You received this message because you are subscribed to a topic in the Google Groups "Drools Usage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/drools-usage/odQjmBess5A/unsubscribe.

To unsubscribe from this group and all its topics, send an email to drools-usage...@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/CAF1TLv6Hm_4eAWSOzeKZuHTA7EvShf0kfESVVxCSVk8-T8qRAA%40mail.gmail.com.

[Manolo Bayona]

Thank you so much, you had saved my ass.

To unsubscribe from this group and all its topics, send an email to drools-usage...@googlegroups.com.

To post to this group, send email to drools...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/drools-usage/a3116aa6-4914-4772-be79-6295659201a1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.