Large Memory Footprint for Rule Engine

[Abdu Hussein]

Hello everyone,

I am working on an webservice application that is using Drools rule engine (I am relatively new with Drools). We are experiencing a significant memory footprint with our application after creating the KieBase from our rules, and we are looking for some insights or suggestions.

Here are some details:

• Our application is a spring boot rest API that is using the Drools rule engine, deployed in a k8s cluster (where resources are critical). 
  
• Library version used 7.7.0.Final . (I know it's an old version but some part of the application is legacy code and trying to upgrade the lib version encountered some issues.)  
• We have 2 separate rule files, so we instantiate 2 rule engines (2 KieBase repositories). One rule file has around 25,000 rules, and the other has around 35,000 rules. Some rules have hundreds of conditions. The rule files sizes are 50 MB and 110 MB, respectively.
  
• Profiling the application shows that around 6 to 8 GB of RAM is used at startup, after creating the two KieBase containers (which is quite a lot for a typical webservice in our cluster and compared to rules files sizes). 
• This memory usage persists for the application’s lifetime, and is required even before processing any requests (firing rules from the engine). Handling requests requires additional memory, which looks to be reclaimed sometime after the request is processed, so that should be fine. However, we need to make sure processing concurrent requests is still possible and doesn't consume too much additional memory.

We are surprised by this large memory footprint and are questioning if there might be a memory leak in the application, although we haven’t identified any issues.

Has anyone encountered similar issues or have any suggestions on how to reduce the memory footprint or identify potential memory leaks?

Thank you in advance for your help!

Best regards, 
Abdu

[A Hussein]

Hi,

any hints or suggestions will be much appreciated!

[Richard Bourner]

KieBases can be large, depending on the number of rules. You must not recreate them each time though, you need to cache your KieBase and just create your KieSessions on the fly from it. 

The size of your sessions depends on your data, but since they should be discarded after each usage, it should not impact the memory footprint too much, unless you have many concurrent sessions.
So check the life-cycle of both your KieBases and KieSessions, it should help you identify what grows overtime.

[A Hussein]

Hi and Thanks for your reply.

Yes I am creating the KieBase once (I believe). This is code example used in class constructor:

private KieBase initialiseKieBase(RuleResourceInfo ruleResourceInfo) {
        log.info("Initialise KieBase");
        KieServices kieServices = KieServices.Factory.get();
        KieFileSystem kfs = kieServices.newKieFileSystem();

        KieModuleModel kieModuleModel = kieServices.newKieModuleModel();
        kieModuleModel.newKieBaseModel(ruleBaseName)
                .setDefault(true)
                .setEqualsBehavior(EqualityBehaviorOption.EQUALITY);
        kfs.writeKModuleXML(kieModuleModel.toXML());

        //add rules to kfs
        kfs.write(String.format("src/main/resources/%s.drl", ruleBaseName),
                kieServices.getResources().newFileSystemResource(ruleResourceInfo.getMainRulesFile().toFile()));

        if (ruleResourceInfo.getProceduralRulesFile() != null) {
            kfs.write(String.format("src/main/resources/%s.drl", ruleBaseName + "_procedural"),
                    kieServices.getResources()
                            .newFileSystemResource(ruleResourceInfo.getProceduralRulesFile().toFile()));
        }

        log.info("Compiling rule base {}", ruleBaseName);
        KieBuilder kieBuilder = kieServices.newKieBuilder(kfs).buildAll();
        Results results = kieBuilder.getResults();
        checkErrors(results);

        log.info("Creating Kie container...");
        KieContainer kc = kieServices.newKieContainer(kieServices.getRepository().getDefaultReleaseId());
        KieBaseConfiguration kieBaseConf = kieServices.newKieBaseConfiguration();
        KieBase kieBase = kc.newKieBase(kieBaseConf);
        log.info("Initialise KieBase.. Done");
        return kieBase;
    }

And KieSessions are created once per API request and disposed at the end of each request:

public void processRequest(Input) {
   KieSession kieSession = kbase.newKieSession();
   try {
      log.info("Processing facts");
      // fire rules;
   } catch (Exception e) {
     // handle ex
   } finally {
      if (kieSession != null) {
        kieSession.dispose();
      }
   }
}

 

So memory shooting happens just while application starting (we can verify with the log messages "Compiling rule base {}" and "Creating Kie container...") and doesn't get down after KieBase is created.

<<

KieBases can be large, depending on the number of rules.

<<

one question when you say it can be large does it get that large about 6G or more for ~50,000 rules? 

Wondering if we put effort to migrate to drools 8.x and use executable model (if I could manage to make the migration) will that make any impact on memory footprint? documentation says it is faster and efficient but not sure about memory.

Also does breaking very large rules (rules with hundreds of conditions) into multiple smaller rules make any difference? probably not!

Regards

[Toshiya Kobayashi]

Hi,

I'm intereted in your report and tried some tests on my end.

50000 rules (a 53MB file) resulted in 3.7GB with Drools 8.44.0.Final with non-executable-model. Smaller than yours but the similar trend.

Executable model required more memory at the build time, so it doesn't seem to solve your issue.

I filed a GH issue for investigation and possible improvements.

https://github.com/apache/incubator-kie-drools/issues/6082

Unfortunately, I don't have an immediate advice. If a kbase is too large, probably it's better to split rules and run them in different processes, if possible.

It would be great if anyone who have lots of rules like 50000 can share their experiences / solutions.

Toshiya

2024年9月11日水曜日 3:42:41 UTC+9 A Hussein:

[A Hussein]

Hi Toshiya,

Thanks for looking into this and raising an issue to investigate this.

yeah our memory usage also is not always that big. I have seen it can differ when running locally or deploying on cluster and maybe based on available system resources but on average it is around 4-5GB which is similar to your results.

<<

Executable model required more memory at the build time, so it doesn't seem to solve your issue.

<<

yes I was trying to build this locally and it was failing with java heap space. 

But if the large memory required is only during build time and not during runtime then that might work for us as we can afford to run build job once on specific nodes. Do you remember how much memory it takes during runtime with the executable model? Would it be possible also to share example with how to run with executable model as I am kinda lost what is needed?

<<

<<

If a kbase is too large, probably it's better to split rules and run them in different processes, if possible.

<<

We will have to look if that is feasible for us and if that will reduce overall memory footprint by the different processes.

Yeah it will be great to hear about others experience with similar situation.

I have attached below output from one heapdump analysis if that might help:

Regards

Abdu