dmn performance tuning

[Ravi Mangalagiri]

Hi Kogito Team,

I would like to run our scenario and get a better understanding of DMN performance. 

Requirement:

Execute a DMN in a basic java project for each entity. Run for 10,000 entities and gather performance numbers.

DMN Logic:

For each Enitit

 "ssn": "100000001"}
{ "ssn": "100000002", "offsets": [{"tc": "706",  "wc": "008"}, {"tc": "826",  "wc": "008"}, {"tc": "979",  "wc": "010"}, {"tc": "706",  "wc": "008"}] }
{ "ssn": "100000003", "offsets": [{"tc": "706",  "wc": "008"}, {"tc": "826",  "wc": "008"}, {"tc": "826",  "wc": "000"}] }
{ "ssn": "100000004", "offsets": [{"tc": "706",  "wc": "008"}, {"tc": "706",  "wc": "008"}] }
{ "ssn": "100000005", "offsets": [{"tc": "826",  "wc": "008"}, {"tc": "979",  "wc": "010"}, {"tc": "979",  "wc": "010"}] }
{ "ssn": "100000006", "offsets": [{"tc": "706",  "wc": "008"}, {"tc": "826",  "wc": "008"}, {"tc": "826",  "wc": "008"}, {"tc": "706",  "wc": "008"}] }
{ "ssn": "100000007", "offsets": [] }
{ "ssn": "100000008"}

get offsetset count incrementally by building the offset keys [tc + ", " wc]. 

[OffsetCounter( key=706, 008, count=7 ), OffsetCounter( key=826, 008, count=5 ), OffsetCounter( key=979, 010, count=3 ), OffsetCounter( key=826, 000, count=1 )]

Issue:

The DMN execution takes 10 - 12 seconds to run for 10,000 entities while Java takes less than 300 ms. Now I do understand Java is faster by default (call by reference), but DMN should not be that off. I wanted help to validate the DMN if I can fine tune for better performance.

Please find the attached 

GitHub repo - https://github.com/ravim786/dmn-performance

DMN - https://github.com/ravim786/dmn-performance/blob/master/src/test/resources/run15perAccount.dmn

DMN Runner - https://github.com/ravim786/dmn-performance/blob/master/src/test/java/com/foo/dmnperformance/DMNPerformanceRunner.java

Java Runner - https://github.com/ravim786/dmn-performance/blob/master/src/test/java/com/foo/dmnperformance/JavaPerformanceRunner.java

I also attached the DMN for reference. I tuned by building the key lists and have another flavor which improved to 9 seconds - https://github.com/ravim786/dmn-performance/blob/master/src/test/resources/run15PerfFix2.dmn

Thanks for a your support

Ravi Mangalagiri

 

[Matteo Mortari]

Hi Ravi,

We always strongly advise for performance testing (and sizing) to refer to your Red Hat Technical Account Manager or at least track it as a Support Case on the Red Hat's Customer Portal.

ref: https://drools.org/product/services.html

That premised, I quickly skimmed anyway the linked code you provided, and I noticed the following observations which might help you refactor a couple of things, also in light of engaging with Red Hat consulting and support.

Observation 1.

In your "performance runner", you are performing I/O operations which are unrelated to the data transformation pertaining to the DMN or Java logic, and it's not immediately clear to me what is your intended target.

Also, and most concerning in terms of performance, you are using internal APIs for DMN which needs to preprocess the incoming data in order to make sure it complies to the ItemDefinition and the DMN semantic; this aspect is not even present in the Java variant of your "performance runner".

This leads me to believe it's a bit of an apple-and-oranges comparison? I hope I was able to explain my doubts when looking at the code.

In this context, we also encourage to use a proper benchmark framework, such as JMH.

Using System.nanoTime() snapshots can be helpful to get an overall ballpark and initial idea, but if you are on a journey of collecting performance metrics, you will likely need a more solid benchmark framework.

You can reference https://github.com/kiegroup/kie-benchmarks for some examples as well.

Observation 2.

In your supplied DMN model, it doesn't seem to me there are any business logic or domain logic concerns.

e.g.: there are no decision tables in this DMN model, there is no "approval"/"rejected" based on data in this DMN model, there is no apparent "DMN Pattern" applied for decision logic.

This leads me to believe this DMN model is not really about making any Decisions?

If your main interest here is about specific data transformation which is never subject of discussion with Business Analyst or Stakeholders, and you found a "better way"(on any dimensions beyond merely performance) which better supports your interest, it could be best to factor-in those considerations in your integration strategies for your use-case.

Hope this helps!

Please don't forget to raise it soon with your Red Hat Technical Account Manager or at least track it as a Support Case on the Red Hat's Customer Portal, so to start a proper engagement which can go in more details and to more extents than my brief couple of high level observations!

MM

--
You received this message because you are subscribed to the Google Groups "Kogito development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kogito-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/CALg%3DC8Ndtwk7H896aiO1-4%3Df6hyDnSxWG%3DMtfKjvPZCAg%3D8fdA%40mail.gmail.com.

--

Matteo Mortari

linkedin.com/in/matteomortari

youtube.com/MatteoMortari