Profiling and memory sampling results with constraint provider

[mayh...@gmail.com]

Hi,

I have been working on a specific vehicle routing problem with chained domain model in Optaplanner. Previously I used easy score calculator to represent the constraints, before switching recently to Constriant Provider implementation.
However, I have come accross certain performance issues since then. I also have some comparisons for Spring based and standalone java implementations, which shows entity initialization is slower in Spring applications.
For info, constraints include minimizing total travel distance (soft) and minimizing arrival delay at a customer (medium).

1.     Initialization takes time with constraint provider:
   

   

   • With constaint provider solution: For a dataset of size 300 entities (customers) and 6 Vehicles, construction heuristics takes nearly 300 seconds in standalone java app,(whereas in Spring application, initialization could happen only for 200 Entities till timeout of 300seconds). Please check profiler result in the attached picture.
   • With easy score calculator: Same dataset takes 177 seconds in standalone app.
     
     
2.     Another issue observed while using constraint provider: Out of memory error: Java out of heap space after running the application for 4-5 hours. This is a case of real time planning with entities (customers here) that are added and removed in realtime when they are served. At a particular time instant, there are around 50 customers and 20 vehicles, which states that the problem size is not the reason of running out of heap space.
   The same application could run with easy score calculator flawlessly for 8 hours untill it was stopped.
   

    When I let the application run, Heap histogram shows few drools classes as the top consumers of memory (as in picture). The first two pictures added here show memory consumption by objects over a difference of an hour, meaning the consumption doubled in an hour. This escalates the problem and jvm runs out of memory within few hours. After I refactored the constraint implementation (removing redundant filter statements and so on), I still get heap memory issue and in this case third picture provides more details on the retained size by ConcurrentHashMap in constraint streams.

Has anybody encountered such issues or know how to resolve them? Any help is greatly appreciated.

Thanks and with kind regards,

Mayur

[Lukas Petrovicky]

Mayur,

On Fri, Jun 12, 2020 at 10:30 AM <mayh...@gmail.com> wrote:

Has anybody encountered such issues or know how to resolve them? Any help is greatly appreciated. 

I'll assume that this PR is in fact from you:

https://github.com/kiegroup/optaplanner/pull/801

I'll be reviewing it next week, but so far everything points to a fairly simple conclusion: it's a bug. Not closing the session would indeed cause these issues.

Thanks for bringing this to our attention!

--

Lukáš Petrovický

He/Him/His

Principal Software Engineer, Business Automation

Red Hat Czech, s. r. o.

lukas.pe...@redhat.com    IM: triceo/lpetrovi

[Mayur Hole]

Thanks Lukas. Yes, the PR is from me. Looking forward to hearing your comments on the same.

With best regards,

Mayur

--
You received this message because you are subscribed to a topic in the Google Groups "OptaPlanner development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/optaplanner-dev/4iM6N6JzUQU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to optaplanner-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/optaplanner-dev/CAEs6%2Bmw6Vw35-fi3QG1UEW8%3Dap5z3resnYMPeMWyUERa3djp6w%40mail.gmail.com.