OPTANO GLPK Memory Consumption

[Marcelo Lopes (mlopes)]

Dear all,

I am currently running a sequence of optimizations using the GLPK Solver (WinGLPK 4.65).

I have tried to trace why on every execution 250MB of memory are not returned to the operating system.

After some execution rounds, the total memory consumption reaches 20GB, which is unfeasible for the system proposed (to complete the executions 48GB of memory is needed).

Is there any way to check if this memory leak is related to the GLPK execution?

I have already tried to spread GC.Collect, Created classes using the IDispose interface with no success.

Thank you for your attention,

Marcelo.

[OPTANO Team]

Hi, 

sounds like a bug. 
Could you provide a code sample, which triggers the issue? 

We're going to start investiagions

best

jp

[jannic...@orconomy.de]

Hi Marcelo,

can you please provide a code sample, that reproduces the reported issue?

When you refer to "some execution rounds", what exactly do you mean?

* Repeated calls to glpkSolver.Solve, 

* or complete (independent) runs of your Solution?

Best Regards

Jannick

[Marcelo Lopes (mlopes)]

Hello Jannick,

I have answered the previous message through e-mail, attaching the sample code, but probably you have not gotten the answer.

The CalcPreDespacho class is the one that calls Optano to prepare the model and call the GLPK solver.

Answering your questions about the rounds: the problem occurs after some repeated calls to the glpkSolver.Solve.

I have tried to call some "dispose" methods, etc. with no success. After each round, the memory increases by 250MB.

When closing the application, all memory is flushed, so If you run the application again, the problem will occur after some rounds have been performed.

Basically, the optimization problem is fed by tons of data (a 9076x9076 double matrix, for example), and the optimization runs interactively using some data from the previous run.

The application does this simulation for some scenarios and months. Considering 6 scenarios and 12 months, the application requires 48GB of memory to run without any issues.

I have attached the entire solution (source code and input data) in the following link: https://drive.google.com/file/d/1IzUtmpcZVAo7UhQyQNfpCMNPg6lrauB3/view?usp=sharing

The main classes to observe are Engine, CalcParams, and CalcPreDespacho. CalcPreDespacho is the one who calls the Optano Solver.

Please let me know if you need any additional information.

Thank you for your attention and Best Regards,

Marcelo.

[Jannick Lange]

Hi Marcelo,

I managed to build + run your application. Can you please tell me the parameters that provoke the memory leak?

There seem to be quite a lot of different optimization modes/settings. (I already figured out what data sets to load in the 3 directory inputs).

Thanks

Jannick

[Marcelo Lopes]

Hello Jannick,

Thank you for your attention.

In the link below you can find a small movie (SoftwareSetup.mp4) showing the software setup process. 

https://drive.google.com/file/d/1LnsL_rTsNk6bQn6H-3fRFUPaVp2HuPWo/view?usp=sharing

In the configuration selected, the "250MB extra RAM consumption" shows its colors.

The input files you can find in the directory data/input sent with the source code.

If you need any other information, please let me know.

Thank you again and Best Regards,

Marcelo.

[Jannick Lange]

Hi Marcelo,

I managed to track down a memory leak during within the GLPKSolver.Dispose() method.

I uploaded an updated/fixed version of the NuGet package: https://www.nuget.org/packages/OPTANO.Modeling.GLPK/4.65.2.1

Note: You need to remove some/all of your "manual" disposal-attempts at the end of the optimization loop.

The most cruical one is (probably) the manual call of solver.Dispose. The "using"-block will take care of this automatically. With the manual call in your code, the application will crash.

The memory profile of the application without vs. with the fixed GLPK Solver looks promising:

• Without fix:
  • 
• After fix:
  • 

As you can see, the 'unmanaged memory' now shrinks back to ~100MB after each iteration.

Please let me know if this resolved your issue, or whether you need further assistance.

Best Regards

Jannick

[Marcelo Lopes]

Hello Jannick,

Thank you very much for all your effort in helping me on this.

I have tried to update the package but I am getting the messages below.

Do you have any clue about this issue?

Thank you again and Best Regards,

Marcelo.

Severity Code Description Project File Line Suppression State
Error NU1102 Unable to find package OPTANO.Modeling with version (>= 3.17.0.618-beta)
  - Found 77 version(s) in nuget.org [ Nearest version: 3.16.0.613 ]
  - Found 0 version(s) in Microsoft Visual Studio Offline Packages
  - Found 0 version(s) in Telerik Nuget Solver D:\GPA\Neoenergia\Nodal\NeoNodal\lib\Solver\Solver.csproj 1
Error Package restore failed. Rolling back package changes for 'Solver'.

[Jannick Lange]

I did not notice that the referenced package was only released on an internal NuGet feed.

I just published OPTANO.Modeling 3.17.0 and Modeling.GLPK 4.65.2.2 on the official NuGet website. Please try to reference the 4.65.2.2 version of the GLPK package.

* https://www.nuget.org/packages/OPTANO.Modeling.GLPK/4.65.2.2

* (https://www.nuget.org/packages/OPTANO.Modeling/3.17.0.625)

Best Regards

Jannick

[Marcelo Lopes]

Hello Jannick,

I have made all modifications and the memory problem seems to be vanished.

Thank you so much for all the effort you put into this matter to help me address the issue.

Best Regards,

Marcelo.