Avoiding memory/reference leaks inside function (?)

[Franz Fuchs]

Good afternoon,

I am attempting to wrap a simmer DES (in this case, a psychiatric hospital with a waiting list) inside a function which would pass various parameters (in this case, total number of incoming admissions, number of beds, average length of stay, and replications) to the simulation for modeling, and return the 5th, 50th and 95th percentile for results of interest. 

The simulation runs for 5 "years", but I truncate the last year of data, bind the list of simulation results together, and process the results in the function itself, so the actual result of the function is only a small data table. 

The end goal is to use mapply to start with a data table of these starting parameters, and build a data table of results from those parameters automatically.

Everything seems to work fine, but I've noticed that RAM consistently creeps upward  -- to the point where, if I have enough parameters and replications, my 32 GB machine is completely consumed and the simulation aborts due to lack of memory. 

I'm suspecting that there is some kind of reference leak in the simulation environment. Definitely not an R expert, so I'm struggling to figure out how to clean these up -- i.e., presumably deleting the simulation results in the environment they are accumulating.

For simplicity, I am not using lapply or mclapply in the function itself -- just a for loop. Simplified code is attached to demonstrate the issue.

Thanks for your time and consideration. 

Very respectfully,

Franz Fuchs

franz...@wyo.gov

[Iñaki Úcar]

Hi Franz,

I am pretty sure that there are no memory leaks. I did find some memory management problems though, namely, memory cleared but not deallocated. I will further investigate this issue and, hopefully, we will see this solved for the next release. Stay tuned.

Regards,

Iñaki

E-Mail to and from me, in connection with the transaction
of public business, is subject to the Wyoming Public Records
Act and may be disclosed to third parties.

--
You received this message because you are subscribed to the Google Groups "simmer-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simmer-devel+unsubscribe@googlegroups.com.
To post to this group, send email to simmer...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/simmer-devel/e64fab46-e78d-4e58-a70e-bc482d9c1d32%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Iñaki Úcar
http://www.enchufa2.es
@Enchufa2

[Franz Fuchs]

hi Iñaki,

Thanks for checking it out. I'm 99% sure the issue is with my code, and so I appreciate the assistance. Very impressed with the package in general -- thanks for building it and putting it out there. 

Vr/

Franz

Policy Coordinator/Legislative Liaison

Wyoming Department of Health

Director's Unit for Policy, Research and Evaluation

401 Hathaway Building

Cheyenne, WY 82002 

(e) franz...@wyo.gov

(o) 307-777-2865

(c) 307-477-0112

--
You received this message because you are subscribed to a topic in the Google Groups "simmer-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/simmer-devel/A4M7uP0Lmgw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to simmer-devel+unsubscribe@googlegroups.com.

To post to this group, send email to simmer...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/simmer-devel/CALEXWq2yA_uzj6EKhz8MpJ0Q%3DdtW4zL7zrDEEqh0bHmtZOQTuQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Iñaki Úcar]

Franz, your code looks good to me. I've found the problem and it's related to how STL containers behave (the C++ part of the package). If you are familiarised with C++, basically it seems that clearing a vector does not deallocate the memory. I didn't know that, and that's why your memory keeps growing. Additionally, I've found another problem with the destructor of the simulator object. More details as soon as there is a working patch.

Thanks for the report.

To view this discussion on the web visit https://groups.google.com/d/msgid/simmer-devel/CAOc6zHZUDk5eRDYOio_9wNomeayyhcXyk0WSnsCUuWVwz8yYww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Iñaki Úcar]

Hi, this issue has been solved on GitHub. As I said in my last mail, I found another problem which prevents the object's destructor from being called in some circumstances. Unfortunately, there is nothing we can do about it right now, because the bug is in magrittr or Rcpp, not in simmer (see the report).

Now, what can you do to prevent your machine from running out of memory?

1. Use the current master branch on GitHub (you can install it with devtools::install_github) until we roll out a new release (at the end of the month, probably). 
   
2. I recommend you to reset your environment once you've extracted the information needed inside your loop, that is: create the env, run it, call get_mon_whatever, and then call reset(env).

Regards,

Iñaki

[Franz Fuchs]

Outstanding. Thanks for tracking down and addressing.

vr/Franz

Policy Coordinator/Legislative Liaison

Wyoming Department of Health

Director's Unit for Policy, Research and Evaluation

401 Hathaway Building

Cheyenne, WY 82002 

(e) franz...@wyo.gov

(o) 307-777-2865

(c) 307-477-0112

To view this discussion on the web visit https://groups.google.com/d/msgid/simmer-devel/CALEXWq0x0ogRmNLmjkoLBJjGvfBydgCepQ7PWh0qNPDD5i0KOw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Iñaki Úcar]

Hi Franz, all,

Thanks to Kevin Ushey, we finally found the bug in magrittr that prevents object destruction (and therefore, memory deallocation) in simmer. I don't expect a new release on CRAN solving this any time soon (note that the last magrittr release dates back to 2014), so I've implemented a workaround in simmer.

With the current master branch, you don't need to reset the environment explicitly anymore. Your code should work now with no memory issues.

Regards,

Iñaki

To view this discussion on the web visit https://groups.google.com/d/msgid/simmer-devel/CAOc6zHarWq5U4_hfWuPYF8zy%3DAFv1BjOfUDC-9zC25t_8BiY3g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Franz Fuchs]

Excellent. Thanks much for tracking this down and resolving. Much appreciated!

Policy Coordinator/Legislative Liaison

Wyoming Department of Health

Director's Unit for Policy, Research and Evaluation

401 Hathaway Building

Cheyenne, WY 82002 

(e) franz...@wyo.gov

(o) 307-777-2865

(c) 307-477-0112

To view this discussion on the web visit https://groups.google.com/d/msgid/simmer-devel/CALEXWq2CraqfEm3bx%2BuEvO6A7O_6mZRNQ9BKfqkYkkdnh03UxQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.