Avoiding memory/reference leaks inside function (?)

39 views
Skip to first unread message

Franz Fuchs

unread,
May 10, 2017, 4:49:02 PM5/10/17
to simmer-devel
Good afternoon,

I am attempting to wrap a simmer DES (in this case, a psychiatric hospital with a waiting list) inside a function which would pass various parameters (in this case, total number of incoming admissions, number of beds, average length of stay, and replications) to the simulation for modeling, and return the 5th, 50th and 95th percentile for results of interest. 

The simulation runs for 5 "years", but I truncate the last year of data, bind the list of simulation results together, and process the results in the function itself, so the actual result of the function is only a small data table. 

The end goal is to use mapply to start with a data table of these starting parameters, and build a data table of results from those parameters automatically.

Everything seems to work fine, but I've noticed that RAM consistently creeps upward  -- to the point where, if I have enough parameters and replications, my 32 GB machine is completely consumed and the simulation aborts due to lack of memory

I'm suspecting that there is some kind of reference leak in the simulation environment. Definitely not an R expert, so I'm struggling to figure out how to clean these up -- i.e., presumably deleting the simulation results in the environment they are accumulating.

For simplicity, I am not using lapply or mclapply in the function itself -- just a for loop. Simplified code is attached to demonstrate the issue.

Thanks for your time and consideration. 

Very respectfully,
Franz Fuchs

WSH-issue.R

Iñaki Úcar

unread,
May 11, 2017, 4:27:46 AM5/11/17
to simmer-devel
Hi Franz,

I am pretty sure that there are no memory leaks. I did find some memory management problems though, namely, memory cleared but not deallocated. I will further investigate this issue and, hopefully, we will see this solved for the next release. Stay tuned.

Regards,
Iñaki

E-Mail to and from me, in connection with the transaction
of public business, is subject to the Wyoming Public Records
Act and may be disclosed to third parties.

--
You received this message because you are subscribed to the Google Groups "simmer-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simmer-devel+unsubscribe@googlegroups.com.
To post to this group, send email to simmer...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/simmer-devel/e64fab46-e78d-4e58-a70e-bc482d9c1d32%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

Franz Fuchs

unread,
May 11, 2017, 10:12:35 AM5/11/17
to simmer...@googlegroups.com
hi Iñaki,

Thanks for checking it out. I'm 99% sure the issue is with my code, and so I appreciate the assistance. Very impressed with the package in general -- thanks for building it and putting it out there. 

Vr/
Franz

Policy Coordinator/Legislative Liaison

Wyoming Department of Health
Director's Unit for Policy, Research and Evaluation
401 Hathaway Building
Cheyenne, WY 82002 



--
You received this message because you are subscribed to a topic in the Google Groups "simmer-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/simmer-devel/A4M7uP0Lmgw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to simmer-devel+unsubscribe@googlegroups.com.

To post to this group, send email to simmer...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Iñaki Úcar

unread,
May 11, 2017, 11:12:44 AM5/11/17
to simmer-devel
Franz, your code looks good to me. I've found the problem and it's related to how STL containers behave (the C++ part of the package). If you are familiarised with C++, basically it seems that clearing a vector does not deallocate the memory. I didn't know that, and that's why your memory keeps growing. Additionally, I've found another problem with the destructor of the simulator object. More details as soon as there is a working patch.

Thanks for the report.


For more options, visit https://groups.google.com/d/optout.

Iñaki Úcar

unread,
May 14, 2017, 8:08:38 AM5/14/17
to simmer-devel
Hi, this issue has been solved on GitHub. As I said in my last mail, I found another problem which prevents the object's destructor from being called in some circumstances. Unfortunately, there is nothing we can do about it right now, because the bug is in magrittr or Rcpp, not in simmer (see the report).

Now, what can you do to prevent your machine from running out of memory?
  1. Use the current master branch on GitHub (you can install it with devtools::install_github) until we roll out a new release (at the end of the month, probably). 
  2. I recommend you to reset your environment once you've extracted the information needed inside your loop, that is: create the env, run it, call get_mon_whatever, and then call reset(env).
Regards,
Iñaki

Franz Fuchs

unread,
May 15, 2017, 10:24:11 AM5/15/17
to simmer...@googlegroups.com
Outstanding. Thanks for tracking down and addressing.

vr/Franz

Policy Coordinator/Legislative Liaison

Wyoming Department of Health
Director's Unit for Policy, Research and Evaluation
401 Hathaway Building
Cheyenne, WY 82002 




For more options, visit https://groups.google.com/d/optout.

Iñaki Úcar

unread,
May 16, 2017, 10:11:02 AM5/16/17
to simmer-devel
Hi Franz, all,

Thanks to Kevin Ushey, we finally found the bug in magrittr that prevents object destruction (and therefore, memory deallocation) in simmer. I don't expect a new release on CRAN solving this any time soon (note that the last magrittr release dates back to 2014), so I've implemented a workaround in simmer.

With the current master branch, you don't need to reset the environment explicitly anymore. Your code should work now with no memory issues.

Regards,
Iñaki


For more options, visit https://groups.google.com/d/optout.

Franz Fuchs

unread,
May 16, 2017, 10:11:46 AM5/16/17
to simmer...@googlegroups.com
Excellent. Thanks much for tracking this down and resolving. Much appreciated!

Policy Coordinator/Legislative Liaison

Wyoming Department of Health
Director's Unit for Policy, Research and Evaluation
401 Hathaway Building
Cheyenne, WY 82002 




For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages