R Bomb w/ Large Number of Random Effects

[Matt Cheng]

Curious if folks have run into this issue? I have an age-structured population model, with 65 years, 30 ages, and 2 sexes. I'm trying to implement random effects across these dimensions ~4000, but everytime I try to fit the model, it R-bombs. MakeADFun works fine, but it bombs out at nlminb.

Simplifying such that the RE are shared across sexes reduces the dimensionality to 2k and runs fine ... I'm wondering if my device is just running into memory limitations and there's nothing I can do about it.

I realize there isn't much to go on here, but would love to hear if folks have any advice / tips or tricks!

Thanks!

[Matt Cheng]

I guess it's also important to show what the RE structure looks like ...

rho_trans = function(x) 2/(1+ exp(-2 * x)) - 1

for(s in 1:n_sexes) {

  # Extract out varaibles and transform into appropriate space
  eps_ya = ln_devs[1,,,s,1]
  rho_b = rho_trans(PE_pars[1,1,s,1]) # correlation across bins
  rho_y = rho_trans(PE_pars[1,2,s,1]) # correlation across years
  sigma2 = exp(PE_pars[1,4,s,1])^2 # get sigma
 
  # Define 2d scale
  scale = sqrt(sigma2) / sqrt(1 - rho_y^2) / sqrt(1 - rho_b^2)
 
  # Define ar1 separable functions
  f1 = function(x) RTMB::dautoreg(x, mu = 0, phi = rho_y, log = TRUE)
  f2 = function(x) RTMB::dautoreg(x, mu = 0, phi = rho_b, log = TRUE)
  ll = ll + RTMB::dseparable(f1, f2)(eps_ya, scale = scale)
}

[Ben Bolker]

My experience has been that I had trouble tracking memory usage by
R's built-in memory profiling tools (i.e. that TMB seemed to be
allocating memory in a way that R didn't track). I used

https://github.com/mrc-ide/memprof

to do memory benchmarking; I was able to see that the memory used by
the largest example I could run (i.e. the largest example that wouldn't
crash the R session) was close to the available memory on my system.

You can also watch the system monitor on your system (what it's
called will depend on your OS) and see what happens to memory use as
your program runs ...

> --
> To post to this group, send email to us...@tmb-project.org. Before
> posting, please check the wiki and issuetracker at https://github.com/
> kaskr/adcomp/ <https://github.com/kaskr/adcomp/>. Please try to create a
> simple repeatable example to go with your question (e.g issues 154, 134,
> 51). Use the issuetracker to report bugs.
> ---
> You received this message because you are subscribed to the Google
> Groups "TMB Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to tmb-users+...@googlegroups.com <mailto:tmb-
> users+un...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/tmb-
> users/330c279c-09bb-49b0-9ad3-4ff775468d3an%40googlegroups.com <https://
> groups.google.com/d/msgid/tmb-
> users/330c279c-09bb-49b0-9ad3-4ff775468d3an%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

--
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Associate chair (graduate), Mathematics & Statistics
Director, School of Computational Science and Engineering
* E-mail is sent at my convenience; I don't expect replies outside of
working hours.

[Kasper Kristensen]

If RTMB is crashing your R session it's an RTMB bug, even if ultimately caused by misuse.

A bug report would then be appreciated.