Problem with R pointer protection stack

[pthomp...@gmail.com]

Hi All,

I'm running a fairly complicated model with RTMB and when I try to build the tape with MakeADFun I am bumping into an error that appears to be related to R's protection stack. The message is "Error: protect(): protection stack overflow". 

I don't totally understand what it means (especially because I cannot find any documentation for an R function called "protect") but I think it basically has to do with having too many recursive expressions. I have maxed out the stack size using "options(expressions = 500000)" and have also changed the value of the command line argument "--max-ppsize" to the same value (the maximum allowed value), but the error still arises.

I think the solution is probably that I need to just fix my model / make it more efficient. The code works as expected when I fit the same model to a subset of the full dataset, too, for what it's worth. But I was wondering if anyone had other suggestions that could be helpful.

I haven't provided an example because the script is pretty long as is, but can attach as a file if folks would like to look at it. I can't think of a good way to produce a minimal working example because the problem is in essence that things are too large. 

But the model is basically estimating parameters using a Kalman filter algorithm with a fairly large "state vector" and some extra modifications to handle outliers. The Kalman filter requires some iterative looping (has to be a loop because the values of state estimates and other things depend on outputs from previous loop indices) and I have taken some advice from this thread to get it running. Before I had replaced all my single-bracket indices (e.g., a[i, j, k] = v for some numeric array a, integers i,j,k, and real number v) with double-bracket indices (e.g., a[[i,j,k]] = v), everything worked fine, but it took forever (>12 hours) just to complete MakeADFun. So I am stuck between having to wait an annoyingly long time or overloading the protection stack; I will probably end up being forced to choose the former but just wanted to see if there were any other options available.

Thank you for any insight and please let me know if there is any additional information I might be able to provide!

Peter Thompson

[Kasper Kristensen]

- Does it work with normal R evaluation of the function (independent of MakeADFun) ?

- Can you tell which code line triggers the error (which RTMB function) ?

- For debugging, it might be useful to read the protection stack size using this function:

ppstack <- inline::cxxfunction(body="return ScalarInteger(R_PPStackTop);",includes="extern int R_PPStackTop;" )
ppstack() ## Read the stack size

If you print the value from within your function, is it significantly higher with MakeADFun than without?

[pthomp...@gmail.com]

Hi Kasper, thanks for looking into this and providing these suggestions. I think your recommendation of debugging through using ppstack() has helped me figure out the problem, which was that I was using "do.call(sum, L)" to sum a list of arrays at the end of my function. I replaced this with an sapply that does the same thing and the error is gone. Thank you for your help!

Peter