Evaluation nested too deeply

[Abhirup Datta]

Hi,

Before I try to post a minimal working example, I wanted to describe the error to see if this is a well known issue. I had done my entire data analysis for a paper using Nimble. I have not touched the code for some months as the paper was under review. I tried to rerun the analysis today for the purpose of a revision and I get the following error in the compilation step: 

> Cmode <- compileNimble(model)

compiling... this may take a minute. On some systems there may be some compiler warnings that can be safely ignored.

Error: evaluation nested too deeply: infinite recursion / options(expressions=)?

Error during wrapup: evaluation nested too deeply: infinite recursion / options(expressions=)?

Note that the entire folder was untouched during these months, so I'm wondering how something that was working fine is not running now. I also tried to update to the latest version of Nimble but the problem persists. It seems that the error is arising from a function call within a for loop. 

I'm deeply troubled by this and hence wanted to post this asap to see if something is known about this. 

Thanks,

Abhi

[Abhirup Datta]

Here is the MWE. I noticed that this error seems to be occurring when a variable inside a function has the same name as the function. However, in R a local variable can have the same name as the function. More importantly, the code was working a few months ago and is now giving this error. 

library(nimble)

constants <- list(n = 10)

data <- list(y = 1:10,

             w = rep(1,10),

             x=rep(c(0,1),5))

inits <- list(a = 0, 

              b1 = 0, 

              b2 = 0)

pred_y<-

  nimbleFunction(

    run = function(a=double(0),b1=double(0),b2=double(0),x=double(0),w=double(0)){

      returnType(double(0))

      pred_y = a+b1*x+b2*w

      return(pred_y)

    })

###NIMBLE code

gpCode <- nimbleCode({

  for(i in 1:n){

      y[i] ~ dnorm(pred_y(a,b1,b2,x[i],w[i]),1)

    }

  a ~ dnorm(0, sd=1000)

  b1 ~ dnorm(0, sd=1000)

  b2 ~ dnorm(0, sd=1000)

})

model <- nimbleModel(code=gpCode, constants=constants, data=data, inits=inits, 

                     check = FALSE)

modelconf <- configureMCMC(model)

Rmcmc <- buildMCMC(modelconf)

nimbleOptions(showCompilerOutput = TRUE) 

Cmode <- compileNimble(model)

[Perry de Valpine]

Hi Abhi,

No, we haven't seen this before.  It is conceivable that the change is due to R (if you have updated R), but that's a wild guess.  It is conceivable that a change in nimble could have introduced a bug by accident, but it sounds like you first tried re-running without updating nimble.

I realize it may be a somewhat involved example, but are you able to send it (or a simplified case that still hits the error) to us?  If it is not something you want to share on the whole list, you could send it to nimble...@gmail.com.  When we get to the bottom of it we could post a note about the issue on the list for those interested.

Perry

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users+unsubscribe@googlegroups.com.
To post to this group, send email to nimble...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/c03f6bcd-2fbf-4ba4-a431-40d1b06093bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Perry de Valpine]

Oh, I replied before I saw the further messages.  Ignore my message!

[Perry de Valpine]

Hi Abhi,

You have pinned down the source of the error.  If you choose a different name than "pred_y" inside the function named pred_y, it should work.

Clearly this is a bug.  We will have to do some detective work to learn how it was introduced.  If you have really not changed your version of nimble from when it worked to when it broke, I am confused.  At least you have a way to move forward with your project.  Thanks for the report and tracking it down so far.

Roughly what is happening seems to be the following.  The compiler step establishes a symbol table as it processes code in order to eventually generate C++.  When it hits a function name, if it is a nimbleFunction, it needs to do some initial processing of that nimbleFunction (if not done yet), before continuing.  Somehow, even though "pred_y" is not used as a function in the nimbleFunction, it is found in the symbol table as a nimbleFunction, so those initial processing steps are triggered, in which it then needs to look up "pred_y" in the symbol table, and so on to infinite recursion. The error is reproducible by compileNimble(pred_y), without even using it in a model.

We'll fix the bug.  Thanks.

Perry

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users+unsubscribe@googlegroups.com.
To post to this group, send email to nimble...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/b37f9e81-9655-4171-b663-12ec40568cf4%40googlegroups.com.

[Perry de Valpine]

Just an update.  I've filed an issue and pull request with a fix on GitHub.  It should be on the development version (branch devel on GitHub) soon and then in an upcoming release.

My previous description wasn't quite accurate.  The recursion was not from symbol table lookups and initial processing.  It was from a later stage, when we must ensure the C++ code for one nimbleFunction knows about the code for other nimbleFunctions it might use.

-Perry

[Abhirup Datta]

Perry,

Thanks for looking into this and glad that the bug was found and fixed. Any idea why the code did run fine a couple of months? I have updated other packages during this time and not sure if any dependencies of Nimble had changed. 

Thanks,

Abhi

To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.

To post to this group, send email to nimble...@googlegroups.com.

[Perry de Valpine]

No, I'm not sure what the story is.  Looking at the history, I see a change from about a year ago (May 2017) that introduced one key part of the bug.  I'm curious what happened, but I doubt it's worth much effort to track this down.  If we were to try, a starting point would be to ask you what version of nimble you had installed when it worked.

To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users+unsubscribe@googlegroups.com.

To post to this group, send email to nimble...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/480e7565-d25d-4227-8068-5cbdc143f015%40googlegroups.com.