checkAndPrepareDistributionInfo: random generation function rAExp is missing returnType or returnType does not match the type of the x argument to the corresponding density function.

[Enrico Ryunosuke Crema]

Hello,

I am a developer of an R package (baorista) that depends on nimble (it basically act as a wrapper with some custom distributions) and I'm encountering an error since I've upgraded to nimble 1.2.0. The error can be reproduced as follow:

library(baorista)
data(samplemat)
x.mat <- createProbMat(pmat=samplemat,timeRange=c(5000,3001),resolution=100)
exponential.fit <- expfit(x.mat) 

And generates the error message:

Compiling nimble model...
Error in FUN(X[[i]], ...) :
checkAndPrepareDistributionInfo: random generation function rAExp is missing returnType or returnType does not match the type of the x argument to the corresponding density function.

I can see from my source code that culprit is how rAExp is generated. The custom distribution dAExp does not (and cannot have) a random generator, and rAExp was automatically generated when `nimbleModel()` was executed. Including the following (e.g. at line 60):

registerDistributions(list(

                        dAExp = list(

                        BUGSdist = 'dAExp(z,r)',

                        Rdist = 'dAExp(z,r'),

                        pqAvail = FALSE,

                        types = c('value = double(2)', 'z=integer(0)', 'r=double(0)'))))

Does not seem to fix the problem and generates the same error message. Any suggestions will be greatly appreciated! Thank you!

All Best,

Enrico

[Enrico Ryunosuke Crema]

Just to follow up on this, I see that there has been some changes in `R/distributions_processInputList.R` from 1.1.0 to 1.2.0, I wonder whether the change from

drcf <- get(densityName, pos = userEnv)
rrcf <- get(simulateName, pos = userEnv)
dtype <- environment(drcf)$nfMethodRCobject$argInfo[['x']]
rtype <- environment(rrcf)$nfMethodRCobject$returnType

to 

dtype <- nfMethodRC_obj$argInfo[['x']]

rtype <- sim_nfMethodRC_obj$returnType

has something to do with the error I'm getting? 

All Best,

Enrico

[Perry de Valpine]

Hi Enrico,

Thanks for letting us know and... sorry for evidently breaking your code. That's definitely not on our to-do list.

The distribution processing system was largely re-written. (The reason for this was to support the new feature that nimbleFunction objects (i.e. with setup code and hence member variables) can now be used to provide user-defined functions or distributions in models. When a user-defined distribution or function needs to use data that does not otherwise need to be in the model, this new feature provides a way to avoid putting such data in the model. Instead it can be held in a nimbleFunction object.)

Your code will work if you remove

rAExp <- NULL # which is in a series of NULL assignments

The new distribution processing system is evidently checking on the existence of an "r" function slightly differently than the old system, in a way we did not realize was changing a behavior. It is looking for something called rAExp to see if an "r" function has been defined or whether a placeholder should be generated. When it finds something called rAExp that is not a nimbleFunction, it results in the error. I will have to look more carefully at exactly how previous behavior was implemented, but apparently it did not result in an error when rAExp existed but was NULL, as was the arrangement in your code.

So, I think this gives you a path forward, and it gives us some details to pursue about the new vs old versions of this piece of nimble. Arguably, the new error-trapping makes sense. But it did break previous behavior you relied on. And the error message could be more informative.

Also, although I haven't checked this, I think your code would work with the previous version of nimble with the workaround suggested above (do not define rAExp at all). That is good because I wouldn't want to suggest a change that would then make your package incompatible with the previous version of nimble. But I think that change should be fine with both previous and current versions.

Before releases, we do use check_packages_in_dir (with the reverse argument) to try to catch problems with packages that depend on nimble (reverse dependencies). However what this achieves is fairly limited, e.g. running R CMD check.

Please let us know any further problems.

Best wishes,

Perry

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/b90fe686-5c2a-496c-99b0-c8c2fd425dben%40googlegroups.com.

[Enrico Ryunosuke Crema]

Hi Perry,

That's very helpful and what you write makes absolute sense, and indeed it does fix the problem... However, the reason why I had `rAExp <- NULL` (and similar for other functions) in the first place was to avoid the following NOTE when running `R CMD check --as-cran`

* checking R code for possible problems ... [12s/12s] NOTE
expfit: no visible binding for global variable ‘rAExp’
expfit : runfun: no visible binding for global variable ‘rAExp’
icarfit: no visible binding for global variable ‘rAOG’
icarfit : runfun: no visible binding for global variable ‘rAOG’
logisticfit: no visible binding for global variable ‘rALog’
logisticfit : runfun: no visible binding for global variable ‘rALog’
Undefined global functions or variables:
  rAExp rALog rAOG

The three undefined global functions are effectively defined internally as NULL to avoid this problem. I (think) could register these distributions outside these functions (as done in some other packages like nimbleEcology), but I'm not sure how I can handle the parallelization as I was effectively internalizing what was suggested in this post... I realise this outside the scope of the mailing list (not really a nimble problem!) but if you have any suggestions that would be great!

All Best,
Enrico

[Perry de Valpine]

Oh, that does make sense. I see why you were defining them as NULL. That's helpful in thinking about whether any changes in nimble are called for. It is somewhat unusual behavior for nimble to insert an object into a calling environment, and we only do it because other steps need to find it there. It would take a deeper revision to nimble to work around this, and we are working on deeper revisions, but in the meantime we need a solution.

In your case, would the following (admittedly even weirder) workaround be any use?

rAExp <- NULL # let CRAN's static code analysis see rAExp defined

rm(rAExp)         # the static code analysis shouldn't (I think) be trying to make sense of this.

or even

rAExp <- NULL

evalq( rm(rAexp) ) # in case CRAN's static code analysis does make sense of the rm(rAExp)

A perhaps simpler (but slightly more onerous) solution would be to define your own placeholder function for rAExp. You can simply look at the function generated by nimble to get an idea of what to define. It just needs to match the arguments and return say 0 or -Inf if it is ever called. We also add an output message so that one can be sure to notice if it is ever called.

Perry

On Wed, Jun 12, 2024 at 9:55 AM Enrico Crema <enrico...@gmail.com> wrote:

Hi Perry,

That's very helpful and what you write makes absolute sense, and indeed it does fix the problem... However, the reason why I had `rAExp <- NULL` (and similar for other functions) in the first place was to avoid the following NOTE when running `R CMD check --as-cran`

* checking R code for possible problems ... [12s/12s] NOTE
expfit: no visible binding for global variable ‘rAExp’
expfit : runfun: no visible binding for global variable ‘rAExp’
icarfit: no visible binding for global variable ‘rAOG’
icarfit : runfun: no visible binding for global variable ‘rAOG’
logisticfit: no visible binding for global variable ‘rALog’
logisticfit : runfun: no visible binding for global variable ‘rALog’
Undefined global functions or variables:
  rAExp rALog rAOG

The three undefined global functions are effectively defined internally as NULL to avoid this problem. I (think) could register these distributions outside these functions (as done in some other packages like nimbleEcology), but I'm not sure how I can handle the parallelization as I was effectively internalizing what was suggested in this post... I realise this outside the scope of the mailing list (not really a nimble problem!) but if you have any suggestions that would be great!

All Best,
Enrico

[Enrico Crema]

Hi Perry,

That's very helpful and what you write makes absolute sense, and indeed it does fix the problem... However, the reason why I had `rAExp <- NULL` (and similar for other functions) in the first place was to avoid the following NOTE when running `R CMD check --as-cran`

* checking R code for possible problems ... [12s/12s] NOTE
expfit: no visible binding for global variable ‘rAExp’
expfit : runfun: no visible binding for global variable ‘rAExp’
icarfit: no visible binding for global variable ‘rAOG’
icarfit : runfun: no visible binding for global variable ‘rAOG’
logisticfit: no visible binding for global variable ‘rALog’
logisticfit : runfun: no visible binding for global variable ‘rALog’
Undefined global functions or variables:
  rAExp rALog rAOG

The three undefined global functions are effectively defined internally as NULL to avoid this problem. I (think) could register these distributions outside these functions (as done in some other packages like nimbleEcology), but I'm not sure how I can handle the parallelization as I was effectively internalizing what was suggested in this post... I realise this outside the scope of the mailing list (not really a nimble problem!) but if you have any suggestions that would be great!

All Best,
Enrico

On 12/06/2024 17:11, Perry de Valpine wrote:

[Enrico R. Crema]

Thanks Perry, I opted for the latter solution (i.e. creating the placeholder function) and it works fine!

All Best,
Enrico