latent variable as an index

[Daniel Linden]

Hi NIMBLE folks,

I've worked through several examples and experienced decent performance gains.  I look forward to customizing samplers and having more control over parameter mixing while keeping things specified in BUGS.

I wanted to clarify a problem I'm having with one model that uses a latent variable as an index.  From reading the manual and other discussions here, I have the sense that this is not possible in NIMBLE.  Here are small pieces of the likelihood without the loops:

s[i]~dcat(probs[1:nG])
d2[i,j]<- (grid[s[i],1]-grid[traps[j],1])^2 + (grid[s[i],2]-grid[traps[j],2])^2

This is a distance calculation in a discrete state space for a spatial capture-recapture model.  The s[i] denote a pixel location for individual i, traps[j] denote the pixel location for trap j, and grid is a 2-column matrix of coordinates for each pixel.  In other versions of this model with a continuous state space the coordinates for individuals are specified as 2 random variables in each dimension and the distance calculation works just fine in NIMBLE.  But here, the s[i] are multinomial random variables determined by pixel-specific probabilities so that individual locations can be associated with grid covariates.

Anyway, the model does not compile without errors and I initially thought this was due to differences in data vs. constants.  I don't think data can serve as a variable index, so I made sure that the traps[j] vector was listed under constants.  But obviously the s[i] are latent variables with nodes, so I'm thinking their indexing of grid is the problem.  I have fit models like this with a custom sampler in R where the individual coordinates are kept as random variables (out of convenience) and then a distance calculation is used to identify the closest pixel location in the grid.  Not sure something like that is possible here, or that it would even solve the problem.

I appreciate any insight!

[Daniel Turek]

Daniel, this is a good question, indeed.  But first, I'm happy to hear that you're enjoying NIMBLE, and you're experiencing worthwhile performance gains.

Regarding your question -- you're trying to do what we refer to as "stochastic indexing"; where a stochasitc variable used to index another variable.  At the moment, we do not support that in NIMBLE.  Any variables used for indexing (within square brackets), at the moment, must be provided as a constant, or be a looping index, or be something deterministically derived from these.

It's important to note that handling of stochastic indexing is prime on our to-do list, along with a few other major system improvements.  So please stay tuned for NIMBLE handling this, although not currently.

I'll also note that there's a work-around you could use; although it will certainly hamper the efficiency gains you've been enjoying.  If your s[i] variable is restricted to not too many potential values, then you can replace the following:

grid[s[i],1]

with this, which contains no stochastic indexing of the grid variable:

equals(s[i],1) * grid[1,1]  +  equals(s[i],2) * grid[2,1]  +  equals(s[i],3) * grid[3,1]  +  ...... 

Again, I don't necessary recommend this, as it will lead to inefficiencies the calculations, but it should compile fine, and give the correct result.

Ok, good luck, and happy holidays!

Daniel

for the NIMBLE team

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To post to this group, send email to nimble...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/9344bc34-5783-444f-ac73-090fb664ff9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Perry de Valpine]

Thanks for asking about this.

Here is another workaround that is somewhat more general.  If you write a user-provided function to do the nested indexing with a model variable as the index, it will work.  Here is a simple example that I think captures what you need:

## begin example

library(nimble)

modelCode <- nimbleCode({

    for(i in 1:3) {

        s[i] ~ dcat(probs[1:3])

        d[i] <- myCalculation(grid[1:3], s[i]) ## User-provided function that will do whatever indexing and other stuff you need

    }

})

## see section 5.2.5 of User Manual

myCalculation <- nimbleFunction(

    run = function(grid = double(1), index = double()) {  ## index could be int() but model variables are represented as double anyway

        return(grid[index])

        returnType(double(0))

    })

model <- nimbleModel(modelCode, data = list(probs = c(.2,.3,.5)), constants = list(grid = as.numeric(c(10, 20, 30))))

## Either of these could be data or constants, or stochastic variables in your model

compiledModel <- compileNimble(model)

## check that the calculation works:

simulate(compiledModel)

compiledModel$s

compiledModel$d

## The values of d should be 10, 20 or 30 according to s

testMcmc <- buildMCMC(model)

compiledMCMC <- compileNimble(testMcmc, project = model)

testMcmc$run(100)

testMcmc$mvSamples$s

## that seems to work.

## end example

There is a potential inefficiency because each element of d will depend on all elements of grid[].  So in a situation where grid is dynamic (changing during the course of the MCMC), then every time any value of grid[] changes, each value of d[i] would be recalculated, even if it turns out to be give the same answer.  But I think in your example grid[] is fixed and you just need the stochastic indexing.

Let us know how it works.

Perry

To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/CAKbe0hqv%2BWNampVS65Lz%2B2w9oYcxfsr1q729tmMOgGuDs2puXw%40mail.gmail.com.

[Dan Linden]

Ah, thank you.  I apologize for inquiring about a clearly-stated unsupported feature!  I suppose I initially thought it had to do with parameter dimensions, as addressed by RJMCMC.

I can think of several reasons why stochastic indices might be tricky, so I appreciate that you guys are working on it.

[Dan Linden]

Excellent, Perry, I will give that a try!  The grid is indeed fixed so this should work as expected.