Hi all,
I'm struggling to solve a modelling challenge and so i'm hoping for guidance and suggestions!
The problem:
I'll avoid the ecological reasons for this for now, but I am working on a joint model comprising a LGCP and a zero-truncated NB count model. I need to be able to compute the expected mean count and include it in the LGCPs log-linear predictor as an offset during model fitting, rather than during a predict() call afterwards.
I am currently fitting the ZTNB likelihood using zeroinflatednbinomial0 with p0 fixed to be very small, the challenge is that the over-dispersion size parameter (which i need to compute the expected mean) is estimated as a hyperparameter, preventing me from accessing it elsewhere in the joint model.
So the key challenge is that i need to find a way to fit a ZTNB to my count data but with the model inputs as named components which can be accessed elsewhere in the joint model.
I have so far tried:
- Fit the PMF non-linearly as a mark in the LGCP: this uses inlabru's iterative INLA approach and works but is very expensive to fit and infeasible in my case.
- Fit the PMF non-linearly as a 1D LGCP: also uses inlabru's iterative INLA, this worked non-spatially quite nicely and was fast, but when adding a spatial field back (which i need) it became pretty much as expensive as the above.
- Poisson-loggamma: tried including an IID observation level random effect to a zero-truncated Poisson group size model to capture the over-dispersion instead, but this still can't be used to compute the mean in the LGCP.
- cloglike: explored building a custom likelihood but it seems to face the same problem with the model construction / requiring hyperparameters. this is also a little outside my skillset..
- Multinomial-poisson transform: this seems promising, i've had some success with transforming the count data into long-format by treating each count observation as a binary 0/1 response out of k = 1,2,..., K possible count values. then fitting the PMF in the predictor of a standard Poisson likelihood using inlabru's iterative INLA to fit. Without a spatial GRF added, this runs almost instantly and seems to be accurate, and critically it gives me control of the input parameters as named model components. However, when i add a spatial field, although it has fitted, it seems to struggle to identify it and there is some issues with the iterative process not converging all the way, i suspect due to the N=1 information constraints mentioned in the article. Another interesting thing is that although initially i was using a normalising constant phi(idx, model = "iid", hyper = list(prec = list(initial = -10, fixed = TRUE))) explained here, it doesn't appear to be required in this case, as the PMF function already enforces normalisation?
Sorry for a lengthy discussion post, i'm open to any suggestions, especially if i am missing an obvious solution!
Also happy to provide a reprex for the mulitnomial trick method
Cheers,
Andrew