Non-centered parameterization for DLM

[Justin Silverman]

Hi All, 

I have been working on implementing dynamic linear models in Stan. One question that comes up, does anyone have any ideas as to how to create a non-centered parameterization (I think you guys refer to this as the Matt Trick) for a DLM? Specifically I have in hierarchical model. 

Since its gaussian I figure there should be a way of doing this but I am having trouble figuring it out.

Thank you!

Justin Silverman 

[Justin Silverman]

I should be a little more specific. 

I am working with the following type of model:

Y ~ Multinomial(pi)

pi = softmax(eta)

eta ~ DLM()

were "eta ~ DLM()"  is a model of the form

eta = F_t' * theta_t + v_t ;  v_t ~ N(0, V_t)

theta_t = G_t * theta_t-1 + w_t ; w_t ~ N(0,W_t)

I am using the Kalman filter to marginalize over the state variables (to reduce the burden on Stan's sampler) as appears to be done here and for gaussian_dlm_obs_log. However this marginalization makes it difficult to see how to create a non-centered parameterization and I was hoping for some help. 

Thank you very much!!

Justin Silverman

[Michael Betancourt]

Everywhere you see 

x ~ N(mu, sigma)

replace it with

x_tilde ~ N(0, 1)

x = mu + sigma * x_tilde

In other words, for your updates

eta = F_t' * theta_t + V_t * v_t ;  v_t ~ N(0, 1)

theta_t = G_t * theta_t-1 + W_t * w_t ; w_t ~ N(0, 1)

What you have is already half-way to a non-centered

parameterization.  It would be fully centered if you had

theta_t ~ N(G_t * theta_t-1, W_t)

eta ~ N(F_t' * theta_t, V_t)

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Justin Silverman]

I agree that this would produce a non-centered parameterization of a DLM. However the second part of what I am trying to do is to marginalize over the states. 

The solution you suggest requires that I provide w_t ~N(0,1) as parameters to Stan. Since my parameter space can be very high dimensional I would prefer to keep these parameters out so that stan does not need to include them in the MCMC. 

Does that make sense? Please excuse me if I have misunderstood your suggestion. 

Thank you,

Justin

[Michael Betancourt]

Centered/non-centered refers to latent model — if you

marginalize the latent states out then there is no longer 

any centering or non-centered to do.  That marginalization,

however, is typically infeasible to do analytically hence we

have to fit the joint model.

If everything in Gaussian and your model reduces to an

incremental Kalman filter then why would you even be

using Stan?  The marginal posterior is analytic at each

step so you don’t need MCMC at all.

[Justin Silverman]

In a sense my question is:

How do you create a non-centered parameterization of the distribution given by gaussian_dlm_obs_log? 

Justin

[Justin Silverman]

I am using stan to get estimates for eta_t,  V_t and W_t. I can just marginalize theta_t using the kalman filter. 

Effectively I can marginalize one specific level of the model but even with that it is still a hierarchical model. 

Justin