dealing with correlated parameters in stan

[Linas Mockus]

Hi,

Can anybody advice how to deal with correlated parameters? I found that loc_phi and scale_phi are correlated (as shown by pairs).  Is it possible to re-parametrize the model so they become uncorrelated?  The model is a random intercepts gravity model where log of GDP per capita, log of population, and log of distance are explanatory variables.

The model and trace are attached.

Thank you,
Linas

[Bob Carpenter]

I don't have any experience with hierarchical gamma priors, so
don't have any suggestions. Anyone?

parameters {
vector<lower=0> [N] phi;
real<lower=0> loc_phi;
real<lower=0> scale_phi;

model {
phi ~ gamma(loc_phi, scale_phi);

...

- Bob

P.S. Tabs in code are problematic because they render differently
depending on how text editors are set up.

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <ranint.stan><FAA_ACI9_noDummies_1_AsiaCountries_NB_Bayes_2012_with_extra_140_ranint.stantracev.pdf>

[Andrew Gelman]

Generally I’m preferring lognormals or log t just cos I have better intuition for them. But I’m sure that there are settings where gamma makes sense.
A

[Linas Mockus]

Hi,

There is no specific reason to use gamma. I just picked it to be always
positive. My question is if there are some quidelines to choose
hyperpriors. The parameter is phi (for negative binomial to be
overdispersed) and since phi is a vector it was hyperparametrized. I
don't know if this is the right approach.

Thank you,
Linas

--
Thank you,
Linas

[Bob Carpenter]

Andrew's about to start our Wiki on recommended priors, so maybe
he'll address this.

- Bob

[Linas Mockus]

Hi,

It looks like that lognormal did the trick. The code is attached. It
increased waic slightly. The correlation is still there but it is less
pronounced.

I wonder if techniques such as Matt's trick and/or adding additional
parameters could reduce correlation (as in eq 2 in Gelman's "Prior
distributions for variance parameters in hierarchical model").

Thank you,
Linas
On Mon, 08 Jun 2015 16:07:36 -0400, Andrew Gelman
<gel...@stat.columbia.edu> wrote:

--
Thank you,
Linas

[Bob Carpenter]

You could definitely try a non-centered version of the lognormal.
I'm not exactly sure what the form should be --- you have to be
careful the posterior's what you want because of the nonlinear
(log) transform of y.

- Bob

> <ranint1.stan>

[Linas Mockus]

Thanks, Bob. I wasn't able to find a way how to construct a non-centered
version of lognormal - is it possible to provide any reference. I have
found non-standard versions for t, chi square, even gamma, but not log
normal. By poking around I got the impression that non-centered version
converges better than centered version so I am eager to try.

Thank you,
Linas

--
Thank you,
Linas

[Bob Carpenter]

I think this'll do it:

Centered:

parameters {
vector<lower=0> a[K];
real mu_a;
real<lower=0> sigma_a;

model {
a ~ lognormal(mu_a, sigma_a);

Non-Centered:

parameters {
vector<lower=0> log_a_raw[K];
real mu_a;
real<lower=0> sigma_a;

transformed_parameters {
vector<lower=0> a[K];
a <- exp(mu_a + sigma_a * log_a_raw);

model {
log_a_raw ~ normal(0, 1);


I've never tried it on the log scale, so I'm curious how well
it'll work.

- Bob

[Linas Mockus]

Thanks, Bob. Mixing improved quite a bit, correlation didn't really change
- see attached trace and pairs. The first pdf is with centered lognormal,
the second pdf is with non-centered lognormal. I wonder if log_a_raw
should span (-inf,+inf) instead of being positive?

Thank you,
Linas

[Linas Mockus]

The code with centered and uncentered versions is attached.

Thank you,
Linas

[Bob Carpenter]

Oh, yes, log_a_raw shouldn't have had lower=0 --- sorry about that.

You should get the same posterior, so I'd hope the correlation of
parameters in the posterior would be the same.

- Bob

> <FAA_ACI9_noDummies_1_AsiaCountries_NB_Bayes_2012_with_extra_140_ranint1.stantracev.pdf><FAA_ACI9_noDummies_1_AsiaCountries_NB_Bayes_2012_with_extra_140_ranint4.stantracev.pdf>