Horseshoe prior on single coefficients?

[Felix Schönbrodt]

Hi all,

I have one technical one one conceptual question, and would be interested in your thoughts.

First, conceptual:

We have a model where we have a set of theoretically relevant variables ("Set 1"). We want to retain them in the model, regardless whether they contribute to the prediction or not.
Furthermore, we have an additional set of control variables ("Set 2"). This is a large set of potential covariates, and we would only like to keep those which make a contribution, but still keep the model as simple as possible. Hence, Set 1 refers to confirmatory hypotheses, and Set 2 is exploratory (however, both are in the same model).

Our idea is to put a weakly informative prior on Set 1 (say, "normal(0, 1)" on standardized variables), and a more aggressive regularization (LASSO or horseshoe) on Set 2.

Does this make sense to you? Any pitfalls we might have overseen?

Second, technical:

The manual says: " The horseshoe prior can be applied on all population-level effects at once (excluding the intercept) by using set_prior("horseshoe(1)")". Is it also possible to set horseshoe or lasso priors on single parameters?

If not: is this an inherent limitation, a limitation of brms, or a limitation of STAN? Ideas for workarounds?

Thanks,
Felix

[Paul Buerkner]

Hi Felix,

your idea sounds reasonable.

The technical problem you are facing is a current limitation of brms. I will think of a way to make this possible.

Best,
Paul

[Paul Buerkner]

Hi Felix,

with the latest dev version of brms from github, you may try the following:

priors <- c(prior(normal(0, 1), nlpar = "eta1")
            prior(horseshoe(), nlpar = "eta2"))
bformula <- bf(y ~ eta1 + eta2,
               eta1 ~ <set1 variables>,
               eta2 ~ 0 + <set2 variables>,
               nl = TRUE)
fit <- brm(bformula, <your data>, prior = priors)

The intercept of eta2 has to be removed or otherwise we will end up with two intercepts.
Please make sure not to use factors in <set2 variables> in order to avoid accidental cell mean coding.

I am curious to hear if this solutions works out for your examples.

Best,
Paul

Am Mittwoch, 22. Februar 2017 17:14:11 UTC+1 schrieb Felix Schönbrodt:

[Felix Schönbrodt]

Wow, that was quick, thanks!

Technically, it works (but it generates more divergent transitions than the other models).

Here's my code:

priors <- c(prior("normal(0, 1)", nlpar = "eta1"),
            prior("horseshoe(1)", nlpar = "eta2"))

bformula <- bf(y ~ eta1 + eta2,
               eta1 ~ age + sex + bmi + map + ldl + tch,
               eta2 ~ 0 + hdl + ltg + glu + tc,
               nl = TRUE)

# adapt delta due to divergent transitions after warmup                            
l.brm.horseshoeSet2 <- brm(bformula, data=df, prior = priors, warmup = 1000, iter = 2000, chains = 4, cores=4, control = list(adapt_delta = 0.999))

I used a demo data set "diabetes" from the lars package, and put hdl + ltg + glu + tc into set 2 with a horseshoe prior.

Intuitively, I would have expected that the Set-2 coefficients from the l.brm.horseshoeSet2 model (purple) are close to the coefficients from the global horseshoe model (blue) - this is not really the case:

In fact, the coefficients from the new feature seem a bit off. But I have to play around more to get a better understanding.

Cheers,
Felix

On 23 Feb 2017, at 1:17, Paul Buerkner wrote:

Hi Felix,

with the latest dev version of brms from github, you may try the following:

priors <- c(prior(normal(0, 1), nlpar = "eta1")
prior(horseshoe(), nlpar = "eta2"))
bformula <- bf(y ~ eta1 + eta2,
eta1 ~ <set1 variables>,
eta2 ~ 0 + <set2 variables>,
nl = TRUE)
fit <- brm(bformula, <your data>, prior = priors)


The intercept of eta2 has to be removed or otherwise we will end up with
two intercepts.
Please make sure not to use factors in <set2 variables> in order to avoid
accidental cell mean coding.

I am curious to hear if this solutions works out for your examples.

Best,
Paul

Am Mittwoch, 22. Februar 2017 17:14:11 UTC+1 schrieb Felix Schönbrodt:

Hi all,

I have one technical one one conceptual question, and would be interested
in your thoughts.

*First, conceptual:*

We have a model where we have a *set of theoretically relevant variables
("Set 1")*. We want to retain them in the model, regardless whether they

contribute to the prediction or not.

Furthermore, we have an *additional set of control variables ("Set 2")*.

This is a large set of potential covariates, and we would only like to keep
those which make a contribution, but still keep the model as simple as
possible. Hence, Set 1 refers to confirmatory hypotheses, and Set 2 is
exploratory (however, both are in the same model).

Our idea is to put a weakly informative prior on Set 1 (say, "normal(0,
1)" on standardized variables), and a more aggressive regularization (LASSO
or horseshoe) on Set 2.

Does this make sense to you? Any pitfalls we might have overseen?

*Second, technical:*

The manual says: " The horseshoe prior can be applied on all
population-level effects at once (excluding the intercept) by using
set_prior("horseshoe(1)")". Is it also possible to set horseshoe or lasso
priors on single parameters?

If not: is this an inherent limitation, a limitation of brms, or a
limitation of STAN? Ideas for workarounds?

Thanks,
Felix

--
You received this message because you are subscribed to a topic in the Google Groups "brms-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/brms-users/7PYXIAhmV4o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to brms-users+...@googlegroups.com.
To post to this group, send email to brms-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brms-users/3cbe0ebd-8246-433f-b0c1-a6e2b6fea951%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Buerkner]

I believe the problem is that I deactivated autoscaling of the horseshoe prior (scaling via the residual SD sigma) in non-linear models (the model is "technically" non-linear although it ends up just being linear).

I will change that and we can try it again. :-)

 
Am Mittwoch, 22. Februar 2017 17:14:11 UTC+1 schrieb Felix Schönbrodt:

[Paul Buerkner]

Ok, scaling should be working now. I have already run your example and the results look sensible to me.