Evidence Ratio for Stanreg objects, and influence by priors

[Dan Biderman]

Dear Paul, 

Hope that my message finds you well.

I am a cognitive psychologist analyzing the data of psychophysical experiments in multilevel regression models. I ran my models in RStanArm, and currently writing my results in a paper. 

I came across your "hypothesis" function and applied it to the Stanreg object.

Does the function take into account the prior distribution I defined for the coefficient in RStanArm? It seems that the function was able to perform a one-sided(a>b), but not two-sided(a=b), hypothesis test. In the latter, I got "NA" as the Evid. Ratio. 

When I ran the same model using the brm function with the argument sample_prior= TRUE, I was able to do the two-sided test.

Is there a way to do the two-sided test for existing Stanreg objects as well?

In addition, what is your opinion regarding the strong influence of the prior on the Evidence Ratio? with priors of normal(0,2) or normal(0,10) for the coeffs, I observed Evidence Ratio of ~13 and ~58, accordingly.

Thanks for the help, and lots of gratitude for creating this wonderful package,

Dan

[Paul Buerkner]

As explained in ?hypothesis, The evidence ratio of a>b is the ratio of posterior probabilities, while a = b is a bayes factor computed via a savage-dickey ratio.

The latter requires samples of the prior distribution, which a stanreg object does not contain. So for a = b, the evid ratio will always be NA for stanreg objects.

Since the evidence ratio of a = b is a bayes factor it will be definition be strongly influenced by the prior (see basically every paper about BFs).

This is one reason I am not a fan of BFs since they depend so critically of the prior.

--
You received this message because you are subscribed to the Google Groups "brms-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+unsubscribe@googlegroups.com.
To post to this group, send email to brms-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/brms-users/fdf30e70-5ccd-43b6-8757-7ddf81c90255%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan Biderman]

thanks Paul

בתאריך יום חמישי, 28 בדצמבר 2017 בשעה 14:48:29 UTC+2, מאת Paul Buerkner:

As explained in ?hypothesis, The evidence ratio of a>b is the ratio of posterior probabilities, while a = b is a bayes factor computed via a savage-dickey ratio.

The latter requires samples of the prior distribution, which a stanreg object does not contain. So for a = b, the evid ratio will always be NA for stanreg objects.

Since the evidence ratio of a = b is a bayes factor it will be definition be strongly influenced by the prior (see basically every paper about BFs).

This is one reason I am not a fan of BFs since they depend so critically of the prior.

2017-12-28 13:44 GMT+01:00 Dan Biderman <danb...@gmail.com>:

Dear Paul, 

Hope that my message finds you well.

I am a cognitive psychologist analyzing the data of psychophysical experiments in multilevel regression models. I ran my models in RStanArm, and currently writing my results in a paper. 

I came across your "hypothesis" function and applied it to the Stanreg object.

Does the function take into account the prior distribution I defined for the coefficient in RStanArm? It seems that the function was able to perform a one-sided(a>b), but not two-sided(a=b), hypothesis test. In the latter, I got "NA" as the Evid. Ratio. 

When I ran the same model using the brm function with the argument sample_prior= TRUE, I was able to do the two-sided test.

Is there a way to do the two-sided test for existing Stanreg objects as well?

In addition, what is your opinion regarding the strong influence of the prior on the Evidence Ratio? with priors of normal(0,2) or normal(0,10) for the coeffs, I observed Evidence Ratio of ~13 and ~58, accordingly.

Thanks for the help, and lots of gratitude for creating this wonderful package,

Dan

--
You received this message because you are subscribed to the Google Groups "brms-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to brms-users+...@googlegroups.com.