Trade-off between divergence and Bayesian Fraction of Missing Information

[Stephen Wood]

I’m trying to replicate the soil carbon example given online at http://mc-stan.org/documentation/case-studies/soil-knit.html. When I run the model with the control parameters given in the example (adapt = 0.9) I get a warning message of having divergent transitions. When I tweak the control parameters to eliminate divergence (say, set adapt = 0.999), I get a different warning of having low Bayesian Fraction of Missing Information. I can’t seem to get around both of the warnings, only one or the other.

Has anyone run into and solved this trade-off between divergence and missing information?

I’ve replicated the model exactly from the example, with the only change being that I modified the original integrate_ode() to integrate_ode_bdf(). Though I have the same problems if I use integrate_ode_rk45(). My .stan and .R files are attached.

Thanks,

Steve Wood

[Michael Betancourt]

It’s not that there is a trade off between divergences and low E-BFMI,

rather the divergences are masking a low E-BFMI.  Remember that

divergences indicate that you’re not able to explore the entire target

distribution, and that incomplete exploration can give you inaccurate

E-BFMI estimates.  Only when you increase adapt_delta and the 

exploration becomes more accurate does the accurate E-BFMI

emerge!

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bob Carpenter]

Now we just need to figure out what that model isn't fitting properly!

Milad --- did you manage to get this working without divergences?

I did manage to run and recover simulated parameters and recover the
results of other fits done in soilR, but nothing says that was getting
the right answer.

- Bob

[Michael Betancourt]

The small E-BFMI typically indicate heavy tails, so it could
be that the priors are not sufficiently identifying the model.
Not unlikely for a complex ODE-based system.

[Milad Kharratzadeh]

In the case study attached, I have examples for both real and simulated data. For real data, with adapt_delta=0.9999 and stepsize=0.003 (four chains, 500 iterations each), I do not get any warning. Also, for more sophisticated models with hierarchical structure (like the ones in this repo) with sufficiently high adapt_delta, I don't get any warnings. 

Milad

On Mon, Mar 20, 2017 at 5:48 PM, Bob Carpenter <ca...@alias-i.com> wrote:

Now we just need to figure out what that model isn't fitting properly!

Milad --- did you manage to get this working without divergences?

I did manage to run and recover simulated parameters and recover the
results of other fits done in soilR, but nothing says that was getting
the right answer.

- Bob

> On Mar 20, 2017, at 4:28 PM, Michael Betancourt <betan...@gmail.com> wrote:
>
> It’s not that there is a trade off between divergences and low E-BFMI,
> rather the divergences are masking a low E-BFMI.  Remember that
> divergences indicate that you’re not able to explore the entire target
> distribution, and that incomplete exploration can give you inaccurate
> E-BFMI estimates.  Only when you increase adapt_delta and the
> exploration becomes more accurate does the accurate E-BFMI
> emerge!
>
>> On Mar 20, 2017, at 3:08 PM, Stephen Wood <stephe...@gmail.com> wrote:
>>
>> I’m trying to replicate the soil carbon example given online at http://mc-stan.org/documentation/case-studies/soil-knit.html. When I run the model with the control parameters given in the example (adapt = 0.9) I get a warning message of having divergent transitions. When I tweak the control parameters to eliminate divergence (say, set adapt = 0.999), I get a different warning of having low Bayesian Fraction of Missing Information. I can’t seem to get around both of the warnings, only one or the other.
>>
>> Has anyone run into and solved this trade-off between divergence and missing information?
>>
>> I’ve replicated the model exactly from the example, with the only change being that I modified the original integrate_ode() to integrate_ode_bdf(). Though I have the same problems if I use integrate_ode_rk45(). My .stan and .R files are attached.
>>
>> Thanks,
>>
>> Steve Wood
>>

>>
>> --
>> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.

>> To post to this group, send email to stan-...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.