Diagnosing very low mean acceptance fraction/ very large autocorrelation time

[A.N.]

Hi all,

I was hoping for any advice as to how to go about diagnosing or interpreting a very low mean acceptance rate / very large autocorrelation time.

For three of six or seven parameters,  the autocorrelation time becomes extremely large and continues to increase during the sampling (say, 3000, 4500 at around 30,000 iterations taking 24h). The acceptance fraction also drops to a very low value (say ~0.079). A single iteration takes a few to several seconds.

It is not immediately clear to me that the posterior distribution should be difficult to sample; in fact I expected it to be fairly flat when marginalised over the parameters causing trouble (and the unconverged results seem to suggest it to be so for at least two of the three troublesome parameters).

Thanks for your time.

[Dan Foreman-Mackey]

Hi - There isn't really anything too general to say here. That kind of low acceptance fraction isn't that uncommon, but it does sound like something pathological is going on with your specific problem. If you want to email more details off-list I'm happy to try to provide some advice.

Dan

--
You received this message because you are subscribed to the Google Groups "emcee users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emcee-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emcee-users/b628befe-c3c4-487a-9576-ef848b928b0cn%40googlegroups.com.

--

Dan Foreman-Mackey

Research Scientist

Flatiron Institute

https://dfm.io

[A.N.]

Thanks very much for your kind offer. I think I sorted out the low acceptance rate and long autocorrelation times and just posting here in case this may help anyone else in the future.

In my particular case, I needed to perform a re-parameterisation of parameters which span more than 1 order of magnitude into log space; the acceptance rate is healthy after that and so are the autocorrelation times. I believe this choice is related to what are described as "scale parameters" and is to do with minimising the relative error.