Thinning or not Thinning
149 views
Skip to first unread message
Saubhagya Singh Rathore
Apr 30, 2020, 11:37:40 AM4/30/20
to emcee users
Hello everyone,
I have not found a definite answer on whether thinking is useful or not. I have read mixed opinions on this. I read that thinning essentially helps in reducing the the memory requirements. However, thinning, for example taking 1 out of every 10 samples, is not as useful as running the chains 10 times as long. Is there something about how emcee functions that makes thinning a must for independent samples?
Best regards,
Saubhagya
Dan Foreman-Mackey
Apr 30, 2020, 11:54:22 AM4/30/20
to emcee users, Saubhagya Singh Rathore
Thinning is just a way to reduce the size of the output without sacrificing precision. If your maximum autocorrelation time is in the hundreds, you only need to save every hundredth sample which will reduce the size of the files you need to save by 2 orders of magnitude and the precision of your results won't change. There isn't anything magic about it but samples from MCMC are not independent so you don't ever need to save all of them.
--
--
You received this message because you are subscribed to the Google Groups "emcee users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emcee-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emcee-users/b68ef178-f44b-46e4-8d88-86e3be48027f%40googlegroups.com.
Saubhagya Singh Rathore
Apr 30, 2020, 11:59:36 AM4/30/20
to emcee users
Ok, that makes sense. And if we are having a large number of walkers which is typical with emcee, I guess we can definitely pick fewer samples from each chain and still get a good sample size. Thanks for the response.
On Thursday, 30 April 2020 11:54:22 UTC-4, Dan Foreman-Mackey wrote:
Thinning is just a way to reduce the size of the output without sacrificing precision. If your maximum autocorrelation time is in the hundreds, you only need to save every hundredth sample which will reduce the size of the files you need to save by 2 orders of magnitude and the precision of your results won't change. There isn't anything magic about it but samples from MCMC are not independent so you don't ever need to save all of them.
--
On Apr 30, 2020, 11:37 AM -0400, Saubhagya Singh Rathore <saubhagy...@gmail.com>, wrote:
Hello everyone,--I have not found a definite answer on whether thinking is useful or not. I have read mixed opinions on this. I read that thinning essentially helps in reducing the the memory requirements. However, thinning, for example taking 1 out of every 10 samples, is not as useful as running the chains 10 times as long. Is there something about how emcee functions that makes thinning a must for independent samples?Best regards,Saubhagya
You received this message because you are subscribed to the Google Groups "emcee users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emcee...@googlegroups.com.
Saubhagya Singh Rathore
May 4, 2020, 8:22:37 AM5/4/20
to emcee users
Just a follow up question. So, once we are convinced that the chain has converged, for analyzing the posterior distribution of parameters, do we still need to take every (autocorrelation tau)th samples or can we use consecutive samples?
Best regards
Saubhagya
Dan Foreman-Mackey
May 4, 2020, 9:02:21 AM5/4/20
to emcee users, Saubhagya Singh Rathore
Your results will not change (within the sampling uncertainty) if you do one or the other!
--
To unsubscribe from this group and stop receiving emails from it, send an email to emcee-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emcee-users/9e5e6770-d297-4df9-b7e7-80295b175a08%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages