optimal acceptance rate

[William Heymann]

In literature I normally see optimal acceptance rates listed as 0.234 and in the emcee paper it is listed as between 0.2 and 0.5.

I am trying to figure out what it should be for a sampler like emcee. 

Normally I run burn in until the acceptance rate stabilizes and if the rate is too high or too low I adjust the step size (alpha for stretch, gamma for DE) and reset burn in to let it stabilize again. I repeat as necessary.

Based on my tests this has helped speed up my actual sampling enormously. 

Is there a better value to target than 0.234 and some delta on either side of it?

My distributions are all uni-modal although sometimes highly non-linearly coupled. The model is expensive to calculate and I have run into some situations where to get 50* tau I need to run tens of millions of evaluations. If a better acceptance rate improved the situation it would help enormously. 

I have tried populations from tens to thousands and those don't have a large impact on the acceptance rate. The DE move works MUCH better than the stretch move and often cuts the time to 1/2 to 1/4 of the original time.

I can probably find a way to rescale the problem so that all the variables have approximately the same mean and variance if that would seriously help.

Thanks

Bill

[David W Hogg]

Two comments:

1. Emcee is exactly invariant to linear scaling and shifts in the parameters. So if that changes anything at fixed initialization and random seed, there’s a bug.

2. The optimal acceptance ratio will be problem dependent.

--
You received this message because you are subscribed to the Google Groups "emcee users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emcee-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emcee-users/2f8ef3f2-7eb9-48b6-bcd1-6dd9dcf3e5fe%40googlegroups.com.

--

David W. Hogg — http://cosmo.nyu.edu/hogg/

[William Heymann]

1. What about to non-linear scaling. I have some parameters that are log scaled. Rescaling is not something I have tried yet I have just wondered if it would help.

2. Is there a way to figure out the optimal acceptance ratio? The best idea I have is to try a bunch of different target acceptance ratios and see how long it takes to hit 50*tau which would take quite a lot of time.

On Monday, November 18, 2019 at 1:38:06 PM UTC+1, David W Hogg wrote:

Two comments:

1. Emcee is exactly invariant to linear scaling and shifts in the parameters. So if that changes anything at fixed initialization and random seed, there’s a bug.

2. The optimal acceptance ratio will be problem dependent.

On Mon, Nov 18, 2019 at 05:40 William Heymann <immu...@gmail.com> wrote:

In literature I normally see optimal acceptance rates listed as 0.234 and in the emcee paper it is listed as between 0.2 and 0.5.

I am trying to figure out what it should be for a sampler like emcee. 

Normally I run burn in until the acceptance rate stabilizes and if the rate is too high or too low I adjust the step size (alpha for stretch, gamma for DE) and reset burn in to let it stabilize again. I repeat as necessary.

Based on my tests this has helped speed up my actual sampling enormously. 

Is there a better value to target than 0.234 and some delta on either side of it?

My distributions are all uni-modal although sometimes highly non-linearly coupled. The model is expensive to calculate and I have run into some situations where to get 50* tau I need to run tens of millions of evaluations. If a better acceptance rate improved the situation it would help enormously. 

I have tried populations from tens to thousands and those don't have a large impact on the acceptance rate. The DE move works MUCH better than the stretch move and often cuts the time to 1/2 to 1/4 of the original time.

I can probably find a way to rescale the problem so that all the variables have approximately the same mean and variance if that would seriously help.

Thanks

Bill

--
You received this message because you are subscribed to the Google Groups "emcee users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emcee...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/emcee-users/2f8ef3f2-7eb9-48b6-bcd1-6dd9dcf3e5fe%40googlegroups.com.

[Saubhagya Singh Rathore]

Hi William,

How are you accessing the acceptance rate value? Is there a way to see it through hdfBackend object while sampling is still in progress? Or we have just print it on the screen?

Thanks a lot in advance. 

Regards

Saubhagya

[William Heymann]

Hi Saubhagya,

Sorry I did not reply earlier. 

For the acceptance fraction normally, I use   

accept = numpy.mean(sampler.acceptance_fraction)

I don't use any of the backends in emcee. Instead I use emcee as a generator, and I use the integrated autocorrelation time to determine stopping. I have my own code for batching and saving the data as my system runs. https://emcee.readthedocs.io/en/stable/tutorials/autocorr/

If you just run for 1000, 10000, etc. number of steps the odds are the answer returned is wrong. How many steps you need depends on the problem and how many steps need to be taken before you get another independent sample. I have some systems which need only 300 steps and others that need hundreds of thousands of steps. 

[Saubhagya Singh Rathore]

Hi William,

Thanks for your response. It makes sense now. 

Best regards,

Saubhagya