Understanding the step size parameter for HMC

[Guido Biele]

Hello,

I was looking for a way to fix the step size aka

discretization time epsilon, such that only the

mass matrix would be adapted during warmup.

On page 25 of the stan manual it reads 

"Stan can be configured with a userspecified

step size or it can estimate an optimal step 

size during warmup using dual averaging",

so that I expected that I could set the step

size using the stepsize flag, e.g.:

./my_model sample algorithm=hmc stepsize=.03 ....

However, when I look at the values in the output

file, they are not 0.03. Maybe the reason is that 

the step size is still adapted achieve the target 

acceptance rate, which would mean that 

"userspecified step size" does not mean that 

the user can set the step size to a specific value,

independent of the target acceptance rate?

It would be nice if anybody could clarify what 

exactly the stepsize parameter specifies, and 

how this relates to the stepsize__ column in

the stan output.

Best - Guido

[Michael Betancourt]

The stepsize parameter defines the initial step size from which the

algorithm begins.  If adaptation is engaged then this is quickly modified,

but it can be helpful to start with a small step size on particularly nasty 

problems to facilitate adaptation.  If adaptation is not engaged, however,

then this is the step size that will be used throughout sampling.

You cannot adapt the metric without adapting the step size as changes

in the metric require corresponding changes to the step size to ensure 

equivalent scales before and after the update.  If you really want this

behavior then you can hack the underlying code, but I highly recommend

against it as it is a very bad idea.

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Guido Biele]

Thanks,

this was very helpful!

PS: 

I remember having read that maybe sometime

it will be possible to read in the mass matrix 

and step size from a previous fit to shorten or

even replace the adaptation phase. This would

be a great feature for stan!

[Bob Carpenter]

Yes, we're nearly there on the big Stan 3 refactor. Then
adding mass matrix input will be a priority.

- Bob