Severe sampling problem (high rejection) with non-identifiable model

[Roy Hughes]

I have recently encountered a severe sampling problem when trying to run stan on a model that I have developed. An example of what I'm talking about is in the attached traceplots. problem.obs.error.5.jpg is the traceplot for one parameter of the model. problem.lp.jpg and problem.acc.rate.jpg are the traceplots for the lp__ variable and the accept_stat__ variable
respectively. The model is in the model.stan file. You can see that at a certain point (around iteration 1500) the acceptance rate goes down significantly, the lp__ value goes up significantly,
and the value for the parameter becomes more constrained. This behavior is actually seen for all parameters in this run but I'm just showing the traceplot for one as an example. Also notice that after a while the acceptance rate goes back up and the parameter starts to sample a distribution similar to what it was sampling before the acceptance rate plummeted. The problem
started to present itself when I added a set of parameters to represent the observation error within the error model. These are the obs_error_var parameters that all have inverse gamma prior distributions. The likelihood for the model makes some parameters non-identifiable but all parameters have proper priors from what I can see. The problem mainly started to present itself when I added the obs_error_var parameters.

I really would like some assistance on this. Could this be a bug or problem with the sampling program itself ?

Roy

[Jonah Gabry]

I'm really not sure if this is the problem in your case, but does it help at all if you use something other than the inverse gamma? In section 6.2 of the latest manual (for version 2.9) there's some stuff about why we try to stay away from inverse gamma priors (in particular on pg. 52).