Bug in MCMC with dev_vectors

[Cole Monnahan]

Dear users and developers,

In some of my testing I noticed that a stock assessment model had discrepancies between the objective function during the MCMC phase compared to when rerun using -mceval. I originally thought it was a bug in the model, but Ian Taylor tracked it down and it is a bug in ADMB itself. You can read more and reproduce it per Ian's post here:

https://github.com/admb-project/admb/issues/107

How serious it is? Well, for one particular assessment model the MCMC appeared to run fine but after -mceval about 10% of the posterior samples lead to a crashed population. So, bad in that case, but it will vary by model.

For now, I recommend models that use this variable type should not be used in MCMC mode. We are open to ideas on what the appropriate fix is. If you have ideas or comments, please add them to the github.com issue so we can more easily track it.

Thanks,

Cole

 

[dave fournier]

On 04/24/2018 10:47 AM, Cole Monnahan wrote:

In model5.cpp around line 15  you can see that the model uses a different parameterization
of the dev vector depending on whether it is doing MCMC (mc_phase).  So if it does not also use
this in the mceval phase things could get ugly. One could either turn on the flag for the eval phase,
or create a new flag for the mceval phase and turn it on, thus avoiding other possible side effects
of the mc_phase switch, or just not use dev vectors.


    if (initial_params::mc_phase)
    {
      set_value_mc(*this,x,ii,minb,maxb);
    }
    else
    {
      ::set_value(*this,x,ii,minb,maxb,pen);
    }

--
You received this message because you are subscribed to the Google Groups "ADMB Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@admb-project.org.
For more options, visit https://groups.google.com/a/admb-project.org/d/optout.

[Cole Monnahan]

Hi Dave, Do you remember why you used different parameterizations whether in mc_phase or not? The easiest thing would be just to eliminate that if statement and always call  ::set_value(*this,x,ii,minb,maxb,pen). What will break if we do that? This has the advantage that the likelihood surface is the same during optimization, MCMC and mceval. -Cole

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@admb-project.org.

[dave fournier]

On 04/27/2018 08:59 AM, Cole Monnahan wrote:

I only have a vague recollection of disappointment(s) with the dev vector thing. It seemed like a good idea,
and worked great for optimization.  Then I wrote the mcmc stuff and I think some more or less subtle problem showed
up.  Of course the mcsave stuff wasn't around yet.  Then the mcsave stuff came along and hence a bug, proving yet again that
more is less.

[dave fournier]

On 04/27/2018 08:59 AM, Cole Monnahan wrote:

Of course one could just do the reparameterization in the mcsave step, i.e. the mcmc seeks to be OK so keep it as it is, but then
take the dev vector parameters and convert them into the usual ones via  reset and xinit.  If one encounters any problems with this conversion it could be enlightening.