Help?

[FB]

Hi,

Thank you for developing STAN. I am currently doing a PhD and I have been using STAN to fit a multi-state back-calculation model to estimate age-specific HIV incidence.

I wrote and ran the model, but I have occasional divergent transitions warnings and the model runs quite slowly (despite it mixes well), so I was wondering if you could help me with that.

This model is the biggest part of my PhD work, so I am not entirely willing to put it on the mailing list. Would it be too much to ask to contact you, and send the codes, via email?

Many thanks
F.

[Daniel Lee]

The first thing to try is to increase adapt_delta to something higher: 0.9 or higher. See the manuals / documentation for the interface you're using to figure out how to set that parameter. It will force the stepsize used in HMC to be smaller, which will hopefully be able to simulate the Hamiltonian trajectory accurately enough to avoid the divergences.

The next thing to do is try to reparameterize. For that, see the manual and look for the section on non-centered parameterization.

Want to try those two things first?

Daniel

--
You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+unsubscribe@googlegroups.com.
To post to this group, send email to stan-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bob Carpenter]

I would worry more about getting people to read your Ph.D. work
than I would about trying to protect it from being stolen.

The more feedback you get up front, the less likely you will
be to reinvent the wheel. And the better the final product will be.
But it's up to you (and your advisor(s) or colleague(s) if it's joint
work).

We can try to answer private questions if we have time, but they're
not a priority. The other issue is which of our devs to send it to.
We all have different specialities as far as modeling and computation
go.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.

[FB]

Hi Daniel and Bob,

Many thanks for your quick replies.

To Daniel: I have been playing a lot with re-parametrizations of the model. For example I have a multivariate normal prior (with known covariance matrix) on some of my parameters: I started by using the covariance matrix formulation but I had a lot of divergent transitions and in that case increasing adapt_delta (to 0.99) did not solve the problem. I then found that parametrizing in terms of the precision (rather than the covariance matrix) leads to very little divergent transitions (0/1/2 depending on the seed), in this case I have not tried increasing adapt_delta but I guess it would help even though it would also make the model even slower (it takes approximately 6h for 2000 iterations now). Concerning non-centered re-parametrization I have read the manual, however my observations are Poisson distributed so I do not know how to re-parametrize in such case.

To Bob: I understand your point and I understand that private questions are not a priority. However, after discussion with my advisor, we concluded that it is not a good idea to make the model public yet. If we get the model to work and the issues I encounter can help other people, we are very happy to make it public later. I have a commented version of the code along with some modeling notes that I wrote to clarify what the codes do that I could share if you can help me.

Thank you
Fran

[Bob Carpenter]

> On Sep 2, 2016, at 7:12 PM, FB <fb...@cam.ac.uk> wrote:
>
> Hi Daniel and Bob,
>
> Many thanks for your quick replies.
>
> To Daniel: I have been playing a lot with re-parametrizations of the model. For example I have a multivariate normal prior (with known covariance matrix) on some of my parameters: I started by using the covariance matrix formulation but I had a lot of divergent transitions and in that case increasing adapt_delta (to 0.99) did not solve the problem. I then found that parametrizing in terms of the precision (rather than the covariance matrix) leads to very little divergent transitions (0/1/2 depending on the seed), in this case I have not tried increasing adapt_delta but I guess it would help even though it would also make the model even slower (it takes approximately 6h for 2000 iterations now). Concerning non-centered re-parametrization I have read the manual, however my observations are Poisson distributed so I do not know how to re-parametrize in such case.

Hard to tell without seeing the model. But it's the parameters
that get non-centered, not the observations. It'll work for
any GLM-type model (basically any hierarchical parameters with
location-scale parameterizations).

We very strongly recommend parameterizing in terms of Cholesky
factors for correlation matrices and scaling them (see the manual).
It will be much much faster and much much more stable arithmetically.

You also may not need 2000 iterations. And whatever you do,
run multiple chains so you can diagnose non-convergence issues.

- Bob

[Michael Betancourt]

Just to play bad cop a bit — I think everyone on the Stan team

came to their current positions because they want to help

scientists to better science, but it’s _really_ hard to do that

without seeing the models and the data.  There’s just too 

much that can go wrong, so if the generic advice in the manual

hasn’t been helpful then there’s not too much more that we can 

do.

We are sympathetic to the sensitivity of the results, but honestly 

private consulting is very time consuming and without proper 

collaboration (including input on the model and co-authorship) 

it’s hard to motivate such help when we could be helping so many

more people with open problems.

Given the constraints I think that your most promising approach 

right now is to try to identifying parts of the model that might be 

causing issues, and abstract those away to examples that you

can share publicly.  It is so much easier to answer specific questions 

with example models.

[Michael Betancourt]

Oh, alternatively, we might be able to offer consulting for

a fee if there’s funds in your advisors budget!

[Bob Carpenter]

I like not being the only bad cop any more. Go Daniel and Michael!
Now maybe we need to hire a good cop :-)

Public consulting's time consuming, too, but hopefully more than
one person can benefit.

Another thing to add is that I don't like being shown things on
a non-disclosure basis, because I can't keep track of where I
learned what. I don't want to violate anyone's confidentiality,
so I simply try to avoid looking at anything that involves
non-disclosure. That includes turning down lots of paid consulting
gigs and invitations to talks that require NDAs. I'll almost never
sign an NDA or a non-compete. As Michael said, I came to this position
because I like working in the open and got tired of NDAs after
ten years of being in industry.

And thanks for asking first. I really don't want people sending
me things and then asking me not to share them. I go through
them too fast and might inadvertently forward them or disclose
something I shouldn't without even realizing it.

- Bob