Some further speed improvement for Hierarchical Model possible?

[Christian Meister]

Hello everyone,

at the moment i'm working on a model for growth prediction of trees for many species. So far it is working - but I think I need a little help with speed improvements for my model. I wrote the vectorized form where it was possible - the "MATT-Trick" is not really clear to me - and if I can use it here.

The input-data for the model are ~ 160.000 individuals within 217 species. The model takes really long to converge (Rhat <1.2) - like 7 days with 4 chains, 100.000 Iterations and 50.000 warmup Iterations - each chain running on a different core at a cluster. Is there a possibility of improvement of my code or something else I could check?

Thank you in advance!

[Krzysztof Sakrejda]

mu_h, sigma_h, alpha_0, alpha_1, beta_0, etc... look like they don't have priors. Is that intentional?

Krzysztof

[Bob Carpenter]

We also very strongly advise against those interval constrained
priors for both statistical and computational reasons.

You shouldn't need those 0.0001 lower bounds either.

You might want to make pred a local variable so it doesn't get saved
each iteration.

You have uniform distributions that don't match their constraints.
All parameters needs support (finite log density) on any values
that satisfy their constraints (that'd be sigma_sp, but I didn't
look for more).

It looks like you only have distribution on a_ind[i] if
(repind[i] == 1). The rest of the a_ind will just be uniform over
(-4, 4).

And you definitely want the non-centered parameterization for all the
parameters that don't get normal(0, 1) priors (what we used to
call the "Matt trick"). See the manual for an explanation. But the
basic idea is to replace

u ~ normal(mu_u, sigma_u);

with

u_raw ~ normal(0, 1);

u <- mu_u + sigma_u * u_raw;

You can either do that literally with local variables or implicitly
wherever you use u.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> <model.stan>

[Christian Meister]

Thanks a lot Bob!

I just adjusted my model with your examples. My results now have a higher n_eff and a good Rhat after fewer iterations. Now I only need ~ 17h to get reasonable results.

The repind[i] == 1 is intentional - I need this only, if an individual is two or more times represented in the dataset.

Regarding the priors: I have no or little information for mu_h, sigma_h, alpha_0 and so on. They are adjusted during the iterations to fit the data as uninformative uniform prior with (-∞,∞) and start at different points for each chain around a value from another publication.

If you have also no further questions this topic can be closed.