Stan models for posterior hyperparameter sampling using NUTS

[Alex Coventry]

Greetings.  Are there existing public stan implementations of the following?  I'd be grateful for any pointers.

• Model a set of observation counts, {{n_00, n_01, ...}, {n_10, n_11, ...}, ...}, as generated by multinomials {m_0, m_1, ...} sampled from a Dirichlet D, and use NUTS to sample from the posterior on D's pseudocounts given some hyperprior.
• Model a set of sets of continuous observations {{x_00, ...}, {x_10, ...}, ...}, as generated by normal distributions {N(μ_0, 𝜎_0), ...} sampled from some conjugate distribution, say a Normal-Inverse-χ², and use NUTS to sample from the posterior on the hyper-parameters μ, ϰ, ν, 𝜎 given some hyperprior.

Best regards,

Alex

[Bob Carpenter]

Yes, see the manual. But we tend not to do inference
on the Dirichlet parameters because it's such a weak prior.
You get a generalization to logistic multivariate normal
in the correlated topic model. But you can also do the
Dirichlet by simple generalizations of the beta-binomial, which
we do describe, including suggestions for hyperpriors.

For the second problem, look at linear regression in the manual.
We don't usually use conjugate priors, though, as they're too
restrictive. But there's nothing stopping you from using conjugate
priors.

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Alex Coventry]

Thanks.  Should I be looking at stan-reference.pdf?  I've looked through there, and the implementations aren't jumping out at me.  Also, it doesn't seem to contain the word "pseudocounts."

Best,

Alex

[Bob Carpenter]

Look at the priors for coefficients and scales section
of the regression chapter.

Look for the correlated topic model, page 163 for
the more general prior case.

I'm not sure what you're calling the pseudocount.
In Dirichleta(alpha), the quantity sum(alpha) is
such that alpha - 1 is the prior count (elementwise
subtraction of 1) for each of K outcomes. At least
if you start with uniform and add for counts observed.

You can also check out the beta-binomial case here,
which is trivial to generalize to Dirichlet:

http://mc-stan.org/documentation/case-studies/pool-binary-trials.html

Here's what it looks like in K dimensions:

parameters {
real<lower=0.1> kappa; // prior count minus K
simplex[K] theta; // prior mean
simplex[K] phi[J]; // parameters


model {
... phi is a K-simplex, either parameters or data ...
for (j in 1:J)
phi[j] ~ dirichlet(theta * kappa); // hierarchical prior on phi[k]
kappa ~ pareto(0.1, 1.5); // hyperprior on prior count
theta ~ dirichlet(alpha); // alpha is hyperprior

It decomposes the Dirichlet into mean theta and prior count (minus K)
kappa. You can just skip the dirichlet prior on theta in which
case it defaults to uniform over simplexes.

I should put this in the manual:

https://github.com/stan-dev/stan/issues/2051#issuecomment-246742910

- Bob

[Alex Coventry]

Thanks a lot, Bob.