LDA new document inference with STAN

[Sudeep Das]

I used the ADVI implementation in STAN to perform LDA on a bunch of documents. 

Is there code to do topic assignment on new documents from the learned model? 

Cheers,

Sudeep 

PS: Side question, the output.csv was humongous because it wrote out samples from the posterior for each word -topic assignment in vocabulary and each document-topic assignment. Is there a way to condense the output? 

[Jonah Gabry]

I don't know about your first question, but maybe someone else will. Regarding your P.S. question, it depends. If you don't need to keep the draws for some parameters, then you can declare those parameters in local blocks and they won't be saved. Hard to say if that's what you want to do without seeing your model.

[Jonah Gabry]

I should clarify and say that anything declared in the "parameters" block is always saved. Anything declared and defined in the model block is discarded. Transformed parameters is somewhere in between. Anything declared at the top-level in transformed parameters is saved, but you can use local blocks within transformed parameters to avoid saving intermediate quantities. For example:

parameters {
 real x;
}
transformed parameters {
  real a;
 {
  real b;
  b <- 2;
  a <- x + b;
 }
}
model {
 real c;
 c <- x^2;
}

Here x and a will be saved but b and c will not.

[Bob Carpenter]

You want to just include those "new" documents in the original
model and fit everything at once. That gives you the full Bayes
solution. Having said that, you can't compute full Bayes for
LDA, so everything you do with this is going to be a hack (not
your fault, just the nature of LDA).

- Bob

> --
> You received this message because you are subscribed to the Google Groups "Stan users mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to stan-users+...@googlegroups.com.
> To post to this group, send email to stan-...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.