Comparing topic models with different preprocessing, what metrics do people use?

[Kyle Jensen]

Hi - 

I'm building a number of models that differ in the way the corpus is preprocessed (stemming, lemmatization, etc.).  However, I do not know how best to compare the "quality" of the resulting models.  I was thinking of using held-out document perplexity.

What do others use?  A rough metric is acceptable to me.

Thanks!

Kyle

[Radim Řehůřek]

Hi Kyle,

it's always best use evaluate the quality directly on your end task,
the one you're trying to solve with these unsupervised models.

Substitute generic metrics like perplexity are also an option, but may
not necessarily correlate well with what you're really trying to
achieve (i.e., a model with lower perplexity could nevertheless
perform worse in practice).

Best,
Radim