日出間健本社総合企画部
unread,Mar 3, 2023, 4:54:26 AM3/3/23Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Gensim
I'm trying to calculate the perplexity using the LDA model log_perplexity.
The official documentation says ”Also output the calculated statistics, including the perplexity=2^(-bound), to log at INFO level.”
Therefore, the following calculation is performed in the program to obtain the actual perplexity value. ”np.exp(-lda.log_perplexity(corpus))”
In general, perplexity should decrease as the number of topics increases. But in my program it rises exponentially.
Is the formula to get the actual perplexity value wrong? Or is the model not learning well? Or is it some other cause?
Everyone in the world, please tell me. from Japan