Key error in LDA model

[Fahra]

Hi,

I am trying to run code from Gensim notebook examples https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/Training_visualizations.ipynb

and keep getting this error when I want to train LDA model. The same happens when trying to run this code: https://github.com/parulsethi/PyconCardiff2017/blob/master/Tutorial.ipynb

I am using the same data as the one in examples, and following versions:

python: 3.5.4

gensim: 3.1.0

numpy:1.13.3

scipy: 0.19.1

OS is Windows 10

This is the code with error below.

Thank you.

KeyError                                  Traceback (most recent call last)
<ipython-input-15-7799a8ce7cb2> in <module>()
      1 # training LDA model
----> 2 lda_model = LdaModel(corpus=training_corpus, id2word=dictionary, num_topics=35, passes=50, chunksize=1500, iterations=200, alpha='auto', callbacks=callbacks)
      3 ## lda = gensim.models.ldamodel.LdaModel
      4 ## lda_model = gensim.models.ldamodel.LdaModel(corpus=training_corpus, id2word=dictionary, num_topics=35, passes=50, chunksize=1500, iterations=200, alpha='auto', callbacks=callbacks)

~\Anaconda3\envs\py35\lib\site-packages\gensim\models\ldamodel.py in __init__(self, corpus, num_topics, id2word, distributed, chunksize, passes, update_every, alpha, eta, decay, offset, eval_every, iterations, gamma_threshold, minimum_probability, random_state, ns_conf, minimum_phi_value, per_word_topics, callbacks)
    338         if corpus is not None:
    339             use_numpy = self.dispatcher is not None
--> 340             self.update(corpus, chunks_as_numpy=use_numpy)
    341 
    342     def init_dir_prior(self, prior, name):

~\Anaconda3\envs\py35\lib\site-packages\gensim\models\ldamodel.py in update(self, corpus, chunksize, decay, offset, passes, update_every, eval_every, iterations, gamma_threshold, chunks_as_numpy)
    701             # append current epoch's metric values
    702             if self.callbacks:
--> 703                 current_metrics = callback.on_epoch_end(pass_)
    704                 for metric, value in current_metrics.items():
    705                     self.metrics[metric].append(value)

~\Anaconda3\envs\py35\lib\site-packages\gensim\models\callbacks.py in on_epoch_end(self, epoch, topics)
    281         for i, metric in enumerate(self.metrics):
    282             label = str(metric)
--> 283             value = metric.get_value(topics=topics, model=self.model, other_model=self.previous)
    284 
    285             current_metrics[label] = value

~\Anaconda3\envs\py35\lib\site-packages\gensim\models\callbacks.py in get_value(self, **kwargs)
    101         cm = gensim.models.CoherenceModel(
    102             self.model, self.topics, self.texts, self.corpus, self.dictionary,
--> 103             self.window_size, self.coherence, self.topn
    104         )
    105         return cm.get_coherence()

~\Anaconda3\envs\py35\lib\site-packages\gensim\models\coherencemodel.py in __init__(self, model, topics, texts, corpus, dictionary, window_size, keyed_vectors, coherence, topn, processes)
    187         self.window_size = window_size
    188         if self.window_size is None:
--> 189             self.window_size = SLIDING_WINDOW_SIZES[self.coherence]
    190         self.texts = texts
    191         self.corpus = corpus

KeyError: 10

[Armin Oliya]

same issue here. somehow it seems 'topn' parm is assigned to self.coherence (e.g. choosing topn=8 when initializing CoherenceMetric will throw KeyError: 8).

is this windows specific? what's the best way to work around it?

[Ivan Menshikh]

Hello Armin,

can you create an issue on GitHub with code & data that need for reproducing your error?

[Armin Oliya]

sure, will do. thanks.

[Armin Oliya]

I just updated to gensim 3.3 and this issue is resolved. thanks.