Error loading saved model (pkl)

[izzy]

Just over a year ago, I saved several Lda models to pkl files.  At the time, I had no problem loading these models and using them to extract document topic distributions as well as save the top words in each topic.  Now, I get an error when I try to load these models.  I know that the version of Gensim has changed.  I don't recall which version I was using, but I have tried this after installing many of the previous versions and get the same error.  Here is what it says:

>>> from gensim import models

>>> import logging

>>> logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)

>>> 

>>> mod = models.LdaModel.load("/lda/models/saved_model.pkl") 

2017-07-18 16:27:06,712 : INFO : loading LdaModel object from /lda/models/saved_model.pkl

2017-07-18 16:27:07,792 : INFO : loading expElogbeta from /lda/models/saved_model.pkl.expElogbeta.npy with mmap=None

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/.local/lib/python2.7/site-packages/gensim/models/ldamodel.py", line 1072, in load

    result = super(LdaModel, cls).load(fname, *args, **kwargs)

  File "/.local/lib/python2.7/site-packages/gensim/utils.py", line 252, in load

    obj._load_specials(fname, mmap, compress, subname)

  File "/.local/lib/python2.7/site-packages/gensim/utils.py", line 283, in _load_specials

    val = numpy.load(subname(fname, attrib), mmap_mode=mmap)

  File "/.local/lib/python2.7/site-packages/numpy/lib/npyio.py", line 370, in load

    fid = open(file, "rb")

IOError: [Errno 2] No such file or directory: '/lda/models/saved_model.pkl.expElogbeta.npy'

By the way, it doesn't really matter if this is on unix, or on windows.  I get a similar error.

[Gordon Mohr]

For gensim models, extra-large numpy arrays are saved as separate files alongside the main pickle-file, for efficiency and (in some cases) to work around pickle-limitations. So, is there a `saved_model.pkl.expElogbeta.npy` file available alongside `saved_model.pkl` where it was originally saved... and perhaps not moved to where you're now trying to load the model?

- Gordon

[izzy]

Bingo!! Thanks. Unfortunately, those files were deleted at some point.  I can see them in the logs, though.  Fortunately, I have the original code I used to fit the model (along with the random seeds), so I'll just have to fit the models again.  Good to know going forward.  By the way, for those of you who may encounter a similar problem, there were also ".state" and ".state.sstats.npy" files that were produced that I also neglected to grab.