NO_CYTHON - Compiled extensions are unavailable.

[joy.v...@gmail.com]

Hello,

I'm attempting to install gensim 4.1.2 on Windows Server 2019 Data Center. I have python 3.8, pip 21.2.4. No errors are visible during gensim's installation, however, if I try to import gensim, I get the below error. 

>>> import gensim

Traceback (most recent call last):

  File "E:\Python\Python38\lib\site-packages\gensim\models\fasttext.py", line 264, in <module>

    from gensim.models.fasttext_corpusfile import train_epoch_sg, train_epoch_cbow

ImportError: DLL load failed while importing fasttext_corpusfile: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "E:\Python\Python38\lib\site-packages\gensim\__init__.py", line 11, in <module>

    from gensim import parsing, corpora, matutils, interfaces, models, similarities, utils  # noqa:F401

  File "E:\Python\Python38\lib\site-packages\gensim\models\__init__.py", line 22, in <module>

    from .fasttext import FastText  # noqa:F401

  File "E:\Python\Python38\lib\site-packages\gensim\models\fasttext.py", line 266, in <module>

    raise utils.NO_CYTHON

RuntimeError: Compiled extensions are unavailable. If you've installed from a package, ask the package maintainer to include compiled extensions. If you're building Gensim from source yourself, install Cython and a C compiler, and then run `python setup.py build_ext --inplace` to retry.

>>>

It references missing compiled extensions, however I've installed gensim the exact same way from several laptops and I've had no issues. Also, I've confrimed that Cython 0.29.23 is installed.  The only difference in what I did on the server was that on the server, I installed python to the E drive rather than the default location on C. Is it possible that gensim's extensions went to some default location and this is why they aren't being seen? Where can I go to look for the missing module\extensions?

I've already confirmed that the following files do exist in gensim\models

fasttext_corpusfile.cp38-win_amd64.pyd

fasttext_corpusfile.cpp

fasttext_corpusfile.pyx

I've also tried:

uninstalling and reinstalling gensim

uninstalling and reinstalling cython, numpy, and scipy

confirming that the versions of cython, numpy, and scipy are the same versions that are working perfectly from laptops 

Thanks very much for any ideas you can give me on this.

Joy

[Radim Řehůřek]

Hi Joy,

How did you install Gensim? Can you post / capture / copy-paste the installation log?

AFAIK you shouldn't need Cython at all. If you use standard `pip install gensim`, that will download Gensim from the official Python's package index (PyPI): https://pypi.org/project/gensim/#files

And we have already precompiled Gensim packages for different Python versions and operating systems there – including Python 3.8 and Windows. No compilation needed.

So I guess something went wrong during the package installation – can you send more info?

Best,

Radim

[Joy]

Hi Radim,

Thanks very much for pointing me in the direction of the installation log. I had been just looking at the information returned in the command line window and there weren't any errors there. When I enabled full logging and dumped everything to a very large file, I can see a whole lot  of messages like these:

Skipping link: unsupported archive format: .exe

Skipping link: none of the wheel's tags (cp27-cp27m-win32) are compatible

Skipping link: unsupported archive format: .egg

I am installing using pip install gensim as you mentioned. But I'm in a corporate environment where the pip command for all machines is redirected to an internal repo where all of the packages have been security scanned and had their licensing requirements verified etc.  I think what they've got in the repo is ok since I'm able to install gensim with no problems from a variety of laptops running Windows 10. My best guess based on the errors is that there's some kind of security on the servers that's preventing the pulling of various file types into the server environment. 

Let me know if you think it's something else based on those errors. But I think my next stop needs to be with my internal devops and\or security teams.

Really appreciate your reply from this morning. It got me unstuck and pointed in the right direction.

Joy

[Radim Řehůřek]

Hi Joy,

You can try downloading your particular wheel (Python3.8, Windows) from PyPI manually and installing it directly:

https://pypi.org/project/gensim/#files

See what error message you get – if any.

Best of luck,

Radim