Fasttext still maintained ? - Numpy compatibility issue

[B Thomas]

Hi All,

I am not aware of any announcements that fasttext is no longer supported. However I just noticed that on Mar 19, 2024 fasttext github repository has been set to a read only archive.

Can anyone confirm that this means fasttext is no longer maintained and hence will no longer accept patches.

Note - fasttext specifies that it depends on "numpy" rather than "numpy < 2.0". The recent release of Numpy 2 seems to have broken fasttext. This results in an error such as

Unable to avoid copy while creating an array as requested.\nIf using `np.array(obj, copy=False)` replace it with `np.asarray(obj)` to allow a copy when needed (no behavior change in NumPy 1.x).\nFor more details, see https://numpy.org/devdocs/numpy_2_0_migration_guide.html#adapting-to-changes-in-the-copy-keyword.

File FastText.py\", line 232, in predict
   return labels, np.array(probs, copy=False)

[Juho Inkinen]

Hi!

It certainly seems that Meta silently dropped support of fastText.

Lately we have been installing fastText for Python from https://github.com/messense/fasttext-wheel, but I assume that won't be updated for NumPy compatibility issues etc.

If anyone knows of a well maintained fastText fork, please inform about it!

-Juho

[Lukas Kahwe Smith]

Same here.
Or viable alternatives for language detection

[Henri Egle Sorotos]

I'm facing similar issues - gonna try this instead. 

[Facundo Calabró]

Very hard to understand why this got deprecated. I mean yes transformers have their benefits, but fasttext is still pretty good, faster to train and faster for inference, and less data hungry. It has lots of upsides for industrial applications

[Juho Inkinen]

Floret by Explosion (the company behind SpaCy) is a fastText fork that seems somewhat maintained, but cannot know about its future either.

-Juho

[Ming Zhang]

I fixed the issue, checkout this link: https://github.com/zmeet-ai/fasttext

[Ray Pereda]

I tried Ming's fork of fasttext to install the python package. Here are the commands:

git clone https://github.com/zmeet-ai/fastText.git

cd fatText

pip3 install .

That gave this error:

 % sudo pip3 install .
Processing /home/ray/code/fasttext2/fastText
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
 
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [10 lines of output]
      /usr/lib/python3/dist-packages/setuptools/dist.py:723: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
        warnings.warn(
      running egg_info
      creating /tmp/pip-pip-egg-info-ulgumhpx/fasttext.egg-info
      writing /tmp/pip-pip-egg-info-ulgumhpx/fasttext.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-pip-egg-info-ulgumhpx/fasttext.egg-info/dependency_links.txt
      writing requirements to /tmp/pip-pip-egg-info-ulgumhpx/fasttext.egg-info/requires.txt
      writing top-level names to /tmp/pip-pip-egg-info-ulgumhpx/fasttext.egg-info/top_level.txt
      writing manifest file '/tmp/pip-pip-egg-info-ulgumhpx/fasttext.egg-info/SOURCES.txt'
      error: package directory 'python/fasttext_module/fasttext' does not exist
      [end of output]
 
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.Lin
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Here are the system info:

OS: Linux Mint Cinnamon

Cinnamon 5.4.12

RAM 15.4G

Hints and consolation are welcomed and appreciate.

[Ray Pereda]

See this alternative package:

https://pypi.org/project/fasttext-numpy2/

Using fasttext-numpy2 fixed problems with NumPy 2.0 breaking the python binding.

pip install fasttext-numpy2

Tested predictions insider of python, which used to break.

Ray