ModuleNotFoundError: No module named 'numpy.f2py'

[Debabrata Sarkar]

Hi,

After the numpy 2.0.0 release,

Warning

Changed in version 2.0.0: There used to be a f2py.compilefunction, which was removed, users may wrap python -m numpy.f2pyvia subprocess.run manually, and set environment variables to interact with meson as required.

my pyinstaller-compiled onedir executable (in Mac OSX Sonoma/ M2 Max) is throwing the following errors while running it on the terminal

Traceback (most recent call last):
  File "pyprog.py", line 47, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/compat/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/tools/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/tools/tools.py", line 6, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/linalg/__init__.py", line 205, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/linalg/_basic.py", line 13, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/linalg/_decomp.py", line 24, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/_lib/_util.py", line 18, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/_lib/_array_api.py", line 21, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/_lib/array_api_compat/numpy/__init__.py", line 1, in <module>
  File "numpy/__init__.py", line 358, in __getattr__
ModuleNotFoundError: No module named 'numpy.f2py'
[4897] Failed to execute script 'pyprog' due to unhandled exception!

Any clue to resolving it?

Regards,

Debabrata

[bwoodsend]

Just add --collect-submodules=numpy.f2py to your pyinstaller command

​

[Debabrata Sarkar]

Hi Brenainn,

Many a thanks for your prompt response. But now it's throwing 'ModuleNotFoundError: No module named 'scipy._lib.array_api_compat.numpy.fft' error after getting the script frozen.

Traceback (most recent call last):
  File "pyprog.py", line 47, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/compat/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/tools/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "statsmodels/tools/tools.py", line 6, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/linalg/__init__.py", line 205, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/linalg/_basic.py", line 13, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/linalg/_decomp.py", line 24, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/_lib/_util.py", line 18, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "scipy/_lib/_array_api.py", line 21, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module

  File "scipy/_lib/array_api_compat/numpy/__init__.py", line 18, in <module>
ModuleNotFoundError: No module named 'scipy._lib.array_api_compat.numpy.fft'
[2941] Failed to execute script 'pyprog' due to unhandled exception!

Regards,

Debabrata

[bwoodsend]

Then add --hiddenimport=scipy._lib.array_api_compat.numpy.fft too

​

[Kergan Sanderson]

Joined this Google Group so that I could confirm that bwoodsend's solution works.

I was using PyInstaller for my application and I got the exact same ModuleNotFoundError that Debabrata had. Using the --collect-submodules= for each missing module like bwoodsend recommended eventually led to a successful executable.

I'm using python 3.12.3. These are the three (luckily, it was only three and not more) modules that I had to use --collect-submodules= with:

- numpy.f2py

- scipy._lib.array_api_compat.numpy.fft

- scipy.special._special_ufuncs

I found these three by adding them to my command one at a time and then checking the crash message for the next module to collect.

My command to run pyinstaller that worked (with my venv activated):

pyinstaller --onefile --noconsole --name=EXECUTABLE_NAME --collect-submodules=numpy.f2py --collect-submodules=scipy._lib.array_api_compat.numpy.fft -collect-submodules=scipy.special._special_ufuncs PATH_OF_MAIN_FILE.py

Out of curiosity, I used a separate venv built with python 3.9.6 and discovered that I only needed to use --collect-submodules= for numpy.f2py. In both instances, I'm using the latest version (as of the day of this post) of all my packages. PyInstaller could have some strange interactions with python 3.12.3 that I'm unaware of.

I'd be surprised if there was not a faster way to include missing submodules, but this one-by-one method worked for me.

On Friday, June 28, 2024 at 5:27:03 AM UTC-4 bwoodsend wrote:

Then add --hiddenimport=scipy._lib.array_api_compat.numpy.fft too

​

[Kergan Sanderson]

My previous post was all in the context of macOS. I ran PyInstaller on my Windows machine that DID NOT have the latest versions of SciPy or Numpy and the executable ran fine.

When I reinstalled my venv on my Windows machine to update to the newest versions of all my packages (including numpy 2.0.0 and the latest scipy), pyinstaller ran, but the executable didn't work. It gave me the same errors as I had on my mac. By running the same PyInstaller command that I had used on my mac (with the / replaced by \ because of Windows), PyInstaller worked, but the executable still had an error.

It seems that Numpy 2.0.0 and/or SciPy 1.14.0 both are slightly problematic with PyInstaller. Both had new versions released within the past month, which is likely why Debabrata and I have the same issue at the same time.

MY RECOMMENDATION: use NumPy v1.26.4 and SciPy v1.13.1 instead of the latest releases. After I uninstalled the latest and installed these two, my Windows and macOS PyInstallers were able to work with no extra --collect-submodules= pieces in the command.

[Debabrata Sarkar]

Yes, Kregan. I did solve the issue per  Brenainn's successive recommendations—the first one (numpy.f2py) with the --collect-submodules=, and the other ones with the --hiddenimport=, though I found that all the three could be collected with the --collect-submodules=. By the way, installing pytest in the venv aids in properly collecting the numpy.f2py as pyinstaller recommends. Thanks and regards, Debabrata

[Debabrata Sarkar]

Hi Brenainn,

Oh! You are great, and the pyinstaller too. Resolved!! Below, please find my freezing command (on OSX terminal) in case someone—facing the same issue after numpy 2.0.0 release—finds it useful.

$ pyinstaller pyprog.py --collect-submodules=numpy.f2py --hiddenimport=scipy._lib.array_api_compat.numpy.fft --hiddenimport=scipy.special._special_ufuncs --name pyprog --onedir --noconfirm --clean

Kindest regards,

Debabrata

On Friday, June 28, 2024 at 2:57:03 PM UTC+5:30 bwoodsend wrote:

Then add --hiddenimport=scipy._lib.array_api_compat.numpy.fft too

​