Re: [PyInstaller] Problem with Pyinstaller and Scikit-learn

[Martin Zibricky]

jonathan...@coveros.com píše v Út 05. 03. 2013 v 18:01 -0800:
> python pyinstaller.py --onefile --debug path\to\code
> \deployed-driver.py

> Can anyone suggest a reason why my executable is not running properly
> in the virtual machine? Let me know if you need any additional
> information to help identify the problem.

First try --onedir mode and ensure all files are bundled properly.

[jonathan...@coveros.com]

Thanks for the quick response.

I tried --onedir mode as you suggested for both my complete program and the minimal example that reproduces the error message.  After creating the executable in --onedir mode for the complete program I received the following error when running the executable:

LOADER: executable is C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\deployed-driver.exe

LOADER: homepath is C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1

LOADER: _MEIPASS2 is NULL

LOADER: archivename is C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\deployed-driver.exe

LOADER: No need to extract files to run; setting extractionpath to homepath

LOADER: Already in the child - running user's code.

LOADER: manifestpath: C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\deployed-driver.exe.man

ifest

LOADER: Activation context created

LOADER: Activation context activated

LOADER: Python library: C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1\python27.dll

LOADER: Manipulating evironment

LOADER: PYTHONPATH=C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1

LOADER: PYTHONHOME=C:\PYINST~1\DEPLOY~1\dist\DEPLOY~1

LOADER: Manipulating Python's sys.path

LOADER: importing modules from CArchive

LOADER: extracted struct

LOADER: extracted pyi_os_path

LOADER: extracted pyi_archive

LOADER: extracted pyi_importers

LOADER: Installing import hooks

LOADER: out00-PYZ.pyz

LOADER: Running scripts

Traceback (most recent call last):

  File "<string>", line 24, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\Detectors.MultiDetector", line 6, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.svm", line 13, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.svm.classes", line 3, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.linear_model", line 15, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.linear_model.least_angle", line 24, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.cross_validation", line 25, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.metrics", line 32, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.metrics.scorer", line 25, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.metrics.cluster", line 8, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 268, i

n load_module

    exec(bytecode, module.__dict__)

  File "C:\pyinstaller-develop\deployed-driver\build\pyi.win32\deployed-driver\o

ut00-PYZ.pyz\sklearn.metrics.cluster.supervised", line 19, in <module>

  File "C:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 404, i

n load_module

    module = imp.load_module(fullname, fp, filename, self._c_ext_tuple)

  File "expected_mutual_info_fast.pyx", line 1, in init sklearn.metrics.cluster.

expected_mutual_info_fast (sklearn\metrics\cluster\expected_mutual_info_fast.c:4

869)

ImportError: No module named lgamma

LOADER: RC: -1 from deployed-driver

LOADER: OK.

LOADER: Deactivating activation context

LOADER: Releasing activation context

LOADER: Done

LOADER: Cleaning up Python interpreter.

I was able to resolve this by adding an explicit import for lgamma to deployed-driver.py.  However, when I transferred the directory produced by Pyinstaller to my virtual machine and attempted to run the executable, I received the same error message that was included in my original post.

I also ran Pyinstaller in --onedir mode for my minimal example and did not have any difficulty running the executable in my build environment, but I received the same WindowsError after transferring the directory produced by Pyinstaller to the virtual machine and attempting to run the executable.

Here are the contents of the dist\ directory:

 Directory of C:\pyinstaller-develop\SklearnMinimalExample\dist\SklearnMinimalEx

ample

03/06/2013  07:37 AM    <DIR>          .

03/06/2013  07:37 AM    <DIR>          ..

04/10/2012  11:31 PM            59,904 bz2.pyd

03/06/2013  07:37 AM    <DIR>          include

03/01/2013  08:28 AM             1,857 Microsoft.VC90.CRT.manifest

03/01/2013  08:28 AM           224,768 msvcm90.dll

03/01/2013  08:28 AM           568,832 msvcp90.dll

03/01/2013  08:28 AM           655,872 msvcr90.dll

02/09/2013  06:05 PM         1,313,415 numpy.core.multiarray.pyd

02/09/2013  06:05 PM           174,793 numpy.core.scalarmath.pyd

02/09/2013  06:05 PM           410,944 numpy.core.umath.pyd

02/09/2013  06:05 PM         2,222,455 numpy.core._dotblas.pyd

02/09/2013  06:06 PM            46,383 numpy.fft.fftpack_lite.pyd

02/09/2013  06:06 PM            41,019 numpy.lib._compiled_base.pyd

02/09/2013  06:06 PM         2,382,083 numpy.linalg.lapack_lite.pyd

02/09/2013  06:06 PM           521,885 numpy.random.mtrand.pyd

04/10/2012  11:31 PM           103,424 pyexpat.pyd

04/10/2012  11:31 PM         2,303,488 python27.dll

10/27/2012  09:20 AM           110,080 pywintypes27.dll

02/16/2013  08:55 AM           180,355 scipy.io.matlab.mio5_utils.pyd

02/16/2013  08:55 AM            48,516 scipy.io.matlab.mio_utils.pyd

02/16/2013  08:55 AM            75,662 scipy.io.matlab.streams.pyd

02/16/2013  08:56 AM           108,705 scipy.linalg.calc_lwork.pyd

02/16/2013  08:56 AM            59,065 scipy.linalg._cblas.pyd

02/16/2013  08:56 AM         2,485,538 scipy.linalg._clapack.pyd

02/16/2013  08:56 AM         2,720,377 scipy.linalg._fblas.pyd

02/16/2013  08:56 AM         6,897,971 scipy.linalg._flapack.pyd

02/16/2013  08:56 AM         2,208,558 scipy.linalg._flinalg.pyd

02/16/2013  09:01 AM            89,564 scipy.sparse.csgraph._min_spanning_tree.p

yd

02/16/2013  09:01 AM           233,527 scipy.sparse.csgraph._shortest_path.pyd

02/16/2013  09:01 AM           145,501 scipy.sparse.csgraph._tools.pyd

02/16/2013  09:01 AM           168,478 scipy.sparse.csgraph._traversal.pyd

02/16/2013  08:57 AM         2,822,178 scipy.sparse.linalg.dsolve._superlu.pyd

02/16/2013  08:58 AM         4,012,276 scipy.sparse.linalg.eigen.arpack._arpack.

pyd

02/16/2013  08:57 AM           928,767 scipy.sparse.linalg.isolve._iterative.pyd

02/16/2013  09:00 AM         1,641,587 scipy.sparse.sparsetools._bsr.pyd

02/16/2013  08:59 AM           675,572 scipy.sparse.sparsetools._coo.pyd

02/16/2013  08:59 AM         1,131,421 scipy.sparse.sparsetools._csc.pyd

02/16/2013  09:00 AM           538,950 scipy.sparse.sparsetools._csgraph.pyd

02/16/2013  08:59 AM         1,503,879 scipy.sparse.sparsetools._csr.pyd

02/16/2013  09:00 AM           571,548 scipy.sparse.sparsetools._dia.pyd

02/16/2013  09:01 AM           229,202 scipy.spatial.ckdtree.pyd

02/16/2013  09:01 AM         1,315,873 scipy.spatial.qhull.pyd

02/16/2013  09:01 AM            46,248 scipy.spatial._distance_wrap.pyd

02/16/2013  09:02 AM         1,483,700 scipy.special.specfun.pyd

02/16/2013  09:02 AM         2,551,131 scipy.special._ufuncs.pyd

02/16/2013  09:02 AM           137,879 scipy.special._ufuncs_cxx.pyd

04/10/2012  11:31 PM             9,728 select.pyd

03/05/2013  08:26 PM            55,808 sklearn.metrics.cluster.expected_mutual_i

nfo_fast.pyd

03/05/2013  08:25 PM           131,072 sklearn.metrics.pairwise_fast.pyd

03/05/2013  08:25 PM            14,336 sklearn.utils.lgamma.pyd

03/05/2013  08:25 PM            56,832 sklearn.utils.murmurhash.pyd

03/05/2013  08:25 PM            66,048 sklearn.utils.sparsefuncs.pyd

03/05/2013  08:25 PM            37,888 sklearn.utils.weight_vector.pyd

03/05/2013  08:24 PM            10,752 sklearn.__check_build._check_build.pyd

03/06/2013  07:37 AM         3,287,022 SklearnMinimalExample.exe

03/06/2013  07:37 AM               488 SklearnMinimalExample.exe.manifest

04/10/2012  11:31 PM           686,592 unicodedata.pyd

10/27/2012  09:21 AM            98,816 win32api.pyd

10/27/2012  09:20 AM            25,600 win32pdh.pyd

10/27/2012  09:20 AM            24,064 win32pipe.pyd

04/10/2012  11:31 PM            74,240 _ctypes.pyd

04/10/2012  11:31 PM           285,184 _hashlib.pyd

04/10/2012  11:31 PM            23,552 _multiprocessing.pyd

04/10/2012  11:31 PM            40,960 _socket.pyd

04/10/2012  11:31 PM           721,920 _ssl.pyd

              63 File(s)     51,804,132 bytes

 Directory of C:\pyinstaller-develop\SklearnMinimalExample\dist\SklearnMinimalEx

ample\include

03/06/2013  07:37 AM    <DIR>          .

03/06/2013  07:37 AM    <DIR>          ..

04/10/2012  11:18 PM            21,529 pyconfig.h

               1 File(s)         21,529 bytes

     Total Files Listed:

              64 File(s)     51,825,661 bytes

               5 Dir(s)  101,922,615,296 bytes free

As far as I can tell there aren't any missing files.

[Martin Zibricky]

jonathan...@coveros.com píše v Út 05. 03. 2013 v 18:01 -0800:

> ut00-PYZ.pyz\sklearn.__check_build", line 26, in raise_build_error
> WindowsError: [Error 3] The system cannot find the path specified: 'C:
> \\Users\\C LEANI~1\\AppData\\Local\\Temp\\_MEI24282\\sklearn\
> \__check_build/*.*'

This is definitely an issue with scikit-learn. It expects some
directories to exist.

Look at file ./sklearn/__check_build/__init__.py and other files there.

scikit-learn package is trying to do 'os.listdir()' for
directory .\sklearn\__check_build that does not exist. However, it does
this already when trying to raise an exception. You could closer at how
sklearn is trying to verify the build!

You could try to create this directory in --onedir mode and if necessary
put there some files.