What's up with this "Cannot import name <blah>" ?

[dbv]

Environment: virtualenv 1.9.1 under Ubuntu 12.04 with Python 2.7.3, Numpy 1.7.0, Cython 1.8 and PyInstaller 2.0.

One of the project programs has the statement:

import random

The PyInstaller (--onedir) generated executable causes the error:

...
  File "/home/ubuntu/Programs/project_name_pi2/build/pyi.linux2/project_name/out00-PYZ.pyz/tempfile", line 34, in <module>
ImportError: cannot import name Random

To account for virtualenv, added both these paths to the .spec pathex:

"/home/ubuntu/ENV/lib/python2.7//", "/usr/lib/python2.7//", ...

The ImportError still occurs.

The warnxet.txt includes:

W: no module named random.Random (top-level import by tempfile)

Created both a hook-random.py and a hook-random.Random.py but that didn't help either.

Why does this occur and is there a solution?  Thx.

[Martin Zibricky]

dbv píše v So 23. 03. 2013 v 11:34 -0700:

> Why does this occur and is there a solution? Thx.

I would advise trying development version.

[dbv]

[dbv]

Am using latest -develop version but same error with virtualenv on both Ubuntu 12.04 and Windows.

[Martin Zibricky]

dbv píše v So 23. 03. 2013 v 11:34 -0700:

> Created both a hook-random.py and a hook-random.Random.py but that
> didn't help either.

This will not help since 'random.Random' is not a module but a class in
the 'random' module.

> Why does this occur and is there a solution? Thx.

You need to provide more information:
- what exact piece of code is failing?
- does it work outside of virtualenv?

[dbv]

Martin

It doesn't work outside of virtualenv either.  I don't have a precise location of where it is failing but the attached might provide a clue.  It is the partial output from running PyInstaller and the Random error may have its origins in the processing of a PyInstaller hook file - but I'm guessing.

[dbv]

I've had some luck resolving the "no module named numpy", "cannot import name <blah>" type of errors.  All cases are exemplified by this example ... The file Python2.7/tempfile.py is attempting to "import random" from the Standard Library but instead is importing the one from Numpy (numpy.random).  Fixes included re-writing how Python imports random, adding hooks and so on.  The two PyInstaller hooks for scipy helped too.  

One set of programs work for both virtualenv and outside of it on linux64 and win32.

I'm now stuck on the second set of programs and the current issue (AttributeError: 'module' object has no attribute 'BufferedIOBase') which is proving difficult to solve.  A Traceback is attached and would appreciate any pointers.  Thx!

[dbv]

The problem has been identified to gzip.py in the standard library that has an "import io" which has been hijacked by the scipy.io (checked with io.__all__).  The numpy module npyio.py calls gzip.  

However, none of my source files perform an "import os" or directly use gzip.  The program that fails is at the statement:

numpy.load(directory_path + "myarray.npy")

numpy.load is in npyio.py and the first statement in that function is "import gzip".

This appears to be a PyInstaller problem as it has not released scipy.io and replaced with the standard library io.

[Martin Zibricky]

dbv píše v Čt 28. 03. 2013 v 02:12 -0700:

What do you suggest to do?

[dbv]

I don't know what to suggest.

Is it a PyInstaller problem?

The project works successfully with the a) Python interpreter and with b) Cython.

[dbv]

It works on win xp.  I wish there were some deep pearls of wisdom to share but I started over again by re-writing the spec file and generating in --onedir mode first.  

I'll provide feedback on the revised documentation soon.

[dbv]

Btw, I'm using the latest pyinstaller-develop version, and have debug=False in the .spec file but the generated executable still outputs debug messages.  Is there something else to do to turn off the messages?

[Martin Zibricky]

dbv píše v So 30. 03. 2013 v 03:52 -0700:

Just tested it with my fork and it works as expected.

Could you provide more defails?

[dbv]

The --debug option is not in the generation statement but it still outputs messages when the executable is run.  Same with --log-level.

What option(s) are you using during generation to suppress debug messages?

[Martin Zibricky]

dbv píše v Út 02. 04. 2013 v 16:19 -0700:

with 'pyinstaller.py --debug' when you run your created executable you
should see debug messages like

LOADER: executable is /path/to/your/exe
LOADER: ...

without '--debug' you should not see them.

Do you mean debug messages starting with 'LOADER: '?

[dbv]

The --onefile executable is created **without** the "--debug" option and when run, sample output is of the form:

# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
import marshal # builtin
import _struct # builtin
import nt # builtin
import imp # builtin
import zlib # builtin
import errno # builtin
import _weakref # builtin
# zipimport: found 78 names in C:\DOCUME~1\mine\LOCALS~1\Temp\_MEI50162\eggs\s
etuptools-0.6c11-py2.7.egg
# zipimport: zlib available
# C:\DOCUME~1\mine\LOCALS~1\Temp\_MEI50162\eggs\setuptools-0.6c11-py2.7.egg\pk
g_resources.pyc has bad mtime
# zipimport: zlib available
import time # builtin
import _sre # builtin
import strop # builtin
import pkg_resources # loaded from Zip C:\DOCUME~1\mine\LOCALS~1\Temp\_MEI5016
2\eggs\setuptools-0.6c11-py2.7.egg\pkg_resources.py
import _codecs # builtin
etc.
etc.

Uhmm!  If PyInstaller debug messages start with "LOADER", wonder what these messages are?  Otherwise, the executable runs successfully.

[Martin Zibricky]

dbv píše v St 03. 04. 2013 v 09:51 -0700:

> Uhmm! If PyInstaller debug messages start with "LOADER", wonder what
> these messages are? Otherwise, the executable runs successfully.

This is not from pyinstaller itself.

Could be there set any python environment variable for debugging?

Could you try package your app without the .spec file?

[dbv]

It was the Python run-time -v option in the .spec file.  Removing the option stopped the run-time messages.

[dbv]

Martin

Continuing to get the "ImportError: cannot import name Random" error in Windows.  The Traceback from running the executable is:

Traceback (most recent call last):
  File "<string>", line 24, in <module>
  File "d:\pyinstaller-develop\PyInstaller\loader\pyi_importers.py", line 273, in load_module
    exec(bytecode, module.__dict__)
  File "d:\...\out00-PYZ.pyz\tempfile", line 35, in <module>

 
ImportError: cannot import name Random

tempfile.py is from the Python27/Lib folder.

The module.__dict__ output in pyi_importers.py at ~line 273 is:

{'__file__': 'C:\\DOCUME~1\\...\\LOCALS~1\\Temp\\_MEI20322\\random\\__init__.pyc', '__loader__': <pyi_importers.FrozenImporter object at 0x00A6F950>, '__path__': ['C:\\DOCUME~1\\...\\LOCALS~1\\Temp\\_MEI20322\\random'], '__name__': 'random', '__package__': 'random', '__doc__': None}

The environment includes Numpy, Scipy and Cython.  The error occurs in both --onedir and --onefile modes.

I'm running out of ideas - any thoughts?

Btw, this error returned after upgrading to the latest pyinstaller-develop to solve the icon issue.  Thx.

[Martin Zibricky]

dbv píše v Po 08. 04. 2013 v 11:58 -0700:

> The environment includes Numpy, Scipy and Cython. The error occurs in
> both --onedir and --onefile modes.
>
> I'm running out of ideas - any thoughts?
>
> Btw, this error returned after upgrading to the latest
> pyinstaller-develop to solve the icon issue. Thx.
>

Could you provide simple example that reproduces your issue?

[dbv]

Attached is set of programs that reproduces the issue.  The folder tto_pi2 contains the .spec and make files.  Hopefully it should make sense.  

[dbv]

Fyi, the current environment is Windows XP (32-bit), python 2.7, numpy 1.7, scipy 0.11, cython 0.18.  

[dbv]

Re-uploaded as plain files.

Put attached files in /tto folder at same level as /pyinstaller-develop folder.  Create .pyd with cythonize.py.  Generate --onedir executable using command line options in 'make.txt' from within /tto folder.  Run executable from /dist.

The sample example is from http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#still-too-slow-preconditioning.

Works in python interpreter; works with pyd files; doesn't work with pyinstaller - see run-time "ImportError cannot import name Random" message. Note that source files do not directly use random or random.Random.

Environment: windows XP/7, python 2.7.4, numpy 1.7.1, scipy 0.12, cython 0.18, pyinstaller 2.0, with and without virtualenv.

[Daniel Hyams]

This is much better, but still not there yet....keep cutting the sample down to the bare minimum needed to reproduce the problem...first get rid of cython, and see if the problem still occurs.  Then cut down other.py down to the minimum.  Maybe find out of the problem reproduces with nothing more than the imports at the top with a 

print "hello from my script"

in the place of all of the computation in other.py?  And if it does, start cutting out all of the imports one by one until you find the one that causes the problem.

http://sscce.org/

--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyinstaller...@googlegroups.com.
To post to this group, send email to pyins...@googlegroups.com.
Visit this group at http://groups.google.com/group/pyinstaller?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Daniel Hyams
dhy...@gmail.com

[dbv]

Thank-you for your ongoing encouragement - it is really appreciated.

The note says, "Works in python interpreter; works with pyd files; doesn't work with pyinstaller - see run-time "ImportError cannot import name Random" message. Note that source files do not directly use random or random.Random." ie. 

a. the project was tested with the Python interpreter and it worked
b. the project was tested with calling just the .pyd (generated with cython) and it worked.
c. the project was tested with pyinstaller using the .pyd and it didn't work.

I have also tested on very little numpy/scipy code and slightly larger (as in the uploaded sample example) and in all cases the resulting error was identical.

I'll be happy to upload the tiny sample example if that makes it easier for you?

[dbv]

Uploaded smaller sample example (http://docs.scipy.org/doc/scipy/reference/sparse.html) using numpy/scipy.

This smaller sample example uses the numpy.random module but the result is the same as the previous sample example which didn't use any random module or function.  As before, this sample example:

a. works with the Python interpreter
b. works when calling just the .pyd (generated with cython)
c. doesn't work with pyinstaller

It appears that the Python//Lib//tempfile.py is called, which imports random but instead of loading the Python//Lib/random module it loads the numpy.random module (which doesn't have a Random). So, somewhere along the line, pyinstaller is not re-setting sys.path.

Anyway, you will probably have a much better explanation and solution and look forward to it.

[Daniel Hyams]

Keep cutting!  Cut until the problem doesn't exhibit any more.  Then back up one step.  Then if there is anything else left to cut that is unrelated, cut that too.

I did just do:

python $my_pyinstaller_dir/pyinstaller.py boot.py

On a linux installation and on a windows installation, and it worked just fine.

I would be willing to bet that your problems are caused by your entries in pathex, in the .spec file.  Why are those there?  Take them out and see if things work; there's no reason that you should have those in there, as far as I know.

--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyinstaller...@googlegroups.com.
To post to this group, send email to pyins...@googlegroups.com.
Visit this group at http://groups.google.com/group/pyinstaller?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Daniel Hyams
dhy...@gmail.com

[Martin Zibricky]

Daniel Hyams píše v Pá 12. 04. 2013 v 21:30 -0400:

>
> I would be willing to bet that your problems are caused by your
> entries in pathex, in the .spec file. Why are those there? Take them
> out and see if things work; there's no reason that you should have
> those in there, as far as I know.
>

In the .spec file you have:

a = Analysis(["boot.py"],
pathex=["C://Python27//Lib//site-packages//",
"C://Python27//Lib//site-packages//numpy//",
"C://Python27//Lib//site-packages//scipy//"],
)

You don't need the "C://Python27//Lib//site-packages//". Unless you use
virtualenv and you haven't enabled virtualenv to use packages from this
directory.

"C://Python27//Lib//site-packages//numpy//",
"C://Python27//Lib//site-packages//scipy//"

You should not use these path since:
- there is a 'random' in the numpy directory
- pyinstaller will use 'random' from numpy when resolving code 'import
random'
- 'random' from numpy is available earlier in the sys.path list
- try to set PYTHONPATH to these values before trying any code example
in the python interpreter and you will get the same error as when
packaged with pyinstaller.

[dbv]

Fixed by removing "C://Python27//Lib//site-packages//" as well as the paths "C://Python27//Lib//site-packages//numpy//",
"C://Python27//Lib//site-packages//scipy//".

Works with and without virtualenv.

Many thanks for spotting that.

During the generation, these errors still appear:

12968 ERROR: lib not found: libifcoremd.dll dependency of C:\bs\lib\site-packages\scipy-0.12.0-py2.7-win32.egg\scipy\linalg\calc_lwork.pyd
13343 ERROR: lib not found: libifcoremd.dll dependency of C:\bs\lib\site-packages\scipy-0.12.0-py2.7-win32.egg\scipy\sparse\linalg\eigen\arpack\_arpack.pyd
13953 ERROR: lib not found: libifcoremd.dll dependency of C:\bs\lib\site-packages\scipy-0.12.0-py2.7-win32.egg\scipy\special\specfun.pyd
14483 ERROR: lib not found: libifcoremd.dll dependency of C:\bs\lib\site-packages\scipy-0.12.0-py2.7-win32.egg\scipy\special\_ufuncs.pyd

But, libifcoremd.dll appears in the /dist list, and the program runs without errors.

[Ernesto Alfonso]

If you have an older version of a module available in the execution environment and it is missing the module in question, you might get this error since the --onefile compiled executable seems to give it priority.

On Saturday, March 23, 2013 2:34:37 PM UTC-4, dbv wrote:

Environment: virtualenv 1.9.1 under Ubuntu 12.04 with Python 2.7.3, Numpy 1.7.0, Cython 1.8 and PyInstaller 2.0.

One of the project programs has the statement:

import random

The PyInstaller (--onedir) generated executable causes the error:

...
  File "/home/ubuntu/Programs/project_name_pi2/build/pyi.linux2/project_name/out00-PYZ.pyz/tempfile", line 34, in <module>

ImportError: cannot import name Random

To account for virtualenv, added both these paths to the .spec pathex:

"/home/ubuntu/ENV/lib/python2.7//", "/usr/lib/python2.7//", ...

The ImportError still occurs.

The warnxet.txt includes:

W: no module named random.Random (top-level import by tempfile)

Created both a hook-random.py and a hook-random.Random.py but that didn't help either.