pyInstaller packaging pytables and Cython

[Bogdan.Neacsa]

Hello,

I've been using pyinstaller to package quite a large program with
numerous dependencies (numpy, scipy, cherrypy, simplejson, sqlalchemy
and more). With some tweaks in the spec file I managed to get it
working. However now I need to integrate pytables, which in turn comes
with Cython as a dependency. I've tried but have no luck whatsoever, I
always get the error:

Traceback (most recent call last):
File "/home/tester/TVB/pyinstaller-1.5.1/Build.py", line 1494, in
<module>
main(args[0], configfilename=opts.configfile)
File "/home/tester/TVB/pyinstaller-1.5.1/Build.py", line 1472, in
main
build(specfile)
File "/home/tester/TVB/pyinstaller-1.5.1/Build.py", line 1429, in
build
execfile(spec)
File "setup_linux.py", line 88, in <module>
pathex=['/home/tester/TVB/pyinstaller-1.5.1'])
File "/home/tester/TVB/pyinstaller-1.5.1/Build.py", line 347, in
__init__
self.__postinit__()
File "/home/tester/TVB/pyinstaller-1.5.1/Build.py", line 298, in
__postinit__
self.assemble()
File "/home/tester/TVB/pyinstaller-1.5.1/Build.py", line 416, in
assemble
analyzer.analyze_script(script)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 572, in
analyze_script
return self.analyze_r('__main__')
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 466, in
analyze_r
newnms = self.analyze_one(name, nm, imptyp, level)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 525, in
analyze_one
mod = self.doimport(nm, ctx, fqname)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 587, in
doimport
mod = parent.doimport(nm)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 761, in
doimport
mod = self.subimporter.getmod(nm)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 355, in getmod
mod = owner.getmod(nm)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 148, in getmod
mod = self._modclass()(nm, pth, co)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 725, in
__init__
self.scancode()
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 728, in
scancode
self.imports, self.warnings, self.binaries, allnms =
scan_code(self.co)
File "/home/tester/TVB/pyinstaller-1.5.1/mf.py", line 966, in
scan_code
all.extend(all_nested)
AttributeError: 'NoneType' object has no attribute 'extend'

Now I've tried to go into the code and see if I can figure out what
the problem is but no chance. However I did find that there is a
#FIXME there saying:

# FIXME: "all" was not updated here nor returned. Was it
the desired
# behaviour?

Now is this some kind of problem that will be fixed? Better has anyone
manage to package a project with pytables/cython dependencies and if
so, how?

regards,
Bogdan

[Martin Zibricky]

Bogdan.Neacsa píše v Po 19. 03. 2012 v 07:19 -0700:

> Now is this some kind of problem that will be fixed? Better has anyone
> manage to package a project with pytables/cython dependencies and if
> so, how?

I don't know about any cython project.
But could you please post any code example to be able to reproduce this
error?

[Bogdan.Neacsa]

I'm not sure exactly what code should I post. The spec file is kind of complex itself, I guess I could atach it here. The problem should reproduce if you try to package any project that has a `from tables import *` or `import tables`. `tables` itself has cython as a dependency and at least to me that seems to cause the problem I posted.

So far I just excluded cython from the included packages and that seemed to fix problem for the moment.

[sgarcia]

Hi list,
I am have the same problem avirl 2012.
This code do not work with pyinstaller (with simple way) unless you
comment import tables.


#####mainwindow.py:

from PyQt4.QtCore import *
from PyQt4.QtGui import *
import tables

class MainWindow(QMainWindow) :
def __init__(self, parent = None,):
QMainWindow.__init__(self, parent)
self.mainWidget = QLabel('yep')
self.setCentralWidget(self.mainWidget)

def startMain():
app = QApplication([ ])
w = MainWindow()
w.show()
app.exec_()

if __name__ =='__main__':
startMain()

######


Does someone could help me ?

Thanks a lot for pyinstaller this is really a great great tool!!!

Samuel

On 19 mar, 22:39, Martin Zibricky <mzibr.pub...@gmail.com> wrote:
> Bogdan.Neacsa píše v Po 19. 03. 2012 v 07:19 -0700:
>
> > Now is this some kind of problem that will be fixed? Better has anyone

> > manage to package a project withpytables/cython dependencies and if

[Samuel Garcia]

Sorry some additional informations:
WinXP 32
Python XY distribution
pyinstaller 1.5.1
tables 2.3.1

Samuel

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Samuel Garcia
Lyon Neuroscience
CNRS - UMR5292 - INSERM U1028 - Universite Claude Bernard LYON 1
Equipe R et D
50, avenue Tony Garnier
69366 LYON Cedex 07
FRANCE
Tél : 04 37 28 74 24
Fax : 04 37 28 76 01
http://olfac.univ-lyon1.fr/unite/equipe-07/
http://neuralensemble.org/trac/OpenElectrophy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Martin Zibricky]

Hi Samuel,

I would try look at the cython source code and look for imported
modules. These modules are not resolved by pyinstaller. Then try add
these modules to the list of hiddenimports in file

PyInstaller/hooks/hook-tables.py

Martin

sgarcia píše v Čt 26. 04. 2012 v 05:24 -0700:

[Samuel Garcia]

Hi,
thanks for helping.
My concern is tables only (but maybe also Cython)

What I understand properly.
tables do some hidden import and actual (in pyinstaler 1.5.1)

PyInstaller/hooks/hook-tables.py

do not resolve them.

So what I guess, is that I need to inspect tables code and find out
which hidden import is done.

Is there a tool to do that ?


I figure out that I also experience problem with :
import pandas



So each tricky package need hook-XXXXXX.py ?


Sorry for that newbie questions.


Thank you again for this great tools.


Samuel

[Martin Zibricky]

Samuel Garcia píše v Čt 26. 04. 2012 v 15:23 +0200:

> Hi,
> thanks for helping.
> My concern is tables only (but maybe also Cython)

Cython is a language derived from python to write native python
extensions easily.

>
> What I understand properly.
> tables do some hidden import and actual (in pyinstaler 1.5.1)

Please use development version

>
> PyInstaller/hooks/hook-tables.py
>
> do not resolve them.
>
> So what I guess, is that I need to inspect tables code and find out
> which hidden import is done.
>
> Is there a tool to do that ?

Just download the source code of PyTables and look especially for
imports in cython files (files ending with .pyx suffix).

there is file ./utils/ArchiveViewer.py to inspect created executables.

>
>
> I figure out that I also experience problem with :
> import pandas
>
>
>
> So each tricky package need hook-XXXXXX.py ?

yes, each tricky package. 'hiddenimports' is the first step if something
is not included.

[Samuel Garcia]

Dev version of PyInstaller fix the problem I think sorry for disturbances.


Does .pyd file are concerned by theses hidden ?

Samuel

[Martin Zibricky]

Samuel Garcia píše v Čt 26. 04. 2012 v 17:02 +0200:

>
> Does .pyd file are concerned by theses hidden ?
>
> Samuel

Yes, .pyd suffix is on windows for native python extensions. In the case
of cython .pyx files are compiled to .pyd files on windows.