Enthought ETS based app loads&exits immediately
jrasinma
I have an application written on top of Enthought Tool Suite (Traits,
TraitsGUI, Chaco, wx etc.). Roughly a year a go a colleague of mine
was able to get it packaged for Windows and Linux with pyinstaller
with some considerable effort to get all the ETS modules imported
nicely.
I now tried to create an OS X app with pyinstaller, and just ran the
stock Makespec and Build commands. Things have seemingly progressed
very nicely for pyinstaller, since I get no import errors:
$ ./yasso
thisfile is ./yasso
_MEIPASS2 (workpath) is NULL
homepath is ./
Extracting binaries
Executing self as child with ./
./:/usr/local/lib:/Users/Jussi/instantclient:
thisfile is ./yasso
_MEIPASS2 (workpath) is ./
homepath is ./
Already have a workpath - running!
./Python
Manipulating evironment
PYTHONPATH=.
importing modules from CArchive
extracted iu
extracted struct
extracted archive
Installing import hooks
outPYZ1.pyz
Running scripts
Back to parent...
BUT the script exits without actuallly opening the UI. I'm running
this using the 32 bit Enthought Python Distribution from a virtualenv
in which I renamed pythonw to python since pythonw is needed to open
the GUI for the ETS app.
python yasso.py does work ok using the virtualenv for the app.
Any ideas whats going wrong or how could I get more debug information?
Cheers,
Jussi
Antonio Valentino
Il giorno Mon, 19 Jul 2010 07:47:36 -0700 (PDT)
jrasinma <jussi.r...@simosol.fi> ha scritto:
> Hi,
>
> I have an application written on top of Enthought Tool Suite (Traits,
> TraitsGUI, Chaco, wx etc.). Roughly a year a go a colleague of mine
> was able to get it packaged for Windows and Linux with pyinstaller
> with some considerable effort to get all the ETS modules imported
> nicely.
Yes, in my experience packging an ETS app with pyinstaller is but an
easy task. I'm pretty sure it can't be done with a "stock Makespec and
Build commands".
One of the problems is that ETS heavily uses pkg_resources, that is not
currently supported by pyinstaller, an has a lot o hidden imports.
> I now tried to create an OS X app with pyinstaller, and just ran the
> stock Makespec and Build commands. Things have seemingly progressed
> very nicely for pyinstaller, since I get no import errors:
[...]
> BUT the script exits without actuallly opening the UI. I'm running
The graphical toolkit is imported only when actually needed and this
import is quite hard to detect for pyinstaller.
> this using the 32 bit Enthought Python Distribution from a virtualenv
> in which I renamed pythonw to python since pythonw is needed to open
> the GUI for the ETS app.
> python yasso.py does work ok using the virtualenv for the app.
>
> Any ideas whats going wrong or how could I get more debug information?
>
> Cheers,
> Jussi
Well, it would be interesting to take a look to the scripts your
colleague used for windows and linux :).
Some related thread:
https://mail.enthought.com/pipermail/enthought-dev/2009-September/024263.html
http://groups.google.it/group/pyinstaller/browse_thread/thread/301fcaddbbb7f13f
best regards
--
Antonio Valentino
Jussi Rasinmäki
> Yes, in my experience packging an ETS app with pyinstaller is but an
> easy task. I'm pretty sure it can't be done with a "stock Makespec and
> Build commands".
I think the changes in ETS related to namespace package imports
referenced in the thread you linked to have helped in this. At least
pyinstaller seemed to happily import ETS packages when I enabled debug
messages in pyinstaller's iu.py. Or then I interpreted the debug
messages incorrectly...
The work by my colleague I referred to was mainly about this stuff,
how to collect needed ETS packages for pyinstaller. So it would seem
that it's not needed anymore. However, it went roughly like this:
import enthought
base = os.path.abspath(os.path.join(os.path.dirname(enthought.__file__),
'..', '..'))
...
for item in os.listdir(base):
if item.lower().startswith('traits-'):
traits = os.path.join(base, item)
print 'added traits from', item
... etc for the other needed ETS packages used with PKG below
basefiles = [os.path.join(HOMEPATH,'support/_mountzlib.py'),
os.path.join(HOMEPATH,'support/useUnicode.py'),
os.path.join(base, 'pkg_resources.py'),
'unpackMetadata.py',
'../yasso.py',
'../modelcall.py',
]
a = Analysis(basefiles,
pathex=[''],
hookspath=[''],
excludes=['enthought', 'wx'])
pyz = PYZ(a.pure)
wx = PKG(Tree(wx), name='wx.pkg')
traits = PKG(Tree(traits), name='traits.pkg')
traitsgui = PKG(Tree(traitsgui), name='traitsgui.pkg')
traitswx = PKG(Tree(traitswx), name='traitswx.pkg')
enable = PKG(Tree(enable), name='enable.pkg')
enthought = PKG(Tree(enthought), name='enthought.pkg')
chaco = PKG(Tree(chaco), name='chaco.pkg')
exe = EXE(wx,
traitswx,
traitsgui,
traits,
enable,
enthought,
chaco,
pyz,
a.scripts, #+[("v", "", "OPTION")],
a.binaries+libfiles,
a.zipfiles,
name=os.path.join('dist', exename),
debug= False,
strip=False,
upx=False,
icon='yasso.ico',
console=False )
There was also a hook-enthought.py:
hiddenimports = ['uuid', 'colorsys', 'wx', 'wx.html']
--------------------------
Then at runtime the unpackMetadata.py was the key:
import carchive
import sys
import os
this = carchive.CArchive(sys.executable)
archives = os.environ['_MEIPASS2']
pkgs = ['wx', 'traits', 'traitswx', 'chaco', 'enthought', 'enable', 'traitsgui']
for pkg in pkgs:
mp = this.openEmbedded('%s.pkg' % pkg)
targetdir = os.path.join(archives,pkg)
os.mkdir(targetdir)
print "Extracting %s ..." % pkg,
for fnm in mp.contents():
try:
stuff = mp.extract(fnm)[1]
outnm = os.path.join(targetdir, fnm)
dirnm = os.path.dirname(outnm)
if not os.path.exists(dirnm):
os.makedirs(dirnm)
open(outnm, 'wb').write(stuff)
except Exception, mex:
print mex
mp = None
sys.path.insert(0, targetdir)
if pkg == 'traitsgui':
imagepath = os.path.join(targetdir, 'enthought', 'traits',
'ui', 'image', 'library')
os.environ['TRAITS_IMAGES'] = imagepath
os.putenv('TRAITS_IMAGES', imagepath)
print "ok"
from resource_path_override import ResourcePathOverride
resource_override = ResourcePathOverride(archives)
-------------------------------------
And then the custom ResourcePathOverride:
import sys
import os
"""This class will extend the original resource_path function so that it
will always point to the archive_path instead of whatever was frozen into it.
WARNING: Do not create an instance of this class before the archives
have been extracted!
"""
class ResourcePathOverride(object):
resource_path_orig = None
archive_path = None
def __init__(self, archive_path):
import enthought.resource.resource_path #here, so that you can
import the module freely (at least in theory)
self.resource_path_orig = enthought.resource.resource_path.resource_path
self.archive_path = archive_path
enthought.resource.resource_path.resource_path = self.resource_path
def resource_path(self, level = 2):
path = self.resource_path_orig(level+1)
path = path.rsplit(''.join((os.path.sep, 'enthought', os.path.sep)), 1)
root = path[0].rsplit(os.path.sep, 1)
path = os.path.join(self.archive_path,
self.get_pkg_name(root[1]), 'enthought', path[1])
return path
def get_pkg_name(self, path):
"""This function must mirror the list in yasso.spec
"""
if path.lower().startswith('traits-'):
return 'traits'
if path.lower().startswith('traitsgui'):
return 'traitsgui'
if path.lower().startswith('traitsbackendwx'):
return 'traitswx'
if path.lower().startswith('enable'):
return 'enable'
if path.lower().startswith('enthought'):
return 'enthought'
if path.lower().startswith('chaco'):
return 'chaco'
> The graphical toolkit is imported only when actually needed and this
> import is quite hard to detect for pyinstaller.
Any ideas how to force this? The GUI is invoked using configure_traits:
...
yasso = Yasso()
if __name__ == '__main__':
yasso.configure_traits()
Jussi
Antonio Valentino
Il giorno Tue, 20 Jul 2010 10:49:51 +0300
Jussi Rasinmäki <jussi.r...@simosol.fi> ha scritto:
> Hi Antonio,
>
> > Yes, in my experience packging an ETS app with pyinstaller is but an
> > easy task. I'm pretty sure it can't be done with a "stock Makespec
> > and Build commands".
>
> I think the changes in ETS related to namespace package imports
> referenced in the thread you linked to have helped in this. At least
> pyinstaller seemed to happily import ETS packages when I enabled debug
> messages in pyinstaller's iu.py. Or then I interpreted the debug
> messages incorrectly...
Yes, pyinstaller correctly imports ets packages like any other python
program does.
The problem is that pynstaller seems to be not able to collect all
the stuffs needed by the frozen environment so you need a hook or
some other trick in the spec file.
> The work by my colleague I referred to was mainly about this stuff,
> how to collect needed ETS packages for pyinstaller. So it would seem
> that it's not needed anymore. However, it went roughly like this:
I disagree.
I still had no time to give your colleague's code a try but I think
you can't get a working app without it.
Exclude enthought from the analysis stage and manually set all needed
packages. Including the entire package tree you automatically get code
for both wx and qt backends. The package gets a little fatter but it is
OK.
> exe = EXE(wx,
> traitswx,
> traitsgui,
> traits,
> enable,
> enthought,
> chaco,
> pyz,
> a.scripts, #+[("v", "", "OPTION")],
> a.binaries+libfiles,
> a.zipfiles,
> name=os.path.join('dist', exename),
> debug= False,
> strip=False,
> upx=False,
> icon='yasso.ico',
> console=False )
>
> There was also a hook-enthought.py:
> hiddenimports = ['uuid', 'colorsys', 'wx', 'wx.html']
OK.
But if one wants to use the qt backend the last two items should be
changed accordingly.
This part is black magic.
I wonder what Giovanni thinks about this solution and, most of all, if
there are plans for an official pkg_resources support in pyinstaller.
> > The graphical toolkit is imported only when actually needed and this
> > import is quite hard to detect for pyinstaller.
>
> Any ideas how to force this? The GUI is invoked using
> configure_traits: ...
> yasso = Yasso()
>
> if __name__ == '__main__':
> yasso.configure_traits()
>
> Jussi
>
Yes, at this point ets looks for available backend and loads it before
actually setting up the GUI.
I you use a stock spec file you are not able to collect all needed
stuffs so the GUI is never painted.
ciao
--
Antonio Valentino
Giovanni Bajo
The solution looks like it is on the right track, though it should be
generalized before being acceptable for PyInstaller.
The generic plan for pkg_resources I had in mind was to leverage its
hook mechanism to install a plugin into it to be able to dynamically
extract files from PyInstaller's pkg archive. This is just a generic
idea, and I don't know if it would really work.
--
Giovanni Bajo :: ra...@develer.com
Develer S.r.l. :: http://www.develer.com
My Blog: http://giovanni.bajo.it
Last post: Compile-time Function Execution in D