PyInstaller and Matplotlib???

1,828 views
Skip to first unread message

arrowtackett

unread,
Jun 17, 2008, 11:22:14 PM6/17/08
to PyInstaller
I have been trying to create a working executable of a wx/Matplotlib
application
on Linux for quite a while with no luck. When I try to build my
application with PyInstaller, I always get one of two errors, no
matter what I try. These are the errors that I get

'Could not find the matplotlib data files'

and

File "testPlot/buildtestPlot/outPYZ3.pyz/numpy", line 16, in <module>
AttributeError: 'module' object has no attribute 'version'


Someone over at the wxPython mailing list directed me to their spec
file, but I can't seem to get that to work either. Here is the link
to it:

http://code.google.com/p/devide/source/browse/trunk/devide/installer/devide.spec

I have found the workaround for this on Windows with py2exe. However,
I cannot
seem to figure out how to fix this issue on Linux. I have only
managed to find
one example of how to do this with Pyinstaller on the web, and it
doesn't work. I've pretty much done everything that I know to do, but
still can't get it to work. Has anyone figured this out? If so,
could you please post how to build a very
simple wx/Matplotlib application such as the one found below? I, and
I am sure
several others around here, would be very greatful!


#!/usr/bin/env python

#This code is from the "examples" folder that
#is shipped with Matplotlib

from numpy import arange, sin, pi
import matplotlib
matplotlib.use('WXAgg')
from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as
FigureCanvas
from matplotlib.backends.backend_wx import NavigationToolbar2Wx
from matplotlib.figure import Figure
from wx import *

class CanvasFrame(Frame):
def __init__(self):
Frame.__init__(self,None,-1, 'CanvasFrame',size=(550,350))
self.SetBackgroundColour(NamedColor("WHITE"))
self.figure = Figure()
self.axes = self.figure.add_subplot(111)
t = arange(0.0,3.0,0.01)
s = sin(2*pi*t)
self.axes.plot(t,s)
self.canvas = FigureCanvas(self, -1, self.figure)
self.sizer = BoxSizer(VERTICAL)
self.sizer.Add(self.canvas, 1, LEFT | TOP | GROW)
self.SetSizerAndFit(self.sizer)
self.add_toolbar()

def add_toolbar(self):
self.toolbar = NavigationToolbar2Wx(self.canvas)
self.toolbar.Realize()
if Platform == '__WXMAC__':
self.SetToolBar(self.toolbar)
else:
tw, th = self.toolbar.GetSizeTuple()
fw, fh = self.canvas.GetSizeTuple()
self.toolbar.SetSize(Size(fw, th))
self.sizer.Add(self.toolbar, 0, LEFT | EXPAND)
self.toolbar.update()

def OnPaint(self, event):
self.canvas.draw()

class App(App):
def OnInit(self):
'Create the main window and insert the custom frame'
frame = CanvasFrame()
frame.Show(True)
return True

app = App(0)
app.MainLoop()

Charl Botha

unread,
Jun 18, 2008, 4:28:56 PM6/18/08
to PyIns...@googlegroups.com
On Wed, Jun 18, 2008 at 5:22 AM, arrowtackett <arrowt...@gmail.com> wrote:
> Someone over at the wxPython mailing list directed me to their spec
> file, but I can't seem to get that to work either. Here is the link
> to it:
>
> http://code.google.com/p/devide/source/browse/trunk/devide/installer/devide.spec

I'm that someone. :) I sent two links in my mail, the other was also
crucial to make this work:
http://code.google.com/p/devide/source/browse/trunk/devide/module_kits/matplotlib_kit/__init__.py
- the important part was this:

if hasattr(sys, 'frozen') and sys.frozen:
# matplotlib supports py2exe by checking for matplotlibdata in
the appdir
# but this is only done on windows (and therefore works for our windows
# installer builds). On non-windows, we have to stick it in the env
# to make sure that MPL finds its datadir (only if we're frozen)
mpldir = os.path.join(theModuleManager.get_appdir(), 'matplotlibdata')
os.environ['MATPLOTLIBDATA'] = mpldir

(it's even nicely commented!)

In your application, you need to set an environment variable so that
matplotlib can find its datafile.

Good luck,
Charl

arrowtackett

unread,
Jun 19, 2008, 10:39:40 PM6/19/08
to PyInstaller
Charl:

Thank you so much for taking the time to help me out. I have gotten
my exe to read the matplotlib data files. However, I still get what
seems to be a matplotlib error. Here is the stack trace:

Traceback (most recent call last):
File "<string>", line 130, in <module>
File "testPlot/buildtestPlot/outPYZ4.pyz/wx._core", line 7836, in
__init__
File "testPlot/buildtestPlot/outPYZ4.pyz/wx._core", line 7433, in
_BootstrapApp
File "<string>", line 77, in OnInit
File "<string>", line 33, in __init__
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.figure", line
554, in add_subplot
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axes", line
5557, in __init__
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axes", line 508,
in __init__
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axes", line 546,
in _init_axis
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axis", line 518,
in __init__
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axis", line 553,
in cla
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axis", line
1033, in _get_tick
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axis", line 96,
in __init__
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.axis", line 285,
in _get_tick1line
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.lines", line
284, in __init__
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.lines", line
405, in set_data
File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.lines", line
423, in recache
AttributeError: 'module' object has no attribute 'getmask'

It seems to not like the Figure.add_subplot() call in my GUI. Anyone
have any ideas to fix this problem?

Here is the spec file that I'm using. I basically just copied what I
though were the relevant parts of Charl's spec file.


import os
import fnmatch
import re
import sys

def helper_remove_start(name, remove_names):
"""Helper function used to remove libraries from list.

Returns true of name starts with anything from remove_names.
"""
name = name.lower()
for r in remove_names:
if name.startswith(r.lower()):
return True

return False

def helper_remove_finds(name, remove_finds):
"""Helper function that returns true if any item in remove_finds
(list) can be string-found in name. Everything is lowercased.
"""
name = name.lower()
for r in remove_finds:
if name.find(r.lower()) >= 0:
return True

return False

def helper_remove_regexp(name, remove_regexps):
"""Helper function to remove things from list.

Returns true if name matches any regexp in remove_regexps.
"""

for r in remove_regexps:
if re.match(r, name) is not None:
return True

return False

INSTALLER_DIR = os.path.abspath(os.getcwd())
specpath = INSTALLER_DIR + os.sep + "testPlot"
APP_DIR = os.path.abspath(os.pardir)

from distutils import sysconfig
MPL_DATA_DIR = os.path.join(sysconfig.get_python_lib(), 'matplotlib/
mpl-data')

if sys.platform.startswith('win'):
exeName = 'buildtestPlot/testPlot.exe'

extraLibs = []
# we can keep msvcr71.dll and msvcp71.dll, in fact they should
just
# go in the installation directory with the other DLLs, see:
# http://msdn.microsoft.com/library/default.asp?url=/library/en-us/
# vclib/html/_crt_c_run.2d.time_libraries.asp
remove_binaries = ['dciman32.dll', 'ddraw.dll', 'glu32.dll',
'msvcp60.dll',
'netapi32.dll', 'opengl32.dll', 'uxtheme.dll']

else:
exeName = 'buildtestPlot/testPlot'

# under some linuxes, libpython is shared -- McMillan installer
doesn't
# know about this...
extraLibs = []

# i'm hoping this isn't necessary anymore!
vi = sys.version_info
if (vi[0], vi[1]) == (2,4):
# ubuntu hoary
extraLibs = [('libpython2.4.so.1.0', '/usr/lib/libpython2.4.so.
1.0', 'BINARY')]

elif (vi[0], vi[1]) == (2,2) and os.path.exists('/usr/lib/
libpython2.2.so.0.0'):
# looks like debian woody
extraLibs = [('libpython2.2.so.0.0', '/usr/lib/libpython2.2.so.
0.0','BINARY')]



#####################################################################
# on ubuntu 6.06, libdcmdata.so.1 and libofstd.so.1 could live in
# /usr/lib, and are therefore thrown out by the McMillan Installer

if os.path.exists('/usr/lib/libdcmdata.so.1') and os.path.exists('/
usr/lib/libofstd.so.1'):
extraLibs.append(('libdcmdata.so.1', '/usr/lib/libdcmdata.so.
1', 'BINARY'))
extraLibs.append(('libofstd.so.1', '/usr/lib/libofstd.so.
1','BINARY'))


######################################################################
# to get this to work on Debian 3.1, we also need to ship libstdc+
+
# and libXinerama
# FIXME: figure some other way out to include the CORRECT libstdc+
+,
# the previous hardcoding of this caused problems with the VL-e
POC

stdc = '/usr/lib/libstdc++.so.6'
if os.path.exists(stdc):
extraLibs.append((os.path.basename(stdc), stdc, 'BINARY'))

xine = '/usr/lib/libXinerama.so.1'
if os.path.exists(xine):
extraLibs.append((os.path.basename(xine), xine, 'BINARY'))


######################################################################
# ubuntu 7.10 has renumbered libtiff 3.7 (or .8) to 4. other
dists of
# course don't have this, so we have to include it.
libtiff = '/usr/lib/libtiff.so.4'
if os.path.exists(libtiff):
extraLibs.append((os.path.basename(libtiff), libtiff,
'BINARY'))



######################################################################
# also add some binary dependencies of numpy that are normally
ignored
# because they are in /lib and/or /usr/lib (see excludes in
bindepend.py)
from distutils import sysconfig
npdir = os.path.join(sysconfig.get_python_lib(), 'numpy')
ladir = os.path.join(npdir, 'linalg')
lplpath = os.path.join(ladir, 'lapack_lite.so')
# use mcmillan function to get LDD dependencies of lapack_lite.so
import bindepend
lpl_deps = bindepend.getImports(lplpath)
for d in lpl_deps:
if d.find('lapack') > 0 or d.find('blas') > 0 or d.find('g2c')
> 0 or d.find('atlas') > 0:
extraLibs.append((os.path.basename(d), d, 'BINARY'))

# end numpy-dependent extraLibs section
##################################################################

# these libs will be removed from the package
remove_binaries = ['libdl.so', 'libutil.so', 'libm.so', 'libc.so',
'libGLU.so', 'libGL.so', 'libGLcore.so',
'libnvidia-tls.so',
'ld-linux-x86-64.so.2', 'libgcc_s.so',
'libtermcap',
'libXft.so', 'libXrandr.so', 'libXrender.so',
'libpthread.so', 'libreadline.so',
'libICE.so',
'libSM.so', 'libX11.so',
'libXext.so', 'libXi.so',
'libXt.so',
'libpango', 'libfontconfig', 'libfreetype',
'libatk', 'libgtk', 'libgdk',
'libglib', 'libgmodule', 'libgobject',
'libgthread',
'librt',
'qt', '_tkinter']

# make sure remove_binaries is lowercase
remove_binaries = [i.lower() for i in remove_binaries]

# global removes: we want to include this file so that the user can
edit it
#remove_binaries += ['defaults.py']

# we have to remove these nasty built-in dependencies EARLY in the
game
dd = config['EXE_dependencies']
newdd = [i for i in dd
if not helper_remove_start(i[0].lower(), remove_binaries)]
config['EXE_dependencies'] = newdd


print "[*] APP_DIR == %s" % (APP_DIR)
print "[*] exeName == %s" % (exeName)
mainScript = os.path.join(APP_DIR, 'testPlot.py')
print "[*] mainScript == %s" % (mainScript)

# generate available kit list
#########################################
# simple form of the checking done by the module_kits package itself

sys.path.insert(0, APP_DIR)
sys.path.append(APP_DIR)
sys.path.append(APP_DIR + os.sep + "numpy_kit")
import numpy_kit

#######################################################################

# all module_kits
module_kits_tree = Tree(os.path.join(APP_DIR, 'numpy_kit'),
'numpy_kit')

print "===== APP_DIR: ", APP_DIR

# MATPLOTLIB data dir
mpl_data_dir = Tree(MPL_DATA_DIR, 'matplotlibdata')

from distutils import sysconfig
numpy_tree = Tree(
os.path.join(sysconfig.get_python_lib(),'numpy'),
prefix=os.path.join('numpy_kit','numpy'),
excludes=['*.pyc', '*.pyo', 'doc', 'docs'])


##########################################################################

SUPPORT_DIR = os.path.join(INSTALLER_DIR, 'support')

a = Analysis([os.path.join(SUPPORT_DIR, '_mountzlib.py'),
os.path.join(SUPPORT_DIR, 'useUnicode.py'),
mainScript],
pathex=[],
hookspath=[os.path.join(APP_DIR, 'installer', 'hooks')])

######################################################################
# sanitise a.pure
#remove_pure_finds = []

#remove_pure_starts = ['numpy_kit']

#for i in range(len(a.pure)-1, -1, -1):
# if helper_remove_finds(a.pure[i][1], remove_pure_finds) or \
# helper_remove_start(a.pure[i][0], remove_pure_starts):
# del a.pure[i]

######################################################################
# sanitise a.binaries

#remove_binary_finds = []

#for i in range(len(a.binaries)-1, -1, -1):
# if helper_remove_finds(a.binaries[i][1], remove_binaries) or \
# helper_remove_start(a.binaries[i][0], remove_binary_finds):
# del a.binaries[i]

######################################################################

# create the compressed archive with all the other pyc files
# will be integrated with EXE archive
pyz = PYZ(a.pure)

# in Installer 6a2, the -f option is breaking things (the support
directory
# is deleted after the first invocation!)
#options = [('f','','OPTION')] # LD_LIBRARY_PATH is correctly set on
Linux
#options = [('v', '', 'OPTION')] # Python is ran with -v
options = []

# because we've already modified the config, we won't be pulling in
# hardcoded dependencies that we don't want.
exe = EXE(pyz,
a.scripts + options,
exclude_binaries=1,
name=exeName,
#icon=os.path.join(APP_DIR, 'resources/graphics/
devidelogo64x64.ico'),
debug=0,
strip=0,
console=True)


all_binaries = a.binaries + module_kits_tree + mpl_data_dir +
extraLibs

coll = COLLECT(exe,
all_binaries,
strip=0,
name='distbuildtestPlot')















On Jun 18, 4:28 pm, "Charl Botha" <cpbo...@gmail.com> wrote:
> On Wed, Jun 18, 2008 at 5:22 AM, arrowtackett <arrowtack...@gmail.com> wrote:
> > Someone over at the wxPython mailing list directed me to their spec
> > file, but I can't seem to get that to work either. Here is the link
> > to it:
>
> >http://code.google.com/p/devide/source/browse/trunk/devide/installer/...
>
> I'm that someone. :) I sent two links in my mail, the other was also
> crucial to make this work:http://code.google.com/p/devide/source/browse/trunk/devide/module_kit...

Charl Botha

unread,
Jun 20, 2008, 3:59:41 AM6/20/08
to PyIns...@googlegroups.com
On Fri, Jun 20, 2008 at 4:39 AM, arrowtackett <arrowt...@gmail.com> wrote:
> Thank you so much for taking the time to help me out. I have gotten
> my exe to read the matplotlib data files. However, I still get what

Great that that's working.

> seems to be a matplotlib error. Here is the stack trace:
>

> File "testPlot/buildtestPlot/outPYZ4.pyz/matplotlib.lines", line
> 423, in recache
> AttributeError: 'module' object has no attribute 'getmask'
>
> It seems to not like the Figure.add_subplot() call in my GUI. Anyone
> have any ideas to fix this problem?

That one also cost me quite some time to fix, it's quite obscure. The
solution is on line 63 of my mplkit __init__.py file:
http://code.google.com/p/devide/source/browse/trunk/devide/module_kits/matplotlib_kit/__init__.py#63

In short, you need to put this somewhere early in your application:

if hasattr(sys, 'frozen') and sys.frozen:

import numpy.core.ma
sys.modules['numpy.ma'] = sys.modules['numpy.core.ma']

The reason why this fixes the getmask problem is documented in my
__init__.py file.

Good luck,
Charl

Reply all
Reply to author
Forward
0 new messages