Four problems with Gordon McMillan's Installer
Gordon Williams
I'm having big problems with Gordon McMillan installer (3i) to make a
standalone package. I'm trying to put together a package using wxPython
2.2.2, Python 2. under W95. I have used earlier versions of the installer,
but this is the first time for 3i. Here is what is happening:
1) Minor problem is that "wxPython.wxc.pyd" is in the standalone directory
but it should be named "wxc.pyd".
2) wx22_2.dll is not found so I am putting it in under "misc". This is the
notice that I am getting (whatever found and not found means):
analyzing multiarray
found python20.dll
found MSVCRT.dll
found KERNEL32.dll
analyzing wxPython.wxc
found wx22_2.dll
lib not found: wx22_2.dll
found python20.dll
found MSVCRT.dll
analyzing _numpy
found python20.dll
found MSVCRT.dll
found KERNEL32.dll
3) I'm getting notices about fixcase for all the files that it is
recompiling. The names before and after are the same!!
Creating BS_Main06B.pyz
name before fixcase e:\program files\python\lib\threading.py
name after fixcase e:\program files\python\lib\threading.py
4) This is the major problem. It is not finding the standard time module.
I am importing time into a number of modules and it cant find it all the
time. If I move the import statement up or down a few lines it sometimes
fixes the problem, only to appear in another module. Here is what I am
getting during a run of the standalone. I cant see anything in the .log
file or the screen output during the build process.
File "e:\program files\python\lib\imputil.py", line 252, in _import_one
module = self._process_result(result, fqname)
File "e:\program files\python\lib\imputil.py", line 281, in
_process_result
exec code in module.__dict__
File "e:\program files\python\junk_files\Main_thread.py", line 1, in ?
import threading, time, Registers, DataSep
File "e:\program files\python\lib\imputil.py", line 88, in _import_hook
top_module = self._import_top_module(parts[0])
File "e:\program files\python\lib\imputil.py", line 170, in
_import_top_module
module = item.import_top(name)
File "e:\program files\python\lib\imputil.py", line 197, in import_top
return self._import_one(None, name, name)
File "e:\program files\python\lib\imputil.py", line 252, in _import_one
module = self._process_result(result, fqname)
File "e:\program files\python\lib\imputil.py", line 281, in
_process_result
exec code in module.__dict__
File "e:\program files\python\lib\threading.py", line 4, in ?
import time
File "e:\program files\python\lib\imputil.py", line 91, in _import_hook
raise ImportError, 'No module named ' + fqname
ImportError: No module named time
I have a web bookmark to Gordon's installer pages that are out of date as
they lead me to version 3f with no indication that there is a new installer
3i available. Maybe these old pages could be deleted so others are not also
mislead.
Needless to say I'm going crazy. Any help or ideas would be appreciated.
Gordon Williams
Thomas Heller
It has been used to create wxPython programs easily.
http://starship.python.net/crew/theller/py2exe
If you insist using Gordon's installer, search c.l.p. Solutions
to your problems have been posted AFAIK.
Regards,
Thomas
Gordon Williams
It worked nicely and very smoothly. The problem that I had with it was that
you pack all the source .pyc files into an easily expandable zip file.
These can be easily extracted and decompiled back to .py for all the world
to see. I dont want people messing with the source so I was looking for
something that was not quite as easy to hack.
Maybe there is something that can be done with your program to at least hide
the python files inside the zip file to provide at least some level of
security? I dont know how others handle this for commercial apps.
Regards,
Gordon Williams
"Thomas Heller" <thomas...@ion-tof.com> wrote in message
news:95dv28$guuo7$1...@ID-59885.news.dfncis.de...
Thomas Heller
> It worked nicely and very smoothly.
The problem that I had with it was that
> you pack all the source .pyc files into an easily expandable zip file.
> These can be easily extracted and decompiled back to .py for all the world
> to see. I dont want people messing with the source so I was looking for
> something that was not quite as easy to hack.
>
> Maybe there is something that can be done with your program to at least hide
> the python files inside the zip file to provide at least some level of
> security? I dont know how others handle this for commercial apps.
Well, this is on the to-do list for later.
I want to do this in two stages:
- first, use a notstandard zip-header so that winzip does
not recognize it
- second, maybe crypt the .pyc files in the archive somehow
and decrypt them in the exe.
Do you think this would be apppropriate?
I have no idea yet for the .pyd files.
Thomas
Robin Dunn
> 4) This is the major problem. It is not finding the standard time module.
> I am importing time into a number of modules and it cant find it all the
> time.
In the Installer sources in support/archive_rt.py after "import imputil" add this line:
sys.path.append(imputil.BuiltinImporter())
--
Robin Dunn
Software Craftsman
ro...@AllDunn.com Java give you jitters?
http://wxPython.org Relax with wxPython!
Dave Brueck
> at least hide
> > the python files inside the zip file to provide at least some level of
> > security? I dont know how others handle this for commercial apps.
> Well, this is on the to-do list for later.
> I want to do this in two stages:
> - first, use a notstandard zip-header so that winzip does
> not recognize it
> - second, maybe crypt the .pyc files in the archive somehow
> and decrypt them in the exe.
>
> Do you think this would be apppropriate?
Yes, that sounds like a good idea... you can use the imp/ihooks (is ihooks
obsolete?) modules to hook into the normal module importer and decrypt them
on the fly. For the zipfile you could still use a normal format but save it
to disk in a munged-up format (something simple like the rotor module would
keep most people out anyway) and then unmunge before reading it.
-Dave
P.S. - I just tried the latest version of py2exe on a wxPython program and
it worked _flawlessly_ and took little time to set up and get going. Very
cool...
Rolander, Dan
I am using Installer with wxPython 2.2.2, and Python 2.0 under Win2K. I had
some initial problems but everything is fine now. Here are my comments:
1) Yes, the problem with wxPython.wxc.pyd is known. Apparently this started
happening with Python 2.0. You'll see this often-- files will be named with
the full package path name. As far as I know the only workaround is to
rename them after the build.
2) Yes, I also include wx22_2.dll on the misc line, like this--
misc= MYSTANDALONE, wxPython.wx22_2
3) I see the fixcase messages also but they don't seem to affect anything.
4) I have an application that is using wxPython and the time module and I
haven't had a problem with it. BUT... I do have a couple of updated files
from Gordon McMillan that are newer than the 3i distribution and did fix A
LOT of problems I was having before. He's advised they will be part of the
3j distribution whenever that is released. I'm attaching them. I recommend
you replace your existing versions with these.
5) Yes, Gordon's starship pages are out of date and I don't know why they
are still there. Be sure to use http://www.mcmillan-inc.com instead.
I hope this helps,
Dan
-------------- MEInc\dist\bindepend.py ----------------
# copyright 1999 McMillan Enterprises, Inc.
# license: use as you please. No warranty.
#
# use dumpbin.exe (if present) to find the binary
# dependencies of an extension module.
# if dumpbin not available, pick apart the PE hdr of the binary
# while this appears to work well, it is complex and subject to
# problems with changes to PE hdrs (ie, this works only on 32 bit Intel
# Windows format binaries)
#
# Note also that you should check the results to make sure that the
# dlls are redistributable. I've listed most of the common MS dlls
# under "excludes" below; add to this list as necessary (or use the
# "excludes" option in the INSTALL section of the config file).
import os
import time
import string
import sys
import tempfile
import finder
seen = {}
excludes = {'KERNEL32.DLL':1,
'ADVAPI.DLL':1,
'MSVCRT.DLL':1,
'ADVAPI32.DLL':1,
'COMCTL32.DLL':1,
'CRTDLL.DLL':1,
'GDI32.DLL':1,
'MFC42.DLL':1,
'NTDLL.DLL':1,
'OLE32.DLL':1,
'OLEAUT32.DLL':1,
'RPCRT4.DLL':1,
'SHELL32.DLL':1,
'USER32.DLL':1,
'WINSPOOL.DRV':1,
'WS2HELP.DLL':1,
'WS2_32.DLL':1,
'WSOCK32.DLL':1,
'WINMM.DLL':1,
'COMDLG32.DLL':1,
'ZLIB.DLL':1,
'ODBC32.DLL':1,
'VERSION.DLL':1}
def getfullnameof(mod, xtrapath = None):
"""Return the full path name of MOD.
MOD is the basename of a dll or pyd.
XTRAPATH is a path or list of paths to search first.
Return the full path name of MOD.
Will search the full Windows search path, as well as sys.path"""
epath = finder.getpath()
if mod[-4:] in ('.pyd', '.PYD'):
epath = epath + sys.path
if xtrapath is not None:
if type(xtrapath) == type(''):
epath.insert(0, xtrapath)
else:
epath = xtrapath + epath
for p in epath:
npth = os.path.join(p, mod)
if os.path.exists(npth):
return npth
return ''
def getImports1(pth):
"""Find the binary dependencies of PTH.
This implementation (not used right now) uses the MSVC utility
dumpbin"""
rslt = []
tmpf = tempfile.mktemp()
os.system('dumpbin /IMPORTS "%s" >%s' %(pth, tmpf))
time.sleep(0.1)
txt = open(tmpf,'r').readlines()
os.remove(tmpf)
i = 0
while i < len(txt):
tokens = string.split(txt[i])
if len(tokens) == 1 and string.find(tokens[0], '.') > 0:
rslt.append(string.strip(tokens[0]))
i = i + 1
return rslt
def getImports2(pth):
"""Find the binary dependencies of PTH.
This implementation walks through the PE header"""
import struct
rslt = []
try:
f = open(pth, 'rb').read()
pehdrd = struct.unpack('l', f[60:64])[0]
magic = struct.unpack('l', f[pehdrd:pehdrd+4])[0]
numsecs = struct.unpack('h', f[pehdrd+6:pehdrd+8])[0]
numdirs = struct.unpack('l', f[pehdrd+116:pehdrd+120])[0]
idata = ''
if magic == 17744:
importsec, sz = struct.unpack('2l', f[pehdrd+128:pehdrd+136])
secttbl = pehdrd + 120 + 8*numdirs
secttblfmt = '8s7l2h'
seclist = []
for i in range(numsecs):
seclist.append(struct.unpack(secttblfmt,
f[secttbl+i*40:secttbl+(i+1)*40]))
#nm, vsz, va, rsz, praw, preloc, plnnums, qrelocs, qlnnums,
flags \
# = seclist[-1]
for i in range(len(seclist)-1):
if seclist[i][2] <= importsec < seclist[i+1][2]:
break
vbase = seclist[i][2]
raw = seclist[i][4]
idatastart = raw + importsec - vbase
idata = f[idatastart:idatastart+seclist[i][1]]
i = 0
while 1:
vsa = struct.unpack('5l', idata[i*20:i*20+20])[3]
if vsa == 0:
break
sa = raw + vsa - vbase
end = string.find(f, '\000', sa)
nm = f[sa:end]
if nm:
rslt.append(nm)
i = i + 1
except IOError:
print "bindepend cannot analyze %s - file not found!" % pth
except struct.error:
print "bindepend cannot analyze %s - error walking thru pehdr" % pth
return rslt
def Dependencies(lTOC):
"""Expand LTOC to include all the closure of binary dependencies.
LTOC is a logical table of contents, ie, a seq of tuples (name, path).
Return LTOC expanded by all the binary dependencies of the entries
in LTOC, except those listed in the module global EXCLUDES"""
for (nm, pth) in lTOC:
fullnm = string.upper(os.path.basename(pth))
if seen.get(string.upper(nm),0):
continue
print "analyzing", nm
seen[string.upper(nm)] = 1
dlls = getImports(pth)
for lib in dlls:
print " found", lib
if excludes.get(string.upper(lib),0):
continue
if seen.get(string.upper(lib),0):
continue
npth = getfullnameof(lib)
if npth:
lTOC.append((lib, npth))
else:
print " lib not found:", lib, "dependency of",
return lTOC
##if getfullnameof('dumpbin.exe') == '':
## def getImports(pth):
## return getImports2(pth)
##else:
## def getImports(pth):
## return getImports1(pth)
def getImports(pth):
"""Forwards to either getImports1 or getImports2
"""
return getImports2(pth)
-------------- support\archive_rt.py ----------------
#
# Gordon McMillan (as inspired and influenced by Greg Stein)
#
# subclasses may not need marshal or struct, but since they're
# builtin, importing is safe.
#
# While an Archive is really an abstraction for any "filesystem
# within a file", it is tuned for use with imputil.FuncImporter.
# This assumes it contains python code objects, indexed by the
# the internal name (ie, no '.py').
# See carchive.py for a more general archive (contains anything)
# that can be understood by a C program.
#archive_rt is a stripped down version of MEInc.Dist.archive.
#It has had all building logic removed.
#It's purpose is to bootstrap the Python installation.
_verbose = 0
_listdir = None
import marshal
import struct
import imp
import sys
import imputil
_c_suffixes = filter(lambda x: x[2] == imp.C_EXTENSION, imp.get_suffixes())
for nm in ('nt', 'posix', 'dos', 'os2', 'mac'):
if nm in sys.builtin_module_names:
mod = __import__(nm)
_listdir = mod.listdir
class Archive(imputil.Importer):
""" A base class for a repository of python code objects.
The extract method is used by imputil.ArchiveImporter
to get code objects by name (fully qualified name), so
an enduser "import a.b" would become
extract('a.__init__')
extract('a.b')
"""
MAGIC = 'PYL\0'
HDRLEN = 12 # default is MAGIC followed by python's magic, int pos
of toc
TOCPOS = 8
TRLLEN = 0 # default - no trailer
TOCTMPLT = {} #
os = None
_bincache = None
def __init__(self, path=None, start=0):
"Initialize an Archive. If path is omitted, it will be an empty
Archive."
self.toc = None
self.path = path
self.start = start
import imp
self.pymagic = imp.get_magic()
if path is not None:
self.lib = open(self.path, 'rb')
self.checkmagic()
self.loadtoc()
if Archive._bincache is None:
Archive._bincache = {}
for fnm in _listdir(sys.path[0]):
for ext, mode, typ in _c_suffixes:
sz = len(ext)
if fnm[-sz:] == ext:
Archive._bincache[fnm[:-sz]] = sys.path[0] + '/' + fnm, (ext,
mode, typ)
if _verbose:
print "Archive caching", fnm[:-sz]
####### Sub-methods of __init__ - override as needed #############
def checkmagic(self):
""" Overridable.
Check to see if the file object self.lib actually has a file
we understand.
"""
self.lib.seek(self.start) #default - magic is at start of file
if self.lib.read(len(self.MAGIC)) != self.MAGIC:
raise RuntimeError, "%s is not a valid %s archive file" \
% (self.path, self.__class__.__name__)
if self.lib.read(len(self.pymagic)) != self.pymagic:
raise RuntimeError, "%s has version mismatch to dll" % (self.path)
def loadtoc(self):
""" Overridable.
Default: After magic comes an int (4 byte native) giving the
position of the TOC within self.lib.
Default: The TOC is a marshal-able string.
"""
self.lib.seek(self.start + self.TOCPOS)
(offset,) = struct.unpack('=i', self.lib.read(4))
self.lib.seek(self.start + offset)
self.toc = marshal.load(self.lib)
######## This is what is called by FuncImporter #######
## Since an Archive is flat, we ignore parent and modname.
def get_code(self, parent, modname, fqname):
if _verbose:
print "get_code(%s, %s, %s, %s)" % (self, parent, modname, fqname)
rslt = self.extract(fqname) # None if not found, (ispkg, code) otherwise
if rslt is None:
if parent:
importer = getattr(parent, "__importer__", None)
if importer:
func = getattr(importer, "func", None)
if func == self.get_code:
file, desc = Archive._bincache.get(fqname, (None, None))
if file:
try:
fp = open(file, desc[1])
except IOError:
pass
else:
module = imp.load_module(fqname, fp, file, desc)
if _verbose:
print "#found", file
return 0, module, {'__file__':file}
if _verbose:
print "#not found"
return None
ispkg, code = rslt
if ispkg:
return ispkg, code, {'__path__' : [fqname]}
return ispkg, code, {}
####### Core method - Override as needed #########
def extract(self, name):
""" Get the object corresponding to name, or None.
For use with imputil ArchiveImporter, object is a python code
object.
'name' is the name as specified in an 'import name'.
'import a.b' will become:
extract('a') (return None because 'a' is not a code object)
extract('a.__init__') (return a code object)
extract('a.b') (return a code object)
Default implementation:
self.toc is a dict
self.toc[name] is pos
self.lib has the code object marshal-ed at pos
"""
ispkg, pos = self.toc.get(name, (0,None))
if pos is None:
return None
self.lib.seek(self.start + pos)
return ispkg, marshal.load(self.lib)
########################################################################
# Informational methods
def contents(self):
"""Return a list of the contents
Default implementation assumes self.toc is a dict like object.
Not required by ArchiveImporter.
"""
return self.toc.keys()
########################################################################
# Building
####### Top level method - shouldn't need overriding #######
## def build(self, path, lTOC):
## """Create an archive file of name 'path'.
## lTOC is a 'logical TOC' - a list of (name, path, ...)
## where name is the internal name, eg 'a'
## and path is a file to get the object from, eg './a.pyc'.
## """
## self.path = path
## self.lib = open(path, 'wb')
## #reserve space for the header
## if self.HDRLEN:
## self.lib.write('\0'*self.HDRLEN)
##
## #create an empty toc
##
## if type(self.TOCTMPLT) == type({}):
## self.toc = {}
## else: # assume callable
## self.toc = self.TOCTMPLT()
##
## for tocentry in lTOC:
## self.add(tocentry) # the guts of the archive
##
## tocpos = self.lib.tell()
## self.save_toc(tocpos)
## if self.TRLLEN:
## self.save_trailer(tocpos)
## if self.HDRLEN:
## self.update_headers(tocpos)
## self.lib.close()
##
##
## ####### manages keeping the internal TOC and the guts in sync #######
## def add(self, entry):
## """Override this to influence the mechanics of the Archive.
## Assumes entry is a seq beginning with (nm, pth, ...) where
## nm is the key by which we'll be asked for the object.
## pth is the name of where we find the object. Overrides of
## get_obj_from can make use of further elements in entry.
## """
## if self.os is None:
## import os
## self.os = os
## nm = entry[0]
## pth = entry[1]
## ispkg = self.os.path.splitext(self.os.path.basename(pth))[0] ==
'__init__'
## self.toc[nm] = (ispkg, self.lib.tell())
## f = open(entry[1], 'rb')
## f.seek(8) #skip magic and timestamp
## self.lib.write(f.read())
##
## def save_toc(self, tocpos):
## """Default - toc is a dict
## Gets marshaled to self.lib
## """
## marshal.dump(self.toc, self.lib)
##
## def save_trailer(self, tocpos):
## """Default - not used"""
## pass
##
## def update_headers(self, tocpos):
## """Default - MAGIC + Python's magic + tocpos"""
## self.lib.seek(self.start)
## self.lib.write(self.MAGIC)
## self.lib.write(self.pymagic)
## self.lib.write(struct.pack('=i', tocpos))
##############################################################
#
# ZlibArchive - an archive with compressed entries
#
class ZlibArchive(Archive):
MAGIC = 'PYZ\0'
TOCPOS = 8
HDRLEN = 12
TRLLEN = 0
TOCTMPLT = {}
LEVEL = 9
def __init__(self, path=None, offset=0):
Archive.__init__(self, path, offset)
# dynamic import so not imported if not needed
global zlib
import zlib
def extract(self, name):
(ispkg, pos, lngth) = self.toc.get(name, (0, None, 0))
if pos is None:
return None
self.lib.seek(self.start + pos)
return ispkg, marshal.loads(zlib.decompress(self.lib.read(lngth)))
## def add(self, entry):
## if self.os is None:
## import os
## self.os = os
## nm = entry[0]
## pth = entry[1]
## ispkg = self.os.path.splitext(self.os.path.basename(pth))[0] ==
'__init__'
## f = open(pth, 'rb')
## f.seek(8) #skip magic and timestamp
## obj = zlib.compress(f.read(), self.LEVEL)
## self.toc[nm] = (ispkg, self.lib.tell(), len(obj))
## self.lib.write(obj)
##
if type(__import__) == type(open):
imputil.ImportManager().install()
sys.path.append(imputil.BuiltinImporter())
Thomas Heller
"Dave Brueck" <dbr...@edgix.com> wrote in message news:mailman.981135004...@python.org...
> > > Maybe there is something that can be done with your program to
> > at least hide
> > > the python files inside the zip file to provide at least some level of
> > > security? I dont know how others handle this for commercial apps.
> > Well, this is on the to-do list for later.
> > I want to do this in two stages:
> > - first, use a notstandard zip-header so that winzip does
> > not recognize it
> > - second, maybe crypt the .pyc files in the archive somehow
> > and decrypt them in the exe.
> >
> > Do you think this would be apppropriate?
>
> Yes, that sounds like a good idea... you can use the imp/ihooks (is ihooks
> obsolete?) modules to hook into the normal module importer and decrypt them
> on the fly.
This is what I had planned.
> For the zipfile you could still use a normal format but save it
> to disk in a munged-up format (something simple like the rotor module would
> keep most people out anyway) and then unmunge before reading it.
>
> -Dave
>
> P.S. - I just tried the latest version of py2exe on a wxPython program and
> it worked _flawlessly_ and took little time to set up and get going. Very
> cool...
>
>
Thanks for trying it,
Thomas
Martin von Loewis
> I dont want people messing with the source so I was looking for
> something that was not quite as easy to hack.
In that case, I recommend freezing. It also allows to integrate all
DLLs (provided you have or can produce static libraries for them).
Regards,
Martin
Franz GEIGER
Regards
Franz GEIGER
"Thomas Heller" <thomas...@ion-tof.com> wrote in message
news:95dv28$guuo7$1...@ID-59885.news.dfncis.de...