Problem with utf-8

110 views
Skip to first unread message

Vinay Sajip

unread,
May 8, 2009, 6:18:49 AM5/8/09
to PyInstaller
I just checked out PyInstaller trunk (r659) to try it out with Python
2.6.2 (the default Python on Ubuntu Jaunty). The following trivial
program:

def main():
print repr(u'\u2029'.encode('utf-8'))

if __name__ == "__main__":
main()

was built using all defaults and --onefile. When the output executable
is run on the same system, it works as expected, and outputs

'\xe2\x80\xa9'

However, if the executable is copied to another machine (Ubuntu Hardy,
running Python 2.5.2) and run, I get the following result:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
File "<string>", line 5, in <module>
File "<string>", line 2, in main
LookupError: unknown encoding: utf-8

Running ArchiveViewer on the executable yields

[(0, 560119, 560119, 0, 'z', 'outPYZ1.pyz'),
(560119, 8149, 21025, 1, 'm', 'iu'),
(568268, 148, 197, 1, 'm', 'struct'),
(568416, 5920, 14797, 1, 'm', 'archive'),
(574336, 1157, 2272, 1, 's', '_mountzlib'),
(575493, 81, 76, 1, 's', 'useUnicode'),
(575574, 91, 96, 1, 's', 'munge'),
(575665, 10520, 24524, 1, 'b', '_multibytecodec.so'),
(586185, 29388, 71928, 1, 'b', 'datetime.so'),
(615573, 7417, 17776, 1, 'b', '_codecs_iso2022.so'),
(622990, 61544, 95596, 1, 'b', '_codecs_tw.so'),
(684534, 88959, 226668, 1, 'b', '_codecs_jp.so'),
(773493, 7422, 16692, 1, 'b', '_heapq.so'),
(780915, 98862, 128364, 1, 'b', '_codecs_cn.so'),
(879777, 74156, 120172, 1, 'b', '_codecs_kr.so'),
(953933, 7591, 21524, 1, 'b', 'readline.so'),
(961524, 13352, 32280, 1, 'b', 'bz2.so'),
(974876, 33133, 140652, 1, 'b', '_codecs_hk.so'),
(1008009, 90474, 196592, 1, 'b', 'libreadline.so.5'),
(1098483, 116783, 248312, 1, 'b', 'libncursesw.so.5'),
(1215266, 94859, 199128, 1, 'b', 'libncurses.so.5'),
(1310125, 29902, 70020, 1, 'b', 'libbz2.so.1.0'),
(1340027, 966526, 2393016, 1, 'b', 'libpython2.6.so.1.0'),
(2306553, 45093, 83552, 1, 'b', 'libz.so.1')]

[N.B. in the above list, the test script is "munge.py" which leads to
the "munge" entry] and the contents of outPYZ1.pyz are

{'StringIO': (False, 261465L, 4591),
'UserDict': (False, 235627L, 2726),
'__future__': (False, 30508L, 1800),
'_abcoll': (False, 473808L, 5954),
'_strptime': (False, 546706L, 6473),
'_threading_local': (False, 344566L, 2571),
'abc': (False, 157248L, 2448),
'base64': (False, 316387L, 4342),
'bdb': (False, 397432L, 6816),
'bisect': (False, 462811L, 960),
'calendar': (False, 145328L, 9233),
'cmd': (False, 75834L, 6043),
'codecs': (False, 243413L, 10222),
'collections': (False, 52284L, 2922),
'copy': (False, 391316L, 4798),
'copy_reg': (False, 444775L, 2461),
'difflib': (False, 178682L, 21734),
'dis': (False, 304689L, 3030),
'doctest': (False, 275829L, 27521),
'dummy_thread': (False, 338274L, 2216),
'encodings': (True, 143151L, 2177),
'encodings.aliases': (False, 418929L, 3125),
'encodings.ascii': (False, 520330L, 843),
'encodings.base64_codec': (False, 73229L, 1276),
'encodings.big5': (False, 34278L, 661),
'encodings.big5hkscs': (False, 74505L, 667),
'encodings.bz2_codec': (False, 336718L, 1556),
'encodings.charmap': (False, 378291L, 1109),
'encodings.cp037': (False, 71935L, 1294),
'encodings.cp1006': (False, 387367L, 1368),
'encodings.cp1026': (False, 238353L, 1304),
'encodings.cp1140': (False, 36270L, 1278),
'encodings.cp1250': (False, 141809L, 1342),
'encodings.cp1251': (False, 460129L, 1338),
'encodings.cp1252': (False, 458795L, 1334),
'encodings.cp1253': (False, 386044L, 1323),
'encodings.cp1254': (False, 457461L, 1334),
'encodings.cp1255': (False, 456140L, 1321),
'encodings.cp1256': (False, 454785L, 1355),
'encodings.cp1257': (False, 453451L, 1334),
'encodings.cp1258': (False, 452113L, 1338),
'encodings.cp424': (False, 390056L, 1260),
'encodings.cp437': (False, 354613L, 3770),
'encodings.cp500': (False, 410170L, 1294),
'encodings.cp737': (False, 363378L, 3913),
'encodings.cp775': (False, 48497L, 3787),
'encodings.cp850': (False, 120960L, 3561),
'encodings.cp852': (False, 117173L, 3787),
'encodings.cp855': (False, 113297L, 3876),
'encodings.cp856': (False, 323374L, 1287),
'encodings.cp857': (False, 108270L, 3574),
'encodings.cp860': (False, 504749L, 3747),
'encodings.cp861': (False, 508496L, 3763),
'encodings.cp862': (False, 44620L, 3877),
'encodings.cp863': (False, 500982L, 3767),
'encodings.cp864': (False, 492669L, 3884),
'encodings.cp865': (False, 496553L, 3765),
'encodings.cp866': (False, 485718L, 3886),
'encodings.cp869': (False, 481914L, 3804),
'encodings.cp874': (False, 322040L, 1334),
'encodings.cp875': (False, 320729L, 1311),
'encodings.cp932': (False, 5620L, 662),
'encodings.cp949': (False, 75172L, 662),
'encodings.cp950': (False, 29845L, 663),
'encodings.euc_jis_2004': (False, 358383L, 669),
'encodings.euc_jisx0213': (False, 443448L, 668),
'encodings.euc_jp': (False, 444116L, 659),
'encodings.euc_kr': (False, 64689L, 659),
'encodings.gb18030': (False, 525609L, 667),
'encodings.gb2312': (False, 253635L, 663),
'encodings.gbk': (False, 155916L, 660),
'encodings.hex_codec': (False, 472518L, 1290),
'encodings.hp_roman8': (False, 384275L, 1769),
'encodings.hz': (False, 266056L, 661),
'encodings.idna': (False, 521845L, 2523),
'encodings.iso2022_jp': (False, 175063L, 666),
'encodings.iso2022_jp_1': (False, 240595L, 667),
'encodings.iso2022_jp_2': (False, 241929L, 667),
'encodings.iso2022_jp_2004': (False, 271276L, 672),
'encodings.iso2022_jp_3': (False, 241262L, 667),
'encodings.iso2022_jp_ext': (False, 32308L, 668),
'encodings.iso2022_kr': (False, 418262L, 667),
'encodings.iso8859_1': (False, 101810L, 1286),
'encodings.iso8859_10': (False, 55939L, 1314),
'encodings.iso8859_11': (False, 404248L, 1350),
'encodings.iso8859_13': (False, 396114L, 1318),
'encodings.iso8859_14': (False, 34939L, 1331),
'encodings.iso8859_15': (False, 32976L, 1302),
'encodings.iso8859_16': (False, 388735L, 1321),
'encodings.iso8859_2': (False, 100499L, 1311),
'encodings.iso8859_3': (False, 99190L, 1309),
'encodings.iso8859_4': (False, 106959L, 1311),
'encodings.iso8859_5': (False, 105656L, 1303),
'encodings.iso8859_6': (False, 104411L, 1245),
'encodings.iso8859_7': (False, 103096L, 1315),
'encodings.iso8859_8': (False, 96579L, 1248),
'encodings.iso8859_9': (False, 95285L, 1294),
'encodings.johab': (False, 6282L, 660),
'encodings.koi8_r': (False, 177321L, 1361),
'encodings.koi8_u': (False, 200416L, 1343),
'encodings.latin_1': (False, 368108L, 858),
'encodings.mac_arabic': (False, 65348L, 3678),
'encodings.mac_centeuro': (False, 347137L, 1348),
'encodings.mac_croatian': (False, 216039L, 1362),
'encodings.mac_cyrillic': (False, 461467L, 1344),
'encodings.mac_farsi': (False, 368966L, 1293),
'encodings.mac_greek': (False, 526276L, 1337),
'encodings.mac_iceland': (False, 154561L, 1355),
'encodings.mac_latin2': (False, 479762L, 2152),
'encodings.mac_roman': (False, 348485L, 1356),
'encodings.mac_romanian': (False, 97827L, 1363),
'encodings.mac_turkish': (False, 201759L, 1358),
'encodings.mbcs': (False, 528430L, 883),
'encodings.palmos': (False, 161832L, 1239),
'encodings.ptcp154': (False, 259353L, 2112),
'encodings.punycode': (False, 341300L, 3266),
'encodings.quopri_codec': (False, 465262L, 1244),
'encodings.raw_unicode_escape': (False, 527613L, 817),
'encodings.rot_13': (False, 111844L, 1453),
'encodings.shift_jis': (False, 500318L, 664),
'encodings.shift_jis_2004': (False, 156576L, 672),
'encodings.shift_jisx0213': (False, 521173L, 672),
'encodings.string_escape': (False, 463771L, 758),
'encodings.tis_620': (False, 303350L, 1339),
'encodings.undefined': (False, 239657L, 938),
'encodings.unicode_escape': (False, 340490L, 810),
'encodings.unicode_internal': (False, 417450L, 812),
'encodings.utf_16': (False, 359052L, 1629),
'encodings.utf_16_be': (False, 367291L, 817),
'encodings.utf_16_le': (False, 242596L, 817),
'encodings.utf_32': (False, 376428L, 1863),
'encodings.utf_32_be': (False, 464529L, 733),
'encodings.utf_32_le': (False, 55206L, 733),
'encodings.utf_7': (False, 57253L, 763),
'encodings.utf_8': (False, 69026L, 811),
'encodings.utf_8_sig': (False, 175729L, 1592),
'encodings.uu_codec': (False, 257405L, 1948),
'encodings.zlib_codec': (False, 271948L, 1452),
'functools': (False, 6942L, 881),
'genericpath': (False, 512259L, 1342),
'getopt': (False, 489604L, 3065),
'gettext': (False, 466506L, 6012),
'heapq': (False, 17L, 5603),
'inspect': (False, 203117L, 12922),
'keyword': (False, 405598L, 1237),
'linecache': (False, 545010L, 1696),
'locale': (False, 124521L, 17288),
'ntpath': (False, 170039L, 5024),
'opcode': (False, 273400L, 2429),
'optparse': (False, 217401L, 18226),
'os': (False, 307719L, 8668),
'os2emxpath': (False, 159696L, 2136),
'pdb': (False, 529313L, 15697),
'pickle': (False, 429946L, 13502),
'posixpath': (False, 349841L, 4772),
'pprint': (False, 81877L, 4188),
'quopri': (False, 254298L, 3107),
'random': (False, 7823L, 9969),
're': (False, 164985L, 5054),
'repr': (False, 69837L, 2098),
'shlex': (False, 406835L, 3335),
'sre_compile': (False, 447236L, 4877),
'sre_constants': (False, 360681L, 2697),
'sre_parse': (False, 422054L, 7892),
'stat': (False, 513601L, 991),
'string': (False, 58016L, 6673),
'stringprep': (False, 411464L, 5986),
'subprocess': (False, 17792L, 12053),
'tempfile': (False, 37548L, 7072),
'textwrap': (False, 379400L, 4875),
'threading': (False, 86065L, 9220),
'token': (False, 163071L, 1914),
'tokenize': (False, 370259L, 6169),
'traceback': (False, 266717L, 4559),
'types': (False, 524368L, 1241),
'unittest': (False, 324661L, 12057),
'warnings': (False, 514592L, 5738)}

So, utf-8 appears to be included ('encodings.utf_8') but the lookup
fails. Also, I would like to understand how the "could not find
platform (in)dependent libraries" messages can be avoided - they may
confuse users who are not Python-savvy.

I do realise that PyInstaller is supposed to be for 1.5 -> 2.4 but
I've seen messages that indicate that the SVN version should work with
later Python versions.

Thanks for any help you can give,


Vinay Sajip

Vinay Sajip

unread,
May 9, 2009, 11:38:54 AM5/9/09
to PyInstaller


On May 8, 11:18 am, Vinay Sajip <vinay_sa...@yahoo.co.uk> wrote:

> So, utf-8 appears to be included ('encodings.utf_8') but the lookup
> fails. Also, I would like to understand how the "could not find
> platform (in)dependent libraries" messages can be avoided - they may
> confuse users who are not Python-savvy.

I've found a partial fix - to the problem of the spurious messages. A
patch is attached to Trac ticket #56. I've tested it in the following
scenarios:

1. Build on Ubuntu Hardy/python2.5, copy to and run on Ubuntu Jaunty/
python2.6.
2. Build on Ubuntu Jaunty/python2.6, copy to and run on Ubuntu Hardy/
python2.5.

Regards,

Vinay Sajip

Giovanni Bajo

unread,
May 12, 2009, 8:46:42 AM5/12/09
to PyIns...@googlegroups.com

Many thanks -- as noted in the bug, I've just committed the fix.
--
Giovanni Bajo
Develer S.r.l.
http://www.develer.com

Vinay Sajip

unread,
May 13, 2009, 6:16:48 AM5/13/09
to PyInstaller
Thanks for checking in the patch.This fixes the spurious messages but
not the encodings problem, for which I've opened ticket #57.

Regards,

Vinay Sajip
Reply all
Reply to author
Forward
0 new messages