New issue 87 by ro...@reportlab.com: FreeBSD 7.1 unladen-2009Q3 build fails
http://code.google.com/p/unladen-swallow/issues/detail?id=87
After using nagy.attila's suggested fix for the initial FreeBSD problems I
can compile most of unladen-2009Q3. I used this configure
and get this error
g++ -pthread -c -IUtil/llvm/include -I./Util/llvm/include
-D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -Woverloaded-virtual
-fno-strict-aliasing -fwrapv -Wall -DNDEBUG -g -O3 -I. -IInclude -I.
-I./Include -DPy_BUILD_CORE -Wno-write-strings -o Util/PyAliasAnalysis.o
Util/PyAliasAnalysis.cc
In file included from /usr/include/c++/4.2/ios:47,
from /usr/include/c++/4.2/ostream:45,
from /usr/include/c++/4.2/iterator:70,
from Util/llvm/include/llvm/ADT/DenseMap.h:20,
from ./Util/ConstantMirror.h:10,
from ./Python/global_llvm_data.h:15,
from ./Util/PyTypeBuilder.h:14,
from Util/PyAliasAnalysis.cc:3:
/usr/include/c++/4.2/bits/localefwd.h:58:34: error: macro "isspace" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:70:34: error: macro "isupper" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:74:34: error: macro "islower" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:78:34: error: macro "isalpha" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:94:34: error: macro "isalnum" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:102:34: error: macro "toupper" passed
2 arguments, but takes just 1
/usr/include/c++/4.2/bits/localefwd.h:106:34: error: macro "tolower" passed
2 arguments, but takes just 1
..........
I guess this must mean that a particular macro is in play when the
definitions from this file are used.
localefwd.h looks like
_GLIBCXX_BEGIN_NAMESPACE(std)
// 22.1.1 Locale
class locale;
// 22.1.3 Convenience interfaces
template<typename _CharT>
inline bool
isspace(_CharT, const locale&);
template<typename _CharT>
inline bool
isprint(_CharT, const locale&);
--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings
This should be a defect (I used the defect template at least). Anyhow I
forgot my
configure line which looked like this
./configure --prefix=/home/memememe/UNLADEN --with-unicode=ucs2
This looks like http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31005
What are your GCC and libstdc++ versions?
$ gcc --version
gcc (GCC) 4.2.1 20070719 [FreeBSD]
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
~/devel/unladen-2009Q3:
$ g++ --version
g++ (GCC) 4.2.1 20070719 [FreeBSD]
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
I have /usr/lib/libstdc++.so.6, but am not exactly sure which exact version.
That's old enough that it could be the bug that I referenced above. Can
you try with
a newer version of GCC? I'm not that familiar with FreeBSD but they seem
to have
more recent versions of GCC available in the ports tree. Something like:
# cd /usr/ports/lang/gcc44 && make install
And then use CC=gcc44 and CXX=g++44 to re-configure and build Unladen
I've started the gcc4.4 build and will try and report back tomorrow.
OK I got the gcc44 built and now the unladen-2009Q3 configure and build
does go
through to the end. However, my simplistic test of working
$ unladen26 unladen-2009Q3/Lib/test/pystone.py
Pystone(1.1) time for 50000 passes = 1.82031
This machine benchmarks at 27467.8 pystones/second
~/devel:
$ python25 unladen-2009Q3/Lib/test/pystone.py
Pystone(1.1) time for 50000 passes = 0.984375
This machine benchmarks at 50793.7 pystones/second
~/devel:
makes python25 seem quite fast. Is this result expected because of the way
that llvm
works?
I'd say that's expected. Unladen Swallow is mostly tuned for long-running
server
processes that can take advantage of the JIT, so to model that in our
benchmarks, we
warm up the benchmark before we start timing. We haven't implemented this
for
PyStone, but you could probably hack it to run twice. The second run
should be
faster than Python 2.5.
I'm not surprised that we run PyStone slowly. It's listed in our Benchmarks
page (http://code.google.com/p/unladen-swallow/wiki/Benchmarks) as a
benchmark we explicitly do not use; it bears little relation to real-world
idiomatic Python code, and certainly not to the kind of applications we're
trying
to optimize.
That said, I'm glad you got it building on FreeBSD. I'll try to apply the
configure changes today.
Comment #9 on issue 87 by collinw: FreeBSD 7.1 unladen-2009Q3 build fails
http://code.google.com/p/unladen-swallow/issues/detail?id=87
Configure changes applied as r884.
That kind of thing could happen when libpython.a got built before some of
the LLVM
libraries it depended on. You would see an error from `ar` toward the top
of the build
log, but because of a bug in our Makefile it wouldn't stop the build. I
fixed that bug and
got the ordering (closer to) right in r938. Sorry for the extra work, but
could you sync
and try again? You probably won't have to `make clean`.
Thanks for the help.
OK I did the update and then another gmake; things certainly proceed
further now.
Right at the end of the build I notice these messages; are these
significant?
Failed to find the necessary bits to build these modules:
_sqlite3 _tkinter bsddb185
dl gdbm imageop
linuxaudiodev spwd sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the
module's name.
Failed to build these modules:
_locale
running build_scripts
creating build/scripts-2.6
copying and adjusting
/usr/home/rptlab/devel/unladen-swallow-read-only/Tools/scripts/pydoc ->
build/scripts-2.6
copying and adjusting
/usr/home/rptlab/devel/unladen-swallow-read-only/Tools/scripts/idle ->
build/scripts-2.6
copying and adjusting
/usr/home/rptlab/devel/unladen-swallow-read-only/Tools/scripts/2to3 ->
build/scripts-2.6
copying and adjusting
/usr/home/rptlab/devel/unladen-swallow-read-only/Lib/smtpd.py
-> build/scripts-2.6
changing mode of build/scripts-2.6/pydoc from 644 to 755
changing mode of build/scripts-2.6/idle from 644 to 755
changing mode of build/scripts-2.6/2to3 from 644 to 755
changing mode of build/scripts-2.6/smtpd.py from 644 to 755
"Failed to find" messages usually mean your system doesn't have some
library that
module depends on. They're generally harmless. "Failed to build" messages
are
usually accompanied by some sort of compiler or linker error. For example,
when I
compile Python on OSX with MacPorts gcc-4.4, I get:
building '_locale' extension
gcc -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -g -I. -
I/Users/jyasskin/src/unladen-swallow/trunk/src/./Include -
I/Users/jyasskin/src/unladen-swallow/trunk/src/./Mac/Include -I. -IInclude
-I../src
-I/usr/local/include -I../src/Include -I/Users/jyasskin/src/unladen-
swallow/trunk/dbg -c /Users/jyasskin/src/unladen-
swallow/trunk/src/Modules/_localemodule.c -o build/temp.macosx-10.3-i386-
2.6-pydebug/Users/jyasskin/src/unladen-
swallow/trunk/src/Modules/_localemodule.o
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.3-i386-2.6-
pydebug/Users/jyasskin/src/unladen-swallow/trunk/src/Modules/_localemodule.o
-
L/usr/local/lib -ldl -o build/lib.macosx-10.3-i386-2.6-pydebug/_locale.so -
framework CoreFoundation
*** WARNING: renaming "_locale" since importing it failed:
dlopen(build/lib.macosx-
10.3-i386-2.6-pydebug/_locale.so, 2): Symbol not found:
_libintl_bindtextdomain
Referenced from: /Users/jyasskin/src/unladen-
swallow/trunk/dbg/build/lib.macosx-10.3-i386-2.6-pydebug/_locale.so
Expected in: dynamic lookup
(You only get the gcc lines if you use plain `make` without -j.) Mine is
probably
caused by a missing #ifdef in _localemodule.c. You may have a different
error.
Missing _locale may cause a couple test failures, but most of the test
suite should
still pass, and Python should work unless you try to import that module.
How does
the test suite do on your box?
I get these outputs from testall.py. I also had to interrupt a threading
test which
seemed to hang.
test test_smtplib failed -- Traceback (most recent call last):
File "/home/rptlab/UNLADEN/lib/python2.6/test/test_smtplib.py", line 225,
in testSend
self.assertEqual(self.output.getvalue(), mexpect)
AssertionError: "---------- MESSAGE FOLLOWS ----------\nA test
message\n------------
END MESSAGE ------------\nerror: uncaptured python exception, closing
channel
<smtpd.SMTPChannel 127.0.0.1:61298 at 0x80c62e560>
(<class 'socket.error'>:[Errno 9]
Bad file descriptor [/home/rptlab/UNLADEN/lib/python2.6/asyncore.py|
readwrite|101]
[/home/rptlab/UNLADEN/lib/python2.6/asyncore.py|handle_write_event|427]
[<string>|getsockopt|1] [/home/rptlab/UNLADEN/lib/python2.6/socket.py|
_dummy|165])\n"
!= '---------- MESSAGE FOLLOWS ----------\nA test message\n------------ END
MESSAGE
------------\n'
I got these after patching and restarting the build. I will repeat after a
gmake
clean in case that has something to do with failure to build the _locale
module.
test_smtplib is known to be flaky, so I wouldn't worry about that one. Does
the
threading test hang every time? Which threading test is it?
The _locale problem appears to be because when built it fails to import ie
I see this
in the gmake
gcc44 -shared
build/temp.freebsd-7.1-RELEASE-amd64-2.6/usr/home/rptlab/devel/unl
building '_locale' extension
gcc44 -pthread -fPIC -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes
-DND
gcc44 -shared
build/temp.freebsd-7.1-RELEASE-amd64-2.6/usr/home/rptlab/devel/unl
*** WARNING: renaming "_locale" since importing it failed:
build/lib.freebsd-7.1
I copied _locale_failed.so into the dynload folder and then
import _local_failed produces this
$ unladen26
Python 2.6.1 (r261:942M, Dec 19 2009, 13:52:00)
[GCC 4.4.3 20091020 (prerelease)] on freebsd7
[Unladen Swallow 2009Q4]
Type "help", "copyright", "credits" or "license" for more information.
>>> import _locale_failed
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError:
/home/rptlab/UNLADEN/lib/python2.6/lib-dynload/_locale_failed.so:
Undefined symbol "libintl_bindtextdomain"
During testall.py it appears to hang in test_threading
the output looks like
test_threading
test_PyThreadState_SetAsyncExc (test.test_threading.ThreadTests) ...
started
worker thread
trying nonsensical thread id
waiting for worker thread to get started
verifying worker hasn't exited
attempting to raise asynch exception in worker
waiting for worker to say it caught the exception
all OK -- joining worker
ok
test_enumerate_after_join (test.test_threading.ThreadTests) ... ok
test_finalize_runnning_thread (test.test_threading.ThreadTests) ... ok
test_finalize_with_trace (test.test_threading.ThreadTests) ... ok
test_foreign_thread (test.test_threading.ThreadTests) ... ok
test_no_refcycle_through_target (test.test_threading.ThreadTests) ... ok
test_various_ops (test.test_threading.ThreadTests) ... task <thread 0> will
run for
60.9 usec
task <thread 1> will run for 15.5 usec
1 tasks are running
task <thread 2> will run for 85.7 usec
task <thread 3> will run for 74.0 usec
2 tasks are running
task <thread 4> will run for 98.4 usec
task <thread 1> 3 tasks are runningtask <thread 6> will run for 76.9 usec
done
task <thread 0> done
<thread 1> is finished. 2 tasks are running
task <thread 5> will run for 82.8 usec
<thread 0> is finished. 1 tasks are running
task <thread 7> will run for 52.0 usec
2 tasks are running
task <thread 3>task <thread 8> will run for 12.4 usec
done
3task <thread 9> will run for 90.0 usec
waiting for all tasks to complete
tasks are running
task <thread 2> done
<thread 3> is finished. 2 tasks are running
<thread 2> is finished. 1 tasks are running
2 tasks are running
3 tasks are running
task <thread 4> done
<thread 4> is finished. 2 tasks are running
3 tasks are running
task <thread 6> done
<thread 6> is finished. 2 tasks are running
3 tasks are running
task <thread 5> done
<thread 5> is finished. 2 tasks are running
3 tasks are running
task <thread 7> done
<thread 7> is finished. 2 tasks are running
task <thread 8> task <thread 9> done
done
<thread 8> is finished. 1 tasks are running
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_large_stack (test.test_threading.ThreadTests) ... with 1MB
thread
stack size...
task <thread 0> will run for 90.4 usec
task <thread 1> will run for 61.5 usec
1 task <thread 2> will run for 10.8 usec
tasks are running
2 tasks are runningtask <thread 3> will run for 54.8 usec
task <thread 4> will run for 80.3 usec
3 tasks are running
task <thread 5> will run for 99.8 usec
task <thread 6> will run for 34.3 usec
task <thread 0> task <thread 7> will run for 12.4 usec
done
<thread 0> is finished. 2 tasks are running
3 tasks are running
task <thread 8> will run for 1.5 usec
task <thread 2> done
task <thread 1> task <thread 9> will run for 22.6 usec
done
<thread 2> is finished. 2 tasks are running
waiting for all tasks to complete
task <thread 7> done
<thread 1> is finished. 1 tasks are running
2 tasks are running
3 tasks are running
<thread 7> is finished. 2 tasks are running
3 tasks are running
task <thread 4> task <thread 5> done
<thread 5> is finished. 2 tasks are running
done
3 tasks are running
<thread 4> is finished. 2 tasks are running
3 tasks are running
task <thread 6> done
<thread 6> is finished. 2 tasks are running
3 tasks are running
task <thread 8> done
<thread 8> is finished. 2 tasks are running
task <thread 3> done
<thread 3> is finished. 1 tasks are running
task <thread 9> done
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_small_stack (test.test_threading.ThreadTests) ... with
256kB thread
stack size...
task <thread 0> will run for 89.5 usec
task <thread 1> will run for 97.8 usec
1 tasks are running
task <thread 2> will run for 56.9 usec
task <thread 3> will run for 3.4 usec
2 tasks are runningtask <thread 4> will run for 49.2 usec
3 tasks are running
task <thread 5> will run for 57.9 usec
task <thread 6> will run for 39.1 usec
task <thread 7> will run for 62.6 usec
task <thread 0> done
task <thread 0> is finished. 2 tasks are running
<thread 1> done
task <thread 8> will run for 26.7 usec
<thread 1> is finished. 1 tasks are running
waiting for all tasks to complete
task <thread 3>2 tasks are running
done
task <thread 9> will run for 65.1 usec
3 tasks are running
<thread 3> is finished. 2 tasks are running
3 tasks are running
task <thread 8> done
task <thread 9> done
<thread 8> is finished. 2 tasks are running
<thread 9> is finished. 1 tasks are running
2 tasks are running
3 tasks are running
task <thread 5> done
<thread 5> is finished. 2 tasks are running
3 tasks are running
task <thread 6> done
<thread 6> is finished. 2 tasks are running
3 tasks are running
task <thread 7> donetask <thread 2> done
<thread 2> is finished. 2 tasks are running
<thread 7> is finished. 1 tasks are running
task <thread 4> done
<thread 4> is finished. 0 tasks are running
all tasks done
ok
test_1_join_on_shutdown (test.test_threading.ThreadJoinOnShutdown) ... ok
test_2_join_in_forked_process
(test.test_threading.ThreadJoinOnShutdown) ... ok
test_3_join_in_forked_from_thread
(test.test_threading.ThreadJoinOnShutdown) ... ^C
275 tests OK.
2 tests failed:
test_io test_locale
36 tests skipped:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/rptlab/UNLADEN/lib/python2.6/threading.py", line 537, in
__bootstrap_inner
self.run()
File "/home/rptlab/UNLADEN/lib/python2.6/threading.py", line 492, in run
self.__target(*self.__args, **self.__kwargs)
File "<string>", line 14, in worker
OSError: [Errno 4] Interrupted system call
Exception KeyboardInterrupt in <module 'threading' from
'/home/rptlab/UNLADEN/lib/python2.6/threading.pyc'> ignored
test__locale test_aepack test_al test_applesingle test_bsddb185
test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
test_dl test_epoll test_gdbm test_gl test_imageop test_imgfile
test_jit_gdb test_lib2to3 test_linuxaudiodev test_macos
test_macostools test_multiprocessing test_normalization
test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages
test_socketserver test_sqlite test_startfile test_sunaudiodev
test_tcl
4 skips unexpected on freebsd7:
test__locale test_lib2to3 test_dl test_bsddb185
~/UNLADEN/lib/python2.6/test:
$ exit
Script done, output file is /tmp/ttt
~/UNLADEN/lib/python2.6/test:
$
Not sure if this is relevant and it's something I have completely missed
building on
this machine is that this freebsd is actually a 64 bit version
ie uname -a
FreeBSD app3.reportlab.com 7.1-RELEASE FreeBSD 7.1-RELEASE #0: Thu Jan 1
08:58:24
UTC 2009 ro...@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
amd64
I have tried rebuilding on a 32bit x386 FreeBSD 8.0 machine with gcc44 and
get much
the same outcome ie module _locale fails to import (for the same reason).
However, I
don't get any hangs in test_threading so the amd64 has something to do with
that. My
freebsd x386 test_all.py ends with
OK
322 tests OK.
3 tests failed:
test_io test_locale test_smtplib
39 tests skipped:
test__locale test_aepack test_al test_applesingle test_bsddb185
test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
test_epoll test_gl test_imgfile test_jit_gdb test_lib2to3
test_linuxaudiodev test_macos test_macostools test_multiprocessing
test_normalization test_ossaudiodev test_pep277 test_py3kwarn
test_scriptpackages test_socketserver test_startfile
test_sunaudiodev test_tcl test_timeout test_unicode_file
test_urllib2net test_urllibnet test_winreg test_winsound
test_zipfile64
3 skips unexpected on freebsd8:
test__locale test_lib2to3 test_bsddb185
Did a bit of checking with the x386 FreeBSD-8.0 system. Python-2.6 builds
_locale.so
OK and doesn't seem to be using libintl.
pyconfig.h:/* #undef HAVE_LIBINTL_H */
pyconfig.h:/* #undef WITH_LIBINTL */
also no references in the .so file.
The unladen build of _locale does seem to produce dyn so which does have
references
to libintl, but for whatever reason the unladen pyconfig doesn't define
WITH_LIBINTL
either the unladen pyconfig contains
#define HAVE_LIBINTL_H 1
/* #undef WITH_LIBINTL */
looking in the ouput of the configure shows
checking for textdomain in -lintl... no
so perhaps the test is failing ie do I need more than
export CC=gcc44 CXX=g++44
to make the configure work?