Build of ECL breaks for parallel builds.
Dr. David Kirkby
parallel, so the build goes wrong
Here it is on a Sun Ultra 27 with a quad core hyper-threaded (8 threads) Xeon
processor. I've set 12 threads running, using 'make -j 12'.
This is on OpenSolaris, but it's clearly been experienced with other operating
systems before, as the installation file in Sage (spkg-install), specifically
disables parallel builds
I get a similar failure with only 2 threads.
http://trac.sagemath.org/sage_trac/ticket/9187
is a patch for Sage which sets
MAKEFLAGS=
export MAKEFLAGS
as the current method of disabling parallel builds in Sage is not working.
Of course, the simple answer is to just accept this and disable parallel builds.
In the short term, this seems most logical. But in the longer term, it would be
nice if the issue could be resolved, as ECL is one of the packages in Sage which
takes a significant time to build. It is along with ATLAS and Maxima take quite
some time to build.
(As a note to Sage developers, we should really take a look at what packages
take a long time to build, and see if they can be speeded up by building in
parallel. Simply setting "$MAKE=make" to disable parallel builds is a hack)
Dave
Libraries have been installed in:
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- use the `-RLIBDIR' linker flag
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
/usr/bin/ginstall -c -m 644
'/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/gc/doc/porting.html'
'/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build/share/gc/porting.html'
make[4]: Leaving directory
`/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build/gc'
make[3]: Leaving directory
`/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build/gc'
make[2]: Leaving directory
`/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build/gc'
cd c; make
make[2]: Entering directory
`/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build/c'
cat
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/symbols_list.h
| \
sed -e 's%{\([A-Z ]*.*".*"\),[^,]*,[ ]*NULL,.*}%{\1,NULL}%g' \
-e 's%{\([A-Z ]*.*".*"\),[^,]*,[ ]*\([^,]*\),.*}%{\1,"\2"}%g' \
-e 's%{NULL.*%{NULL,NULL}};%' >
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/symbols_list2.h
if test -f ../CROSS-DPP ; then ../CROSS-DPP
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/main.d tmp.c
; else ./dpp
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/main.d tmp.c
; fi
if test -f ../CROSS-DPP ; then ../CROSS-DPP
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/symbol.d
tmp.c ; else ./dpp
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/symbol.d
tmp.c ; fi
if test -f ../CROSS-DPP ; then ../CROSS-DPP
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/package.d
tmp.c ; else ./dpp
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/package.d
tmp.c ; fi
if test -f ../CROSS-DPP ; then ../CROSS-DPP
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/list.d tmp.c
; else ./dpp
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/list.d tmp.c
; fi
if test -f ../CROSS-DPP ; then ../CROSS-DPP
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/apply.d
apply.c ; else ./dpp
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/apply.d
apply.c ; fi
if test -f ../CROSS-DPP ; then ../CROSS-DPP
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/eval.d tmp.c
; else ./dpp
/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/src/c/eval.d tmp.c
; fi
/bin/sh: line 1: ./dpp: not found
/bin/sh: line 1: ./dpp: not found
/bin/sh: line 1: ./dpp: not found
/bin/sh: line 1: ./dpp: not found
make[2]: *** [package.o] Error 127/bin/sh: line 1: ./dpp: not found
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [symbol.o] Error 127
make[2]: *** [main.o] Error 127
make[2]: *** [apply.c] Error 127
make[2]: *** [list.o] Error 127
/bin/sh: line 1: ./dpp: not found
make[2]: *** [eval.o] Error 127
make[2]: Leaving directory
`/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build/c'
make[1]: *** [libeclmin.a] Error 2
make[1]: Leaving directory
`/export/home/drkirkby/sage-4.4.4.alpha1/spkg/standard/ecl-10.4.1/src/build'
make: *** [all] Error 2
Juan Jose Garcia-Ripoll
If one attempts to build ECL 10.4.1 (latest available stable release) in
parallel, so the build goes wrong
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
Dr. David Kirkby
> On Fri, Jun 18, 2010 at 6:12 PM, Dr. David Kirkby
> <david....@onetel.net>wrote:
>
>> If one attempts to build ECL 10.4.1 (latest available stable release) in
>> parallel, so the build goes wrong
>
>
> Yes, this is known, but I do not have time / skills to track the
> dependencies that break this. In general I run away from parallel builds for
> my software because they cause headaches, forcing one to really sort the
> dependency graph manually so as not to break anything.
OK, thank you. It would be worth adding something to the Makefile to stop
parallel builds in that case. I think you can add something like ".NOTPARALLEL",
though I don't know exactly how to do it.
> Incidentally you complain about ECL's long build time, but parallel build
> will NOT help you. Since in Sage you already use a previously built GMP
> library (or whatever variant) the largest chunk of time is spent in running
> ECL's compiler, a process that CAN NOT be parallelized, due to delicate
> dependencies in bootstrapping code.
OK, fair enough.
> In any case I am surprised to read that ECL's build time is the bottleneck.
> On my laptop it takes just 1+2 minutes to build it and it is a slow OS X
> machine (32 bits). What build flags are you using?
>
> Juanjo
ECL is certainly not the longest item to build, but it is far from the shortest
either. There are no flags other than --prefix.
It takes 10 minutes on an old 900 MHz Sun Blade 1000. In comparison, many
packages take under a minute.
* Flint 1 minute
* Readline 2 minutes
* GD 2 minutes.
* Matplotlib 4 minutes
* Cython 4 minutes
* Gap 5 minutes
* Lapack 6 minutes.
* ATLAS 8 minutes
* MPIR 9 minutes
* ECL 10 minutes
* MPFR 10 minutes
* NTL 12 minutes
* Python 28 minutes.
* Singular 40 minutes.
On my Sun Ultra 27 (3.33 GHz Xeon) ECL takes 1 minute and 25 seconds to build,
so it is not as painful there!
Dave