Build fails on RHEL4 - gnutls

[gedaliah]

This looks like the relevant portion of the log --

if /bin/sh /local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/
src/build-aux/missing --run makeinfo -I ../doc -I . \
-o gnutls.info gnutls.texi; \
then \
rc=0; \
cd .; \
else \
rc=$?; \
cd . && \
$restore $backupdir/* `echo "./gnutls.info" | sed 's|[^/]*$||'`; \
fi; \
rm -rf $backupdir; exit $rc
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
signatures.texi:14: Unknown command `euro'.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
signatures.texi:14: Misplaced {.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
signatures.texi:14: Misplaced }.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
signatures.texi:15: Unknown command `euro'.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
signatures.texi:15: Misplaced {.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
signatures.texi:15: Misplaced }.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
pgp-api.texi:143: warning: unlikely character [ in @var.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
pgp-api.texi:143: warning: unlikely character ] in @var.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
pgp-api.texi:279: warning: unlikely character [ in @var.
/local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
pgp-api.texi:279: warning: unlikely character ] in @var.
makeinfo: Removing output file `gnutls.info' due to errors; use --
force to preserve.
make[6]: *** [gnutls.info] Error 1
make[6]: Leaving directory `/local/ftp/linux/sage/sage-3.2.3/spkg/
build/gnutls-2.2.1.p1/src/doc'
make[5]: *** [all-recursive] Error 1
make[5]: Leaving directory `/local/ftp/linux/sage/sage-3.2.3/spkg/
build/gnutls-2.2.1.p1/src/doc'
make[4]: *** [all] Error 2
make[4]: Leaving directory `/local/ftp/linux/sage/sage-3.2.3/spkg/
build/gnutls-2.2.1.p1/src/doc'

****************************************************
Host system
uname -a:
Linux xxxxxxx.njit.edu 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20
EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
****************************************************
****************************************************
GCC Version
gcc -v
Reading specs from /usr/lib/gcc/x86_64-redhat-linux/3.4.6/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --
infodir=/usr/share/info --enable-shared --enable-threads=posix --
disable-checking --w
Thread model: posix
gcc version 3.4.6 20060404 (Red Hat 3.4.6-10)

Let me know if you need anything else.

Thanks

[mabshoff]

On Feb 9, 9:09 am, gedaliah <gwol...@gmail.com> wrote:

Hi,

> This looks like the relevant portion of the log --
>
> if /bin/sh /local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/
> src/build-aux/missing --run makeinfo -I ../doc  -I . \
>  -o gnutls.info gnutls.texi; \
> then \
>   rc=0; \
>   cd .; \
> else \
>   rc=$?; \
>   cd . && \
>   $restore $backupdir/* `echo "./gnutls.info" | sed 's|[^/]*$||'`; \
> fi; \
> rm -rf $backupdir; exit $rc
> /local/ftp/linux/sage/sage-3.2.3/spkg/build/gnutls-2.2.1.p1/src/doc//
> signatures.texi:14: Unknown command `euro'.

Texi doesn't know about the Euro.

<SNIP>

> GCC Version
> gcc -v
> Reading specs from /usr/lib/gcc/x86_64-redhat-linux/3.4.6/specs
> Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --
> infodir=/usr/share/info --enable-shared --enable-threads=posix --
> disable-checking --w
> Thread model: posix
> gcc version 3.4.6 20060404 (Red Hat 3.4.6-10)
>
> Let me know if you need anything else.

I have seen the above error reported once way back and it was also
RHEL4. As mentioned above Texi doesn't know what the Euro is - tsk
tsk.

Anyway, so more info on your system: How current a RHEL 4 system is
it? I have a RHEL 4.6 or 4.7 vmaware image, but I have never broken it
out to test the build of Sage on it. We could add it to our testing
lineup if there was sufficient demand.

We could also probably fix the issue by having gnutls not build any
Texi documentation since I don't think anyone would miss that. Another
possibility would be to edit the afftected file and throw out the
macros that cause the problems and replace them by something else.

> Thanks

Cheers,

Michael

[gedaliah]

On Feb 9, 4:41 pm, mabshoff <mabsh...@googlemail.com> wrote:

<SNIP>

> Anyway, so more info on your system: How current a RHEL 4 system is
> it? I have a RHEL 4.6 or 4.7 vmaware image, but I have never broken it
> out to test the build of Sage on it. We could add it to our testing
> lineup if there was sufficient demand.
>

RHEL 4.7

> We could also probably fix the issue by having gnutls not build any
> Texi documentation since I don't think anyone would miss that. Another
> possibility would be to edit the afftected file and throw out the
> macros that cause the problems and replace them by something else.
>

Edited out the euro macro and got past the gnutls. Now failing at
atlas. Gzipped install log can be downloaded http://web.njit.edu/~gwolosh/install.log.gz.

Thanks


> > Thanks
>
> Cheers,
>
> Michael

[mabshoff]

On Feb 10, 10:39 am, gedaliah <gwol...@gmail.com> wrote:
> On Feb 9, 4:41 pm, mabshoff <mabsh...@googlemail.com> wrote:
>
> <SNIP>

Hi,

> > Anyway, so more info on your system: How current a RHEL 4 system is
> > it? I have a RHEL 4.6 or 4.7 vmaware image, but I have never broken it
> > out to test the build of Sage on it. We could add it to our testing
> > lineup if there was sufficient demand.
>
> RHEL 4.7

Sounds current enough to me :)

> > We could also probably fix the issue by having gnutls not build any
> > Texi documentation since I don't think anyone would miss that. Another
> > possibility would be to edit the afftected file and throw out the
> > macros that cause the problems and replace them by something else.
>
> Edited out the euro macro and got past the gnutls. Now failing at
> atlas. Gzipped install log can be downloadedhttp://web.njit.edu/~gwolosh/install.log.gz.

You got

"res/ssymvN_5000_0 : VARIATION EXCEEDS TOLERENCE, RERUN WITH HIGHER
REPS."

So just restart the build via "make" do retune ATLAS. This is a long
standing annoyance that if the ATLAS tuning fails we do not restart
automatically, but it is high on my list of things to fix :)

> Thanks

Keep me in the loop about any other problems you see. If the gnutls
problem is the only one preventing us from building on RHEL 4.7 this
should be trivial to fix to add official support for RHEL 4.

Cheers,

Michael

[gedaliah]

On Feb 10, 4:48 pm, mabshoff <mabsh...@googlemail.com> wrote:
<SNIP>

>

> Keep me in the loop about any other problems you see. If the gnutls
> problem is the only one preventing us from building on RHEL 4.7 this
> should be trivial to fix to add official support for RHEL 4.

Got past atlas -- fails for singular. Gzipped install log can be
downloaded http://web.njit.edu/~gwolosh/install.log.gz.

Thanks

>
> Cheers,
>
> Michael

[mabshoff]

On Feb 11, 7:40 am, gedaliah <gwol...@gmail.com> wrote:
> On Feb 10, 4:48 pm, mabshoff <mabsh...@googlemail.com> wrote:
> <SNIP>

Hi,

> > Keep me in the loop about any other problems you see. If the gnutls
> > problem is the only one preventing us from building on RHEL 4.7 this
> > should be trivial to fix to add official support for RHEL 4.
>
> Got past atlas -- fails for singular. Gzipped install log can be
> downloadedhttp://web.njit.edu/~gwolosh/install.log.gz.

This one puzzles me since it basically all goes well until the final
linking stage:

g++ -I. -I../kernel -I/afs/cad/u/g/w/gwolosh/ftp/linux/sage/
sage-3.2.3/local/include -I. -I../kernel -I/afs/cad/u/g/w/gwolosh/ftp/
linux/sage/sage-3.2.3/local/include
-I/afs/cad/u/g/w/gwolosh/ftp/linux/sage/sage-3.2.3/local/include -I/
afs/cad/u/g/w/gwolosh/ftp/linux/sage/sage-3.2.3/local/include -
DSTANDALONE_PARSER -o libparse li
bparse_main.o utils.o ../kernel/fegetopt.o -L/afs/cad/u/g/w/gwolosh/
ftp/linux/sage/sage-3.2.3/local/lib -ldl -rdynamic -L../kernel -
lkernel -L/afs/cad/u/g/w/gwolosh/f
tp/linux/sage/sage-3.2.3/local/lib -L/afs/cad/u/g/w/gwolosh/ftp/linux/
sage/sage-3.2.3/local/lib -lomalloc_ndebug
../kernel/libkernel.a(mmstd.o)(.text+0x76): In function `calloc':
/afs/cad/u/g/w/gwolosh/ftp/linux/sage/sage-3.2.3/local/include/
omalloc.c:32: undefined reference to `omMarkAsStaticAddr'
../kernel/libkernel.a(mmstd.o)(.text+0xa0):/afs/cad/u/g/w/gwolosh/ftp/
linux/sage/sage-3.2.3/local/include/omalloc.c:32: undefined reference
to `omMarkAsStaticAddr'
../kernel/libkernel.a(mmstd.o)(.text+0xe9): In function `malloc':
/afs/cad/u/g/w/gwolosh/ftp/linux/sage/sage-3.2.3/local/include/
omalloc.c:42: undefined reference to `omMarkAsStaticAddr'
../kernel/libkernel.a(mmstd.o)(.text+0x113):/afs/cad/u/g/w/gwolosh/ftp/
linux/sage/sage-3.2.3/local/include/omalloc.c:42: undefined reference
to `omMarkAsStaticAddr'
../kernel/libkernel.a(mmstd.o)(.text+0x3f6): In function `realloc':
/afs/cad/u/g/w/gwolosh/ftp/linux/sage/sage-3.2.3/local/include/
omalloc.c:77: undefined reference to `omMarkAsStaticAddr'
../kernel/libkernel.a(mmstd.o)(.text+0x4d6):/afs/cad/u/g/w/gwolosh/ftp/
linux/sage/sage-3.2.3/local/include/omalloc.c:99: more undefined
references to `omMarkAsStaticA
ddr' follow
../kernel/libkernel.a(mmstd.o)(.text+0x50d): In function `freeSize':
/afs/cad/u/g/w/gwolosh/ftp/linux/sage/sage-3.2.3/local/include/
omalloc.c:107: undefined reference to `_omDebugFree'
collect2: ld returned 1 exit status
make[4]: *** [libparse] Error 1
make[4]: Leaving directory `/afs/cad.njit.edu/u/g/w/gwolosh/ftp/linux/
sage/sage-3.2.3/spkg/build/singular-3-0-4-4-20080711.p2/src/Singular'
make[3]: *** [install] Error 1
make[3]: Leaving directory `/afs/cad.njit.edu/u/g/w/gwolosh/ftp/linux/
sage/sage-3.2.3/spkg/build/singular-3-0-4-4-20080711.p2/src'
make[2]: *** [/afs/cad/u/g/w/gwolosh/ftp/linux/sage/sage-3.2.3/local/
bin/Singular-3-0-4] Error 2
make[2]: Leaving directory `/afs/cad.njit.edu/u/g/w/gwolosh/ftp/linux/
sage/sage-3.2.3/spkg/build/singular-3-0-4-4-20080711.p2/src'
Unable to build Singular.

There are some other odd things in the logs, i.e compile warnings I
have never seen in that form before for Singular. Since the Singular
build is about 4000 lines of log this might take some time to figure
out.

In the past we have seen strange things happen on afs. I don't want to
point my finger at it, but would it be possible to build Sage in /tmp
on such a box? If it isn't that I would guess I need to break out my
RHEL 4.7 VMWare image and try to debug this.

If you don't hear back from me in the next couple days just bump the
thread.

Cheers,

Michael

[gedaliah]

On Feb 11, 12:56 pm, mabshoff <mabsh...@googlemail.com> wrote:
> On Feb 11, 7:40 am, gedaliah <gwol...@gmail.com> wrote:
>
> > On Feb 10, 4:48 pm, mabshoff <mabsh...@googlemail.com> wrote:
> > <SNIP>
>
> Hi,
>
> > > Keep me in the loop about any other problems you see. If the gnutls
> > > problem is the only one preventing us from building on RHEL 4.7 this
> > > should be trivial to fix to add official support for RHEL 4.
>
> > Got past atlas -- fails for singular. Gzipped install log can be
> > downloadedhttp://web.njit.edu/~gwolosh/install.log.gz.
>
> This one puzzles me since it basically all goes well until the final
> linking stage:
>

<SNIP>

>
> There are some other odd things in the logs, i.e compile warnings I
> have never seen in that form before for Singular. Since the Singular
> build is about 4000 lines of log this might take some time to figure
> out.
>
> In the past we have seen strange things happen on afs. I don't want to
> point my finger at it, but would it be possible to build Sage in /tmp
> on such a box? If it isn't that I would guess I need to break out my
> RHEL 4.7 VMWare image and try to debug this.
>

I can try to build it locally on the machine but ultimately, sage will
need to live in AFS land. I understand there are problems moving it.

> If you don't hear back from me in the next couple days just bump the
> thread.
>

Thanks, I appreciate the help.

Gedaliah


> Cheers,
>
> Michael

[Carl Witty]

On Wed, Feb 11, 2009 at 10:05 AM, gedaliah <gwo...@gmail.com> wrote:
> I can try to build it locally on the machine but ultimately, sage will
> need to live in AFS land. I understand there are problems moving it.

There have been (and probably still are) problems with moving Sage,
but if you find any, please report it as a bug and it will eventually
get fixed.

Actually, many (most?) users "move" sage, because they use a binary
distribution that gets unpacked into a different directory from where
it was built, so AFAIK moving sage mostly works.

Carl

[mabshoff]

On Feb 11, 10:48 am, Carl Witty <carl.wi...@gmail.com> wrote:

> On Wed, Feb 11, 2009 at 10:05 AM, gedaliah <gwol...@gmail.com> wrote:
> > I can try to build it locally on the machine but ultimately, sage will
> > need to live in AFS land. I understand there are problems moving it.
>
> There have been (and probably still are) problems with moving Sage,

I am not aware of any problem when moving Sage on Linux. There is some
corner case on OSX 10.4 only when rebuilding certain spkgs after
moving a Sage install without rebuilding one specific spkg since it
does dumb things while hard coding some include path and the OSX 10.4
ld makes things worst. Other than that everything just works.

> but if you find any, please report it as a bug and it will eventually
> get fixed.
>
> Actually, many (most?) users "move" sage, because they use a binary
> distribution that gets unpacked into a different directory from where
> it was built, so AFAIK moving sage mostly works.

Yes, afs is a strange filessystem and might be the root cause of your
trouble, but that is far from certain at this point.

> Carl

Cheers,

Michael

[gedaliah]

On Feb 11, 1:54 pm, mabshoff <mabsh...@googlemail.com> wrote:

>
> Yes, afs is a strange filessystem and might be the root cause of your
> trouble, but that is far from certain at this point.

No longer far from certain. The build completed without ANY problems,
including getting past gnutls without error. This is not unprecedented
but somewhat surprising nevertheless. Running make test now.

Gedaliah

> Cheers,
>
> Michael

[mabshoff]

On Feb 11, 1:03 pm, gedaliah <gwol...@gmail.com> wrote:
> On Feb 11, 1:54 pm, mabshoff <mabsh...@googlemail.com> wrote:

Hi Gedaliah,

> > Yes, afs is a strange filessystem and might be the root cause of your
> > trouble, but that is far from certain at this point.
>
> No longer far from certain. The build completed without ANY problems,
> including getting past gnutls without error. This is not unprecedented
> but somewhat surprising nevertheless.

Well, any of the really fancy "Enterprise class" filesystems are
usually less well debugged than ext3+NFS :). And even NFS causes
enough trouble that I my first reaction in case of odd failures when a
network filesystem is involved is to move to /tmp to build to isolate
the network file system from the equation. I have build Sage on one
SLES 9 Itanium box where I couldn't even start Sage on some fancy
enterprise class SAN file system which shall remain nameless since I
don't remember all the details 100%.

Anyway, once the test pass run

./sage -bdist 3.2.3-RHEL4

and look in $SAGE_ROOT/dist afterwards. Untar that tallball into your
$HOME somewhere and start Sage once to assure that it starts (it will
also rewrite a bunch of pyc and la files) and run make check again to
assure that AFS isn't breaking some doctest. In that case you should
report it so we can attempt to fix or work around it.

Once you want to upgrade the current Sage install take the original
binary dist tarball, unpack it in temp, run it once to get everything
rewritten in case it needs to, run "./sage -upgrade", run -bdist and
lather, rinse, repeat ;).

Let me know if you run into any more trouble.

> Running make test now.

Excellent. I have made this

#5235 (Detect when Sage is build on AFS and issue a warning)

In case you have a shell code snipped that identifies that the current
working directory is on an AFS mount I would be happy to integrate it.
Given that this is the second independent report of the problem (both
on RHEL 4.x no less) I assume that others have failed and never
bothered to report the issue.

> Gedaliah

Cheers,

Michael

[gedaliah]

On Feb 11, 4:24 pm, mabshoff <mabsh...@googlemail.com> wrote:


>
> Anyway, once the test pass run

Oops, book_stein_ent.py test failed, not sure if this is a problem or
not --

Trying:
qsieve(n)###line 289:_sage_ : qsieve(n)
Expecting:
([6340271405786663791648052309,
46102313108592180286398757159], '')
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***
*** *** Error: TIMED OUT! *** ***
[360.3 s]
exit code: 1024

----------------------------------------------------------------------
The following tests failed:


sage -t --verbose "-3.2.3/devel/sage/sage/tests/book_stein_ent.py"
Total time for all tests: 360.3 seconds

>
>  ./sage -bdist 3.2.3-RHEL4
>
> and look in $SAGE_ROOT/dist afterwards. Untar that tallball into your
> $HOME somewhere and start Sage once to assure that it starts (it will
> also rewrite a bunch of pyc and la files) and run make check again to
> assure that AFS isn't breaking some doctest. In that case you should
> report it so we can attempt to fix or work around it.
>
> Once you want to upgrade the current Sage install take the original
> binary dist tarball, unpack it in temp, run it once to get everything
> rewritten in case it needs to, run "./sage -upgrade", run -bdist and
> lather, rinse, repeat ;).

Will do

>
> Let me know if you run into any more trouble.
>
> > Running make test now.
>
> Excellent. I have made this
>
>  #5235 (Detect when Sage is build on AFS and issue a warning)
>
> In case you have a shell code snipped that identifies that the current
> working directory is on an AFS mount I would be happy to integrate it.

This will work unless somebody very foolishly changed the afs mount
point to something other that /afs.

[[ $(pwd | cut -d'/' -f2) = 'afs' ]] && echo "we are in afs"

[mabshoff]

On Feb 11, 2:37 pm, gedaliah <gwol...@gmail.com> wrote:
> On Feb 11, 4:24 pm, mabshoff <mabsh...@googlemail.com> wrote:
>
>
>
> > Anyway, once the test pass run
>
> Oops, book_stein_ent.py test failed, not sure if this is a problem or
> not --
>
> Trying:
>     qsieve(n)###line 289:_sage_    : qsieve(n)
> Expecting:
>     ([6340271405786663791648052309,
>       46102313108592180286398757159], '')
> *** *** Error: TIMED OUT! PROCESS KILLED! *** ***
> *** *** Error: TIMED OUT! *** ***
>          [360.3 s]
> exit code: 1024
>
> ----------------------------------------------------------------------
> The following tests failed:
>
>         sage -t --verbose "-3.2.3/devel/sage/sage/tests/book_stein_ent.py"
> Total time for all tests: 360.3 seconds

This is a known problem once observed by Alex Ghitza, but it never
made it into trac, so this is now

http://trac.sagemath.org/sage_trac/ticket/5237

The code in question is old, i.e. a pre-release snapshot of qsieve
from 2007 that has been replaced by a much better and faster version
in FLINT 1.0.x. (we ship FLINT 1.1 and will ship FLINT 1.1.1 in Sage
3.3.rc1). We even compile the new sieve code, we just don't copy it
into $SAGE_LOCAL/bin. Bill Hart has explained to me a couple times why
the new code isn't used from Sage, but I pinged him offlist to get it
in writing this time so I can open a ticket to switch to the new
code.

<SNIP>

> > In case you have a shell code snipped that identifies that the current
> > working directory is on an AFS mount I would be happy to integrate it.
>
> This will work unless somebody very foolishly changed the afs mount
> point to something other that /afs.

Well, given what I have seen many people do foolish things I not
longer assume people do sane things per default any more. But a test
that catches the problem 99% of the time is better than a perfect test
never written and merged :)

> [[ $(pwd | cut -d'/' -f2) = 'afs' ]] && echo "we are in afs"

Thanks, I have added that info to the ticket. It will probably not
make it into 3.3, but we will see.

Cheers,

Michael

[mabshoff]

On Feb 11, 2:59 pm, mabshoff <mabsh...@googlemail.com> wrote:
> On Feb 11, 2:37 pm, gedaliah <gwol...@gmail.com> wrote:

<SNIP>

Bill says:
[quote]
FLINT ships with a much improved version of qsieve. It is now called
mpQS.

If you do make mpQS in the main directory of FLINT (make all also
builds it) it will build a program which replaces the old qsieve.
However this program will deal with much smaller integers, and is much
faster overall.

It should have far fewer issues than the old code.

It also handles numbers with small factors, but still won't handle
integers which are perfect powers or primes. These should be scanned
for before running mpQS.

The new program actually uses FLINT for some parts of the computation,
so it cannot be built standalone (it doesn't link against libflint, it
just includes the files it needs). I have just verified this program
still builds (and works) on sage.math.

Bill.
[end quote]

This is now http://trac.sagemath.org/sage_trac/ticket/5238

Cheers,

Michael