request for testing of new flint.spkg
mabshoff
Bill has fixed a couple of bugs in flint since r1072 that were corner
cases that only happened on Core Duos, so I have updated the spkg to
r1075. It is available at
http://sage.math.washington.edu/home/mabshoff/flint-0.9-r1075.spkg
It runs the spkg-check script per default - this will be deactivated
in the final 2.8.14 release. We do this because Bill Hart needs
feedback and extended testing for FLINT that the Sage doctests cannot
provide. The tests take about 15-30 minutes to run (15 on sage.math,
30 in my laptop). Please report back whether the tests failed or
succeeded, together with info about CPU, operating system and
compiler. Bill: is there a standard format you would like?
Cheers,
Michael
One more thing: Bill, when I checked out trunk, i.e. r1086 compilation
fails with:
gcc -std=c99 -I/tmp/Work-mabshoff/release-cycles-2.8.14/
sage-2.8.14.rc0/local/include/ -I/tmp/Work-mabshoff/release-
cycles-2.8.14/sage-2.8.14.rc0/local/include -I/tmp/Work-mabshoff/
release-cycles-2.8.14/sage-2.8.14.rc0/local/include -funroll-loops -
fexpensive-optimizations -mtune=opteron -march=opteron -fPIC -funroll-
loops -O3 -o delta_qexp delta_qexp.o mpn_extras.o mpz_extras.o memory-
manager.o ZmodF.o ZmodF_mul.o ZmodF_mul-tuning.o fmpz.o fmpz_poly.o
mpz_poly-tuning.o mpz_poly.o ZmodF_poly.o long_extras.o -L/tmp/Work-
mabshoff/release-cycles-2.8.14/sage-2.8.14.rc0/local/lib/ -L/tmp/Work-
mabshoff/release-cycles-2.8.14/sage-2.8.14.rc0/local/lib/ -L/tmp/Work-
mabshoff/release-cycles-2.8.14/sage-2.8.14.rc0/local/include -lgmp -
lpthread -lm -lntl
make: *** No rule to make target `BLTcubes.c', needed by `BLTcubes'.
Stop.
Error building flint shared library.
Since there are some modifications to the makefiles in the spkg (it
seems that we add targets for OSX) we might want to merge those
changes back into trunk. I will check and ping the person who wrote
the makefile.
Bill Hart
There's a memory leak in _fmpz_poly_reverse, which is used in all the
newton division functions. So machines with small quantities of RAM
might abort because of that. It's also possible they might get stuck
on fmpz_poly_div_newton or earlier because of this. But these issues
are fixed in the latest revision and I'd appreciate any test feedback
people can give me with the current version you've packaged despite
these known faults. On many systems the memory leaks won't be severe
enough to cause any problems. If you do want to fix it you can just
take the current version of the function _fmpz_poly_reverse in the
file fmpz_poly.c from HEAD in SVN and put it into the rev 1075 code
that you have.
In the latest revision (HEAD) I've completely redone the FLINT build
system, so you will need to move the new makefile up into SAGE as the
old one won't work any more. Before running the makefile, you now have
to source flint_env (or replace it with a SAGE specific thing which
does the same thing). Basically what I've done is take the existing
SAGE lib_makefile and rewritten it to automate some things,
distributing bits of it into the FLINT makefile and the rest into
flint_env. But I've also removed some SAGE specific stuff that isn't
needed in FLINT itself. Naturally you can still override all that as
you were doing before, but there should be much less to do now in
lib_makefile.
One of the concerning things is that making FLINT into a shared
library with -fPIC incurs upto a 20% speed penalty. Ouch!!
As we'll be adding more tuning information and platform specific stuff
soon, it might be worth letting the FLINT build system do as much work
as possible otherwise you'll be continuously rewriting lib_makefile
for FLINT. At present all I've done is add some tuning details for the
G5 but there'll be more soon enough.
BTW, I've ripped off some stuff directly from lib_makefile, but I
didn't know who to credit it to. If you let me know I'll add the
appropriate copyright notices to the new FLINT build files. Basically
some of that stuff has now been moved upstream into FLINT. Many thanks
to whoever wrote that.
By the way, as of the most recent revision of FLINT, the SAGE trac
ticket related to u_int32_t etc, should be able to be closed. However
I haven't fixed this in the old FLINT-QS, since I plan to replace it
with the new FLINT-QS, which is part of FLINT instead of being
distributed separately. I'm not 100% sure that the new sieve actually
works on all systems yet, but it's going to work very soon if it
doesn't already, meaning we can replace the old FLINT-QS in SAGE with
the new one.
By the way, I added a function for cleaning up after the stack based
memory manager in FLINT after SAGE is done with it. It also audits the
stack and complains if a problem was found. This is how I discovered
the hitherto unnoticed memory leak!
Bill.
On 24 Nov, 03:46, mabshoff <Michael.Absh...@fsmath.mathematik.uni-
Bill Hart
something is wrong. Is it linking against the right GMP?
Bill Hart
psychedelic patterns on your screen etc, either post them here or
email me for a FLINT trac account.
Please let me know what your system is and what the OS is and which
test caused the fault. If FLINT diagnostic information gets printed,
that could be helpful (if it isn't screens full).
I'd very much like to hear about passes as well as failures.
Thanks everyone.
Bill.
Bill Hart
test_fmpz_poly_div_newton. It's supposed to take 3s! The problem is
because of the memory leak in _fmpz_poly_reverse. Perhaps we better
fix this before people do any serious testing if that is OK. I hadn't
noticed it since I test with the debugging memory manager on, which
ironically doesn't pick up memory leaks and stops this problem from
manifesting.
To fix the problem, simply insert the following line between lines
1485 and 1486 of fmpz_poly.c (i.e. make it the third last line of the
function _fmpz_poly_reverse):
flint_stack_release();
Bill.
mabshoff
On Nov 24, 6:26 am, Bill Hart <goodwillh...@googlemail.com> wrote:
> No, you are correct. The tests spend many minutes doing
> test_fmpz_poly_div_newton. It's supposed to take 3s!
scratch because I had that 6 minute figure in mmy head.
>The problem is
> because of the memory leak in _fmpz_poly_reverse. Perhaps we better
> fix this before people do any serious testing if that is OK. I hadn't
> noticed it since I test with the debugging memory manager on, which
> ironically doesn't pick up memory leaks and stops this problem from
> manifesting.
>
> To fix the problem, simply insert the following line between lines
> 1485 and 1486 of fmpz_poly.c (i.e. make it the third last line of the
> function _fmpz_poly_reverse):
>
> flint_stack_release();
See, automated testing benefits everybody ;)
>
> Bill.
Cheers,
Michael
mabshoff
http://sage.math.washington.edu/home/mabshoff/flint-0.9-r1075.p0.spkg
I now get
All tests passed
real 6m38.729s
user 6m33.477s
sys 0m5.220s
on sage.math
Cheers,
Michael
Bill Hart
Thanks Michael.
Bill.
On 24 Nov, 10:17, mabshoff <Michael.Absh...@fsmath.mathematik.uni-
Jaap Spies
> Cool. looking forward to results from other platforms.
>
Fedora 7: Linux paix 2.6.23.1-21.fc7 #1 SMP Thu Nov 1 21:09:24 EDT 2007 i686 i686 i386 GNU/Linux
gcc -v:
Using built-in specs.
Target: i386-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release
--with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk
--disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode
--with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-cpu=generic --host=i386-redhat-linux
Thread model: posix
gcc version 4.1.2 20070925 (Red Hat 4.1.2-27)
[jaap@paix python2.4]$ gcc --version
gcc (GCC) 4.1.2 20070925 (Red Hat 4.1.2-27)
All tests passed
real 8m58.109s
user 8m44.349s
sys 0m3.359s
Successfully installed flint-0.9-r1075.p0
Almost the same on Fedora 8.
Jaap
Joel B. Mohler
> Bill has fixed a couple of bugs in flint since r1072 that were corner
> cases that only happened on Core Duos, so I have updated the spkg to
> r1075. It is available at
>
> http://sage.math.washington.edu/home/mabshoff/flint-0.9-r1075.spkg
On a hyperthreaded P4:
All tests passed
real 9m19.415s
user 9m6.906s
sys 0m9.101s
Successfully installed flint-0.9-r1075.p0
On a Core Duo:
All tests passed
real 9m4.825s
user 9m0.230s
sys 0m2.090s
Successfully installed flint-0.9-r1075.p0
Both running gentoo with gcc 4.1.1.
--
Joel
William Stein
>
> On Friday 23 November 2007 22:46, mabshoff wrote:
> > Bill has fixed a couple of bugs in flint since r1072 that were corner
> > cases that only happened on Core Duos, so I have updated the spkg to
> > r1075. It is available at
> >
> > http://sage.math.washington.edu/home/mabshoff/flint-0.9-r1075.spkg
>
This is all with the version with a memory leak, i.e. posted above:
* On OSX 10.4 Intel (a mac mini I think):
Darwin leibniz.math.harvard.edu 8.10.1 Darwin Kernel Version 8.10.1:
Wed May 23 16:33:00 PDT 2007; root:xnu-792.22.5~1/RELEASE_I386 i386
i386
gcc version 4.0.1 (Apple Computer, Inc. build 5367)
All tests passed
real 11m10.282s
user 8m1.407s
sys 0m33.591s
* On a Xeon 32-bit Ubuntu Linux machine:
All tests passed
real 12m3.832s
user 5m45.310s
sys 0m23.857s
Linux debian32 2.6.18-4-686 #1 SMP Mon Mar 26 17:17:36 UTC 2007 i686 GNU/Linux
gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
* Sparc Solaris 10 -- flint doesn't build, since maybe u_int16_t
doesn't exist or
isn't properly included:
SunOS abel 5.10 Generic_125100-10 sun4u sparc SUNW,Sun-Fire-V440
gcc version 4.1.1
gcc -std=c99 -I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include/
-I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include
-I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include
-funroll-loops -fexpensive-optimizations -fPIC -funroll-loops -O3
-c fmpz_poly.c -o fmpz_poly.o
fmpz_poly.c: In function ?~@~Xfmpz_poly_bit_pack?~@~Y:
fmpz_poly.c:169: error: ?~@~Xu_int16_t?~@~Y undeclared (first use in
this function)
fmpz_poly.c:169: error: (Each undeclared identifier is reported only once
fmpz_poly.c:169: error: for each function it appears in.)
fmpz_poly.c:169: error: expected ?~@~X;?~@~Y before ?~@~Xlower?~@~Y
fmpz_poly.c:222: error: ?~@~Xlower?~@~Y undeclared (first use in this function)
fmpz_poly.c:222: error: expected ?~@~X;?~@~Y before ?~@~Xtemp?~@~Y
make: *** [fmpz_poly.o] Error 1
gcc -std=c99 -I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include/
-I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include
-I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include
-funroll-loops -fexpensive-optimizations -fPIC -funroll-loops -O3
-c mpn_extras-test.c -o mpn_extras-test.o
gcc -std=c99 -I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include/
-I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include
-I/hearsthome/home/math1/was/solaris10/sage-2.8.13.rc2/local/include
-funroll-loops -fexpensive-optimizations -fPIC -funroll-loops -O3
-c fmpz_poly.c -o fmpz_poly.o
fmpz_poly.c: In function ?~@~Xfmpz_poly_bit_pack?~@~Y:
fmpz_poly.c:169: error: ?~@~Xu_int16_t?~@~Y undeclared (first use in
this function)
fmpz_poly.c:169: error: (Each undeclared identifier is reported only once
fmpz_poly.c:169: error: for each function it appears in.)
fmpz_poly.c:169: error: expected ?~@~X;?~@~Y before ?~@~Xlower?~@~Y
fmpz_poly.c:222: error: ?~@~Xlower?~@~Y undeclared (first use in this function)
fmpz_poly.c:222: error: expected ?~@~X;?~@~Y before ?~@~Xtemp?~@~Y
make: *** [fmpz_poly.o] Error 1
Error building flint shared library.
* OSX 10.4 PPC -- works fine, all tests pass.
mabshoff
> * Sparc Solaris 10 -- flint doesn't build, since maybe u_int16_t
> doesn't exist or
> isn't properly included:
couple defines) that we also use for Sage in general. On Solaris 10
flint should build, but I haven't checked. There already is a ticket
about this, see #1178.
Cheers,
Michael
William Stein
<Michael...@fsmath.mathematik.uni-dortmund.de> wrote:
>
> Hello Bill,
>
> > * Sparc Solaris 10 -- flint doesn't build, since maybe u_int16_t
> > doesn't exist or
> > isn't properly included:
>
> this is because Solaris 9 isn't C99 conform. I have a fix for this (a
> couple defines) that we also use for Sage in general.
> On Solaris 10
> flint should build, but I haven't checked. There already is a ticket
> about this, see #1178.
I was building on Solaris 10 not solaris 9 (see below):
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Bill Hart
u_int32_t should work, but it doesn't on all c99 systems. However I
looked into this and uint32_t should always work on "c99 compliant"
systems, even on systems which the former is a problem for. Therefore
in the latest FLINT release I've simply replaced all occurrences of
the former with the latter.
I've tried #defines, e.g. #ifndef u_int32_t, etc., however these
#defines themselves caused build problems on some systems for me.
Hopefully what I've done in the latest FLINT will always work on
systems which claim to be c99 compliant.
Bill.
mabshoff
On Nov 24, 7:39 pm, Bill Hart <goodwillh...@googlemail.com> wrote:
> It's OK. In the latest revision of FLINT this will build. Essentially
> u_int32_t should work, but it doesn't on all c99 systems. However I
> looked into this and uint32_t should always work on "c99 compliant"
> systems, even on systems which the former is a problem for. Therefore
> in the latest FLINT release I've simply replaced all occurrences of
> the former with the latter.
>
> I've tried #defines, e.g. #ifndef u_int32_t, etc., however these
> #defines themselves caused build problems on some systems for me.
> Hopefully what I've done in the latest FLINT will always work on
> systems which claim to be c99 compliant.
>
> Bill.
can be used from spkg-install in a clean way. Best would be in my
opinion to merge the changes we make to build OSX dylibs upstream. I
will look into this tomorrow when I will hopefully have time to play
with 2.8.14 on Solaris.
Cheers,
Michael
Bill Hart
uint32_t. That is apparently possible as many compilers don't
implement the c99 standard properly. It's also apparently a
fundamental limitation of some platforms, though hopefully none of the
ones we want to support, since FLINT absolutely requires it on
machines whose unsigned long is 32 bits.
There is the file written by Paul Hsieh to fix this for many systems:
http://www.azillionmonkeys.com/qed/pstdint.h
The problem is, the line
#ifndef uint16_t
does not work on some systems. Even though uint16_t is defined, the
compiler will not recognise that it is so (e.g. in Cygwin).
Nonetheless, it apparently does no harm to redefine it, as pstdint.h
does and tthen FLINT builds and runs fine.
This is probably the closest we will get to portability.
I'll include pstdint.h in the next revision of FLINT suitable for
inclusion in SAGE and we'll see if it still builds on all the other
systems it currently builds on. Pretend c99 compliance is a real
problem, and had I known about it before starting, I would have not
made use of anything in the c99 standard.
Bill.
Bill Hart
you do make library, it should make a dylib if you are compiling on
Darwin. (If you have other systems you want to compile a dylib instead
of a .so, just edit the file flint_env in the FLINT trunk and send me
the modifications and I'll commit them.)
The only things spkg_install should need to do is:
1) Tell it where GMP is, i.e:
export FLINT_GMP_LIB_DIR=....
export FLINT_GMP_INCLUDE_DIR=....
2) Make sure LD_LIBRARY_PATH has the GMP library path in it (currently
if LD_LIBRARY_PATH is empty FLINT sets it to FLINT_GMP_LIB_DIR,
otherwise it assumes you set it yourself)
3) source flint_env
4) make library
5) Move the library and header files to wherever you need them to be
Everything else should be done by flint_env automatically, including
all the platform specific stuff. If not, let me know and I'll fix it.
Similarly if you are building the test programs, you'd do the above
but with make test instead of make library. There is also make
examples, make tune (only makes the tuning targets at this point,
doesn't run them, and they aren't ready to be used yet), make profiles
(again just makes profiling programs, doesn't do anything with them)
and make all (does all of the above except it doesn't make a library).
Eventually SAGE will want to build FLINT by doing make tune before
make library. But this is not useful yet.
Bill.
On 24 Nov, 18:42, mabshoff <Michael.Absh...@fsmath.mathematik.uni-
Bill Hart
Is there a simple way to either:
1) Use a bash script to parse the existing $LD_LIBRARY_PATH to see if
one of the path strings it contains is equal to $FLINT_GMP_LIB_DIR
2) Add $FLINT_GMP_LIB_DIR to $LD_LIBRARY_PATH if it isn't already in
there (I see how to add it whether or not it is in there, but this
could lead to a duplicate path string)
I've read piles of tutorials on bash scripts and I can't find a way of
doing either without piles of lines of bash script.
Bill.
Bill Hart
On 24 Nov, 19:22, Bill Hart <goodwillh...@googlemail.com> wrote:
> There is the file written by Paul Hsieh to fix this for many systems:
>
> http://www.azillionmonkeys.com/qed/pstdint.h
Bill.
Bill Hart
standard and it says:
1) uint8_t, uint16_t, uint32_t and uint64_t are not required to be
defined.
2) If the compiler provides integer types of 8, 16, 32 or 64 bits it
*must* supply the corresponding uintN_t types.
Therefore if FLINT fails to build because of this, the compiler isn't
c99 compliant or it doesn't provide the required integer types and
either way there is nothing that can be done to fix the problem
(without changing the compiler).
Bill.
boo...@u.washington.edu
Redistributions in binary form must not misrepresent ... other materials provided with the distribution.
sounds like the following would violate this: you distribute documentation with your software... the distributed binary is compatable only with 64-bit hardware, and you claim to be 32-bit compatable in the docs.
Also, the third paragraph is missing a "not", which makes it nongrammatical. *yay*.
Fortunately, it's close enough to the MIT license, that if you pointed these flaws out to him, he might change.
Paul Hsieh
dual license, GPL being one of them.
--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/
Paul Hsieh
> Yuck, that's a terrible license. The second paragraph is rather ambiguous:
>
> Redistributions in binary form must not misrepresent ... other materials provided with the distribution.
>
> sounds like the following would violate this: you distribute documentation with your software... the distributed binary is compatable only with 64-bit hardware, and you claim to be 32-bit compatable in the docs.
>
> Also, the third paragraph is missing a "not", which makes it nongrammatical. *yay*.
>
> Fortunately, it's close enough to the MIT license, that if you pointed these flaws out to him, he might change.
like SCO into legal trouble if they attempted to claim they "owned"
pstdint.h just because they happened to be using it and claiming
someone else did and therefore was engaging in copyright
infringement. (Like they did with the system V malloc implementation
and Linux.)
I am considering just dual licensing everything just GPL v2.1 and BSD
though I guess I should look at the MIT license. Basically I would
just like the file to be widely enough used that I should be able to
take credit for it just enough that people would generally know that I
did it (and get a few kudos for it), without that itself being too
much of a restriction, and for people not to use it in an abusive way.
William Stein
> > Fortunately, it's close enough to the MIT license, that if you pointed these flaws out to him, he might change.
>
> Yeah, I did fly by night thing. What I wanted was to put companies
> like SCO into legal trouble if they attempted to claim they "owned"
> pstdint.h just because they happened to be using it and claiming
> someone else did and therefore was engaging in copyright
> infringement. (Like they did with the system V malloc implementation
> and Linux.)
>
> I am considering just dual licensing everything just GPL v2.1 and BSD
> though I guess I should look at the MIT license. Basically I would
> just like the file to be widely enough used that I should be able to
Many thanks for being so helpful!
I don't personally see a reason to dual-license under both GPL and BSD,
since anything BSD licensed can be used in any GPL project. Technically,
the GPL license is just "BSD + extra restrictions". I would chose
BSD over MIT, since BSD is more popular and widely known.
(Just make sure to use the modified BSD rather than the original
version.)
> take credit for it just enough that people would generally know that I
> did it (and get a few kudos for it), without that itself being too
> much of a restriction, and for people not to use it in an abusive way.
I'm not sure what to say about this, except that unless you are
the copyright holder no matter which of the above licenses you use.
-- William
mabshoff
On Nov 24, 11:14 pm, Bill Hart <goodwillh...@googlemail.com> wrote:
> Michael,
>
> Is there a simple way to either:
>
> 1) Use a bash script to parse the existing $LD_LIBRARY_PATH to see if
> one of the path strings it contains is equal to $FLINT_GMP_LIB_DIR
>
> 2) Add $FLINT_GMP_LIB_DIR to $LD_LIBRARY_PATH if it isn't already in
> there (I see how to add it whether or not it is in there, but this
> could lead to a duplicate path string)
>
> I've read piles of tutorials on bash scripts and I can't find a way of
> doing either without piles of lines of bash script.
>
> Bill.
>
I opened ticket #1260 for this and pasted this and your previous
comments into the description. It is assigned to me, so I might came
back with some questions down the road. We can combine the cleanup
with an update to a more recent revision of FLINT in case you think it
is the right thing to do.
Cheers,
Michael
Paul Hsieh
> > I am considering just dual licensing everything just GPL v2.1 and BSD
> > though I guess I should look at the MIT license. Basically I would
> > just like the file to be widely enough used that I should be able to
>
> Many thanks for being so helpful!
>
> I don't personally see a reason to dual-license under both GPL and BSD,
> since anything BSD licensed can be used in any GPL project. Technically,
> the GPL license is just "BSD + extra restrictions". I would chose
> BSD over MIT, since BSD is more popular and widely known.
> (Just make sure to use the modified BSD rather than the original
> version.)