Is 'flex' needed to build Sage?
Dr. David Kirkby
It has both times failed with:
make install in Singular
make[4]: Entering directory
`/export/home/drkirkby/32/sage-4.4.3/spkg/build/singular-3-1-0-4-20100214/src/Singular'
sh flexer.sh -I -Pyylp -t libparse.l >libparse.cc.lmp
flexer.sh: flex: not found
flexer.sh: test: argument expected
make[4]: *** [libparse.cc] Error 1
So is flex a requirement to build Sage? If it is, then I'll add to the 'prereq'
script a test for flex. If, which I doubt, it is only needed on Solaris, then
I'll make the test specific to Solaris.
The Solaris system has 'lex' in the path, and whilst 'flex' comes as part of
Solaris, it is in a directory (/usr/sfw/bin) which was not in the path on this
particular attempt at building.
I want to avoid forcing everyone to have flex installed if it is possible to
build Sage without flex, though I doubt that is the case.
Dave
Mike Hansen
<david....@onetel.net> wrote:
> So is flex a requirement to build Sage? If it is, then I'll add to the
> 'prereq' script a test for flex. If, which I doubt, it is only needed on
> Solaris, then I'll make the test specific to Solaris.
For what it's worth, Sage / Singular builds on Cygwin where "flex"
does not seem to be installed.
Another thing which needs to be added to prereq (at least on Cygwin) is "file".
--Mike
William Stein
Flex and bison used to be required to build Sage. But they definitely haven't
been required for years on Linux, etc. I'm not sure what's going on
with Solaris.
William
>
> Dave
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>
--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org
Dr. David Kirkby
> On Sun, Jun 6, 2010 at 1:14 AM, Dr. David Kirkby
> <david....@onetel.net> wrote:
>> I've tried twice to build Sage on a machine in which 'flex' was not in the
>> path. It has both times failed with:
>>
>> make install in Singular
>> make[4]: Entering directory
>> `/export/home/drkirkby/32/sage-4.4.3/spkg/build/singular-3-1-0-4-20100214/src/Singular'
>> sh flexer.sh -I -Pyylp -t libparse.l>libparse.cc.lmp
>> flexer.sh: flex: not found
>> flexer.sh: test: argument expected
>> make[4]: *** [libparse.cc] Error 1
>>
>>
>> So is flex a requirement to build Sage? If it is, then I'll add to the
>> 'prereq' script a test for flex. If, which I doubt, it is only needed on
>> Solaris, then I'll make the test specific to Solaris.
>>
>> The Solaris system has 'lex' in the path, and whilst 'flex' comes as part of
>> Solaris, it is in a directory (/usr/sfw/bin) which was not in the path on
>> this particular attempt at building.
>>
>> I want to avoid forcing everyone to have flex installed if it is possible to
>> build Sage without flex, though I doubt that is the case.
>
> Flex and bison used to be required to build Sage. But they definitely haven't
> been required for years on Linux, etc. I'm not sure what's going on
> with Solaris.
>
> William
Interesting.
Certainly adding /usr/sfw/bin to the path, where 'flex' is, results in a
successful build of Singular
drkirkby@swan:~/32/sage-4.4.3$ ls spkg/installed/sin*
spkg/installed/singular-3-1-0-4-20100214
drkirkby@swan:~/32/sage-4.4.3$
singular-3-1-0-4-20100214/src/Singular/flexer.sh
is the file which calls 'flex;.
In Singular's 'configure.in' I see some checks around 'lex' and 'flex'. It may
be that it will build without 'lex', but fails to build with 'lex'. (Remember,
flex is a GNU replacement for 'lex'). On Solaris 'lex' was in my path, but not
'flex'.
There's also a check for bison.
# bison
AC_CHECK_PROGS(BISON, bison)
if test "${BISON+set}" != set; then
AC_MSG_WARN(did not find bison -- make might fail)
fi
So it is perhaps not wise to build without bison.
I note from the comment on sage-support about the package 'LiE' that bison is
required to install that.
Do these two checks make sense to add to 'prereq'?
1) 'file' - on all platforms. That is currently used in 'prereq' anyway.
I'm surprised Cygwin does not come with the 'file' command, since it's a POSIX
requirement. It's also an amazingly useful command.
2) 'flex' - only check for that on Solaris.
Dave
Dr. David Kirkby
> On Sun, Jun 6, 2010 at 1:14 AM, Dr. David Kirkby
> <david....@onetel.net> wrote:
>> So is flex a requirement to build Sage? If it is, then I'll add to the
>> 'prereq' script a test for flex. If, which I doubt, it is only needed on
>> Solaris, then I'll make the test specific to Solaris.
>
> For what it's worth, Sage / Singular builds on Cygwin where "flex"
> does not seem to be installed.
Thanks mike.
> Another thing which needs to be added to prereq (at least on Cygwin) is "file".
>
> --Mike
>
I'm really surprised that 'file' does not exist on Cygwin. It is required as
part of POSIX
http://www.opengroup.org/onlinepubs/9699919799/utilities/file.html
and IMHO is a pretty common and useful command. Something like 'lex' and 'yacc'
I can see are less useful to most people, but I would have thought having 'file'
installed was almost as useful as 'ls'!
dave
Willem Jan Palenstijn
> I've tried twice to build Sage on a machine in which 'flex' was not in
> the path. It has both times failed with:
>
> make install in Singular
> make[4]: Entering directory
> `/export/home/drkirkby/32/sage-4.4.3/spkg/build/singular-3-1-0-4-20100214/src/Singular'
> sh flexer.sh -I -Pyylp -t libparse.l >libparse.cc.lmp
> flexer.sh: flex: not found
> flexer.sh: test: argument expected
> make[4]: *** [libparse.cc] Error 1
>
>
> So is flex a requirement to build Sage? If it is, then I'll add to the
> 'prereq' script a test for flex. If, which I doubt, it is only needed on
> Solaris, then I'll make the test specific to Solaris.
The way this usually works is that packages are shipped with the output of
flex, so that flex doesn't have to be run again if the user doesn't have it.
In this case that also seems to be the case, since the singular package
includes src/Singular/libparse.cc. However, it's one second older than
libparse.l. A 'touch src/Singular/libparse.cc' in the spkg-install should work
around this, I think.
-Willem Jan
Dr. David Kirkby
Thanks, I'll give that a try.
I can't understand in that case how Singular can possibly install on any
operating system which does not have 'flex' installed. But Mike is saying
Singular builds OK on Cygwin, despite the fact flex is not installed. How can
that be?
I'm using an old version of GNU make (3.80, dated 2002). Perhaps more recent
versions ignore very small time differences. I could see this could be
dangerous, but I'm also aware that if systems are not properly synced in time,
and use shared file systems, files can appear to have a date in the future. It
may be that certain versions of make take this into account, and ignore small
differences. This does not seem a very plausible explanation, but I can't think
of a better one!
Dave
Willem Jan Palenstijn
Yeah, I was thinking the same thing. FAT32 only has a timestamp resolution of 2
seconds IIRC, so if the files are on FAT32, that might also be the reason. Or
possibly make on cygwin has a greater time tolerance because of this FAT32
property, even when running on other filesystems.
-Willem Jan
Dr. David Kirkby
Thanks. You seem to have solved that. I made the change you suggested, and it
works fine. I also sorted out a couple of other less major issues.
http://trac.sagemath.org/sage_trac/ticket/9160
If anyone could review it, that would be great. I've put you as an author too,
as that seems most appropriate, though I guess that means you can't review it
unfortunately.
Dave
Dr. David Kirkby
Mike, I'll update 'prereq' to check for 'file' - it seems I don't need to bother
with 'flex' but I might as well do it 'file'. I have tried to put a helpful
message in 'prereq' to indicate how to get things that are missing. Can you give
any suggestions for wording for Cygwin?
I'd suggest wording that is unlikely to be inaccurate next time Cygwin is updated.
I'm really quite surprised Cygwin lacks such a basic command. Does 'ls' come as
standard, or do you need to install that too?
Dave
leif
> On Sun, Jun 06, 2010 at 09:14:09AM +0100, Dr. David Kirkby wrote:
> > I've tried twice to build Sage on a machine in which 'flex' was not in
> > the path. It has both times failed with:
>
> > make install in Singular
> > make[4]: Entering directory
> > `/export/home/drkirkby/32/sage-4.4.3/spkg/build/singular-3-1-0-4-20100214/src/Singular'
> > sh flexer.sh -I -Pyylp -t libparse.l >libparse.cc.lmp
> > flexer.sh: flex: not found
> > flexer.sh: test: argument expected
> > make[4]: *** [libparse.cc] Error 1
>
> > So is flex a requirement to build Sage? If it is, then I'll add to the
> > 'prereq' script a test for flex. If, which I doubt, it is only needed on
> > Solaris, then I'll make the test specific to Solaris.
>
> The way this usually works is that packages are shipped with the output of
> flex, so that flex doesn't have to be run again if the user doesn't have it.
examples; the auto-tools are another one).
Anyone who includes/updates an spkg should make sure that no
"developer tools" are required to perform a conventional "user" build/
install, i.e.:
- the pregenerated files are included in the source tarball,
- file modification times do not cause unintended dependencies/
rebuild of provided files.
Unfortunately the sections on spkgs in the Developer's Guide have been
split into creation of new and update/patching of existing spkgs
("Producing New Sage Packages" [2], "Patching a Sage Package" [3]);
I'm not sure if everybody will read the introductory "Disseminating
Code for Sage" [1]. Perhaps a hint in potentially affected spkgs'
SPKG.txt wouldn't be bad, too.
(The file permissions/ownership issue could also be mentioned in these
sections.)
-Leif
[1] http://www.sagemath.org/doc/developer/disseminating_code.html#disseminating-code-for-sage
[2] http://www.sagemath.org/doc/developer/producing_spkgs.html#producing-new-sage-packages
[3] http://www.sagemath.org/doc/developer/patching_spkgs.html#patching-a-sage-package
Dr. David Kirkby
though I can't think of how I would improve it.
Things like creating .spkg's should arguably not be subsections of
"Disseminating Code for Sage"
The section about .spkg's called "Avoiding trouble" could certainly do with
expansion on these issues.
http://www.sagemath.org/doc/developer/producing_spkgs.html#avoiding-troubles
It's only seven sentences long. It should certainly state that programs used by
developers (yacc, lex, bison, flex, autoconf, automake should not be assumed to
be available). However, from discussions on sage-support, the optional package
Lie needs bison. Perhaps the bison output for that could be added into the
package, so one does not need bison. I don't know how practical that would be,
and given it is an optional package and has a more fundamental flaw, I can't be
bothered to try to do anything about that - installing bison would be easiest.
I doubt however there is much a Sage developer could have done to notice the
problem with Singular. In general, they are not going to look into the source
code of the individual packages, and compare time stamps on files. Even if they
do, with 'ls -l' they are not going to see that a file is one second older than
another, as the dates are back in 2008 and this information is not shown in the
normal output of 'ls'. How many of them will know what 'yacc' and 'lex' do
anyway? (Or their GNU clones 'bison' and 'flex').
Some things I look at in Sage and wonder how anyone could be so stupid to miss.
In this case, I don't think any Sage developer can be blamed. I suspect they
have untarred the Singular source code, so the dates have remained what they
originally were.
Anyway, it is good that it is found.
Could I twist your arm to review the changes?
Dave
leif
> On 06/ 6/10 04:43 PM, leif wrote:
> > [2]http://www.sagemath.org/doc/developer/producing_spkgs.html#producing-...
> > [3]http://www.sagemath.org/doc/developer/patching_spkgs.html#patching-a-...
> The section "Disseminating Code for Sage" does seem rather mis-placed to me,
> though I can't think of how I would improve it.
> Things like creating .spkg's should arguably not be subsections of
> "Disseminating Code for Sage"
>
> The section about .spkg's called "Avoiding trouble" could certainly do with
> expansion on these issues.
> http://www.sagemath.org/doc/developer/producing_spkgs.html#avoiding-t...
> It's only seven sentences long. It should certainly state that programs used by
> developers (yacc, lex, bison, flex, autoconf, automake should not be assumed to
> be available). However, from discussions on sage-support, the optional package
> Lie needs bison. Perhaps the bison output for that could be added into the
> package, so one does not need bison. I don't know how practical that would be,
> and given it is an optional package and has a more fundamental flaw, I can't be
> bothered to try to do anything about that - installing bison would be easiest.
skeleton files or even libraries, too (i.e. providing the pregenerated
files does not always suffice).
> I doubt however there is much a Sage developer could have done to notice the
> problem with Singular.
(Obviously this depends on the system as well; usually one had to
run ./configure before...)
perhaps grep <tool> in (test) install.log
(testing on mostly vanilla "user" installations is another option)
> In general, they are not going to look into the source
> code of the individual packages, and compare time stamps on files. Even if they
> do, with 'ls -l' they are not going to see that a file is one second older than
> another, as the dates are back in 2008 and this information is not shown in the
> normal output of 'ls'. How many of them will know what 'yacc' and 'lex' do
> anyway? (Or their GNU clones 'bison' and 'flex').
>
> Some things I look at in Sage and wonder how anyone could be so stupid to miss.
> In this case, I don't think any Sage developer can be blamed. I suspect they
> have untarred the Singular source code, so the dates have remained what they
> originally were.
>
> Anyway, it is good that it is found.
in the long run), not a bugfix. ;-)
> Could I twist your arm to review the changes?
(And in principle the patch should be tested on other Solaris systems,
too... I currently can only give it a "Linux-ok" - as soon as the
sysloads drop reasonably...)
-Leif
Dr. David Kirkby
I've tested on multiple Solaris systems, both with and without 'flex' in the
path - see trac ticket for further details.
The more I look at the Singular package in Sage, the more I realise it is a
complete mess. There are 5 copies of install-sh for example. Sage's spkg-install
has options that are not listed as options to the Singular 'configure' script,
and don't do what their names imply (i.e. --without-bison and --without-lex).
I've just downloaded the latest Singular source, and that is no less of a mess.
So both Sage's spkg-install and the current Singular source code are a bit poor
in my honest opinion.
As such, I think this hack is the best we can do.
It at least allows Sage to build when flex is not installed.
Dave
William Stein
<david....@onetel.net> wrote:
> On 06/ 6/10 07:29 PM, leif wrote:
>>
>> On 6 Jun., 19:02, "Dr. David Kirkby"<david.kir...@onetel.net> wrote:
>
>>> Could I twist your arm to review the changes?
>>
>> Be a little patient, awaiting 4.4.4.alpha0...
>> (And in principle the patch should be tested on other Solaris systems,
>> too... I currently can only give it a "Linux-ok" - as soon as the
>> sysloads drop reasonably...)
>>
>>
>> -Leif
>>
>
> I've tested on multiple Solaris systems, both with and without 'flex' in the
> path - see trac ticket for further details.
>
> The more I look at the Singular package in Sage, the more I realise it is a
> complete mess. There are 5 copies of install-sh for example. Sage's
> spkg-install has options that are not listed as options to the Singular
> 'configure' script, and don't do what their names imply (i.e.
> --without-bison and --without-lex).
>
> I've just downloaded the latest Singular source, and that is no less of a
> mess. So both Sage's spkg-install and the current Singular source code are a
> bit poor in my honest opinion.
In case anybody from Singular is reading this -- the above is the
opinion of David and not the Sage project as a whole. I'll add that,
in my experience, Singular is fairly easy to build and portable, more
so than many other programs. When I've personally poked around in
the Singular source code and actually tried to understand the code,
it's been surprisingly easy to understand, compared to many other
sophisticated C++ programs.
Regarding multiple copies of install-sh, please note that like Sage,
Singular is built up from other libraries and components. One could
criticize Sage on the same grounds for including multiple copies of
various autoconf scripts in the various src/ subdirectories of spkg's.
> As such, I think this hack is the best we can do.
>
> It at least allows Sage to build when flex is not installed.
I think using touch is reasonable given the circumstances.
-- William
François Bissey
> the path - see trac ticket for further details.
>
> The more I look at the Singular package in Sage, the more I realise it is
> a complete mess. There are 5 copies of install-sh for example. Sage's
> spkg-install has options that are not listed as options to the Singular
> 'configure' script, and don't do what their names imply (i.e.
> --without-bison and --without-lex).
>
> I've just downloaded the latest Singular source, and that is no less of a
> mess. So both Sage's spkg-install and the current Singular source code
> are a bit poor in my honest opinion.
>
> As such, I think this hack is the best we can do.
>
> It at least allows Sage to build when flex is not installed.
Apart from moving to the latest upstream I think the singular spkg
is due for a spring clean. It build an enormous amount of targets
in a way that looks like a very careful choreography and apart
from libsingular and the singular binary there is no indication
sage uses any of the other stuff built.
I have just read William's opinion and it may be true that is generally easy
to build. But reproducing the set up used in the sage spkg from a packaging
point of view is quite difficult.
Francois
William Stein
My main point is that it can be unduly offensive to criticize software
quality based purely on looking for a few minutes and deciding it
looks complicated or ugly. I think one has to take a broader view and
consider how the software works in practice, how easy it is to
understand when you're seriously interesting in hacking on the code,
etc.
A huge amount of effort has gone into Singular, and I greatly
appreciate that it is open source. I think the Sage project has a
very good relationship with the Singular project, and I hope criticism
of the quality of their work will be constructive and not offensive.
That's all.
It would be cool if some the ideas you guys have could be added to
http://wiki.sagemath.org/days23.5
which is the joint Sage/Singular meeting this summer.
Dr. David Kirkby
> In case anybody from Singular is reading this -- the above is the
> opinion of David and not the Sage project as a whole. I'll add that,
> in my experience, Singular is fairly easy to build and portable, more
> so than many other programs. When I've personally poked around in
> the Singular source code and actually tried to understand the code,
> it's been surprisingly easy to understand, compared to many other
> sophisticated C++ programs.
That may be true, but the build process is far from clean.
Type
$ configure
$ make
$ make distclean
and it appears to delete a load of files. Then type:
$ configure
$ make
and it reports:
make: `/tmp/Singular-3-1-1/ix86-SunOS/Singular-3-1-1' is up to date.
in other words, the 'make distclean' is not cleaning things properly. There is
no 'make check' or 'make test'. The configure script silently ignores what
appear to be unsupported options like '--without-bison'
I've not looked at the C/C++ source code, but accept it might be quite clear.
The build process is far from clean.
The spkg-install in Sage is also rather messy, with tons of options that are not
listed by the configure script as being valid options.
> Regarding multiple copies of install-sh, please note that like Sage,
> Singular is built up from other libraries and components. One could
> criticize Sage on the same grounds for including multiple copies of
> various autoconf scripts in the various src/ subdirectories of spkg's.
>
>> As such, I think this hack is the best we can do.
>>
>> It at least allows Sage to build when flex is not installed.
>
> I think using touch is reasonable given the circumstances.
Me too.
Dave
Dr. David Kirkby
> yes singular is a downright mess, upstream and in sage.
I'm glad I'm not alone in my view.
> Apart from moving to the latest upstream I think the singular spkg
> is due for a spring clean. It build an enormous amount of targets
> in a way that looks like a very careful choreography and apart
> from libsingular and the singular binary there is no indication
> sage uses any of the other stuff built.
It does take a long time to build compared to most other packages, which is
probably due to the fact the package is large and so has a lot of source code.
If a lot of the targets are unnecessary, then I suspect the build time could be
reduced. It took 40 minutes on my old 900 MHz SPARC, though I could probably
reduce that if targets can be built in parallel. But even then, it is still
going to be quite lengthy.
> I have just read William's opinion and it may be true that is generally easy
> to build. But reproducing the set up used in the sage spkg from a packaging
> point of view is quite difficult.
> Francois
>
Dave
William Stein
<david....@onetel.net> wrote:
> On 06/ 6/10 11:48 PM, François Bissey wrote:
>
>> yes singular is a downright mess, upstream and in sage.
>
> I'm glad I'm not alone in my view.
I just want to clarify something again (which David already
clarified): you guys are both talking about the build system, and I'm
talking only about the C++ source code. So we aren't disagreeing.
Tim Daly
All you have to do is send a 'diff -Naur' patch that improves the build.
You got hundreds of megabytes of code for free and you're complaining
because the install script isn't written the way you advocate?
Advocacy is volunteering.
Send them a thank-you note for all the free code and include a patch file.
Tim
"Open source... now it's YOUR problem to solve" -- daly
leif
> On Sun, Jun 6, 2010 at 4:10 PM, Dr. David Kirkby
>
> > On 06/ 6/10 11:48 PM, François Bissey wrote:
>
> >> yes singular is a downright mess, upstream and in sage.
>
> > I'm glad I'm not alone in my view.
>
> I just want to clarify something again (which David already
> clarified): you guys are both talking about the build system, and I'm
> talking only about the C++ source code. So we aren't disagreeing.
not the source code or Singular's quality in general.
I cannot resist to give some excerpts from just src/Singular/
Makefile.in:
##
## various programs
##
@SET_MAKE@
CC = @CC@
LD = @LD@
CXX = @CXX@
LEX = sh flexer.sh
PERL = @PERL@
BISON = bison
INSTALL = @INSTALL@
INSTALL_PROGRAM = @INSTALL_PROGRAM@
INSTALL_DATA = @INSTALL_DATA@
MKINSTALLDIRS = ./mkinstalldirs
LN_S = @LN_S@
...
SOURCES=${CSOURCES} ${CXXSOURCES} \
grammar.y scanner.l libparse.l \
utils.cc utils.h \
tesths.cc mpsr_Tok.cc claptmpl.cc
...
##
## Build Targets
##
.l.cc:
@if test -r scanner.cc; then \
touch scanner.cc ;\
elif test "x${LEX}" = x; then \
echo Error: no lex given, could not rebuilt scanner.cc;\
exit 1; \
fi
${LEX} -s -I -t $< > scanner.cc.lmp
cp scanner.cc.lmp scanner.cc
.y.cc:
@if test -r grammar.cc; then \
touch grammar.cc ;\
else \
if test "x${BISON}" = x; then \
echo Error: no bison given, could not rebuilt grammar.cc;\
exit 1; \
fi;\
${BISON} -d -t -o grammar.cc $<;\
chmod +x testgh;\
./testgh;\
fi
Note that these are all from the same file. You could also take a look
at flexer.sh...
-Leif
leif
> I'm sure Singular has a process to fix the problems you mention.
> All you have to do is send a 'diff -Naur' patch that improves the build.
> You got hundreds of megabytes of code for free and you're complaining
> because the install script isn't written the way you advocate?
(Especially in the context of MS Windows/Cygwin.)
> "Open source... now it's YOUR problem to solve" -- daly
for support hotlines).
-Leif
William Stein
To make your complaints clearer and more *constructive*, I wonder if
you could explain roughly how you would rewrite the above code?
-- William
François Bissey
> All you have to do is send a 'diff -Naur' patch that improves the build.
>
> You got hundreds of megabytes of code for free and you're complaining
> because the install script isn't written the way you advocate?
>
> Advocacy is volunteering.
>
> Send them a thank-you note for all the free code and include a patch file.
>
> Tim
>
> "Open source... now it's YOUR problem to solve" -- daly
"there are two groups of people: those who can and those who complain"
I volunteer and try to do stuff but while I can see some problems I am not
necessarily competent to do something about it.
Actually in the case of singular I may end up making it worse.
But I think the singular spkg can be improved and that I think I can do.
I will certainly have a shoot, once I have marked the 200 copies that are
coming my way in the next week.
Francois
William Stein
>> I'm sure Singular has a process to fix the problems you mention.
>> All you have to do is send a 'diff -Naur' patch that improves the build.
>>
>> You got hundreds of megabytes of code for free and you're complaining
>> because the install script isn't written the way you advocate?
>>
>> Advocacy is volunteering.
>>
>> Send them a thank-you note for all the free code and include a patch file.
>>
>> Tim
>>
>> "Open source... now it's YOUR problem to solve" -- daly
> OK I sympathise, that's in the same line as
> "there are two groups of people: those who can and those who complain"
> I volunteer and try to do stuff but while I can see some problems I am not
> necessarily competent to do something about it.
> Actually in the case of singular I may end up making it worse.
+1
I would like to very strongly encourage people to complain about
anything they find very frustrating in Sage, since it is difficult to
know what to fix without these complaints. This sort of user
feedback is extremely valuable, especially from users who aren't
capable of contributing back.
You can use this link to complain anonymously:
http://spreadsheets.google.com/viewform?key=pCwvGVwSMxTzT6E2xNdo5fA
I'll try to be as constructive as possible with my complaints about
upstream components though. In the past, I've sometimes not done so,
and always regretted it.
> But I think the singular spkg can be improved and that I think I can do.
> I will certainly have a shoot, once I have marked the 200 copies that are
> coming my way in the next week.
Cool, thanks! And if nothing else you can post some of your findings to
http://wiki.sagemath.org/days23.5
so somebody like me can think about this at the Sage/Singular sage days.
William
>
> Francois
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>
--
leif
builds on Solaris (too) at the moment.
(By the way, I did not complain.)
There's of course room for another ticket.
What's wrong with the above snippets should be clear to everyone
interested... ;-)
First of all, if you use configure/autotools, don't hardcode lex and
yacc (in this case, actually bison and a shell script that just deals
with different versions of flex, and does not even test if or where
flex is present).
Less of a problem if up-to-date versions of the generated files are
shipped to "end users" (with correct m-times; as said before, in
general ordinary users shouldn't need "developer tools" like lex&yacc
or autotools).
The assumption that there is only one source file matching the .l.cc
rule is false.
That rule has a "misplaced" if-elif-fi.
Both rules assume that any present generated file is always up-to-
date.
-Leif
François Bissey
> > is due for a spring clean. It build an enormous amount of targets
> > in a way that looks like a very careful choreography and apart
> > from libsingular and the singular binary there is no indication
> > sage uses any of the other stuff built.
>
> It does take a long time to build compared to most other packages, which
> is probably due to the fact the package is large and so has a lot of
> source code.
of singular without what looks like extra baggage.
It will have some impact on the build time but probably not as much
as you would like. On my computer:
before: merge time: 22 minutes and 36 seconds.
after : merge time: 18 minutes and 52 seconds.
Francois
leif
> > It does take a long time to build compared to most other packages, which
> > is probably due to the fact the package is large and so has a lot of
> > source code.
>
> Well I am trying that now. Running the test against a lighter version
> of singular without what looks like extra baggage.
> It will have some impact on the build time but probably not as much
> as you would like. On my computer:
> before: merge time: 22 minutes and 36 seconds.
> after : merge time: 18 minutes and 52 seconds.
Singular interpreter (sage -singular).
The new Singular spkg at #9160 has now been merged into the upcoming
Sage 4.4.4.alpha1.
It fixes the flex issue and some other things, but there's much left
to do on the package in general (most probably including upgrading to
a newer upstream release).
We will see what Sage Days 23.5 bring... (see William's post above).
-Leif
François Bissey
sage -t -long devel/sage/sage/lib/singular/
as you did in your ticket after removing everything that happens
after installing libsingular. My notes in my ebuild are saying:
---------------------------------------------------------------------------------------------
The following is needed to get libfac.a libcf.a? libcfmem.a?
libsingcf_g.a? libsingfac.a? (the last 2 have some defects)
We should check that all these are really needed.
Then if needed check if we could build a specific target rather than
reconfiguring.
-------------------------------------------------------------------------------------------
after removing those:
sage -t -long devel/sage/sage/libs/singular/
sage -t -long "devel/sage/sage/libs/singular/function_factory.py"
[15.9 s]
sage -t -long "devel/sage/sage/libs/singular/singular.pyx"
[19.2 s]
sage -t -long "devel/sage/sage/libs/singular/option.pyx"
[17.1 s]
sage -t -long "devel/sage/sage/libs/singular/polynomial.pyx"
[16.5 s]
sage -t -long "devel/sage/sage/libs/singular/groebner_strategy.pyx"
[17.5 s]
sage -t -long "devel/sage/sage/libs/singular/singular.pxi"
[0.3 s]
sage -t -long "devel/sage/sage/libs/singular/__init__.py"
[0.3 s]
sage -t -long "devel/sage/sage/libs/singular/ring.pyx"
[21.9 s]
sage -t -long "devel/sage/sage/libs/singular/function.pyx"
[24.7 s]
sage -t -long "devel/sage/sage/libs/singular/singular-cdefs.pxi"
[19.8 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 153.3 seconds
I also tested
sage -t -long devel/sage/sage/interfaces/singular.py
sage -t -long "devel/sage/sage/interfaces/singular.py"
[27.8 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 27.8 seconds
So I think it could removed. It may have used at some point in time
but not any more.
Francois
leif
ticket.)
What machine did you test on? (sysload?)
My old slow Pentium4 (single core with "hyper threading", i.e.
slightly faster context switches) is about 3-4 times faster... ;-)
> So I think it could removed. It may have used at some point in time
> but not any more.
-Leif
François Bissey
are valid. I will report on that later.
My system: AMD Athlon(tm) XP 2600+ (1.5GB ram)
I was building paraview-3.8.0 at the same time.
Francois
Alex Ghitza
> I haven't tested your new spkg but I ran
> sage -t -long devel/sage/sage/lib/singular/
> as you did in your ticket after removing everything that happens
> after installing libsingular. My notes in my ebuild are saying:
> ---------------------------------------------------------------------------------------------
> The following is needed to get libfac.a libcf.a? libcfmem.a?
> libsingcf_g.a? libsingfac.a? (the last 2 have some defects)
> We should check that all these are really needed.
> Then if needed check if we could build a specific target rather than
> reconfiguring.
> -------------------------------------------------------------------------------------------
> after removing those:
> sage -t -long devel/sage/sage/libs/singular/
> Total time for all tests: 153.3 seconds
>
> I also tested
> sage -t -long devel/sage/sage/interfaces/singular.py
> All tests passed!
> Total time for all tests: 27.8 seconds
>
> So I think it could removed. It may have used at some point in time
> but not any more.
I am not convinced that testing only libs/singular/ and
interfaces/singular.py is sufficient to conclude this. Functionality
from various parts of Singular is used all over the Sage library. I
would be a bit more confident if a global "make testlong" was run, or at
the very least if sage/rings and sage/schemes were tested.
Best,
Alex
--
Alex Ghitza -- http://aghitza.org/
Lecturer in Mathematics -- The University of Melbourne -- Australia
leif
> I am not convinced that testing only libs/singular/ and
> interfaces/singular.py is sufficient to conclude this. Functionality
> from various parts of Singular is used all over the Sage library. I
> would be a bit more confident if a global "make testlong" was run, or at
> the very least if sage/rings and sage/schemes were tested.
cover all functionality required by Sage (of course not every Singular
function, but its main features, s.t. they'd fail if some component of
Singular was broken or not installed).
(I did not examine these tests, though I had been surprised how "fast"
they were finished.)
-Leif
François Bissey
were all static libraries:
<<< obj /opt/sage/local/lib/libfac.a
<<< obj /opt/sage/local/lib/libcfmem.a
<<< obj /opt/sage/local/lib/libcf.a
Some other may have slightly changed has the build procedure in the spkg
was actually building them twice.
In particular two of the libraries that were built twice were:
libsingcf_g.a
libsingfac.a
When I copy the building procedure from sage exactly in an ebuild for gentoo
I get the following warnings from the build system
i686-pc-linux-gnu-strip:/media/anthill/portage/sci-mathematics/sage-
singular-3.1.0.4.20100214/image/opt/sage/local/lib/libsingcf_p.a(readcf.y):
Unable to recognise the format of file: File format not recognized
opt/sage/local/lib/libsingcf_g.a
i686-pc-linux-gnu-strip:/media/anthill/portage/sci-mathematics/sage-
singular-3.1.0.4.20100214/image/opt/sage/local/lib/libsingcf_g.a(readcf.y):
Unable to recognise the format of file: File format not recognized
opt/sage/local/lib/libsingfac.a
Removing the "extra baggage" those are still built but do not exhibit these
problems any more. So in actual fact, some deep problems with these two
libraries have been cured.
More systematically:
1) they are all static libraries so we can look at all components depending on
singular and see if they still build. If they need a static library that has
been removed, the build system will at the very least complain, more likely
stop.
2) because they are all static libraries they have to be explicitly linked and
you can therefore just grep for their presence.
Only sage itself depends on singular. Unless some optional packages also
needs it the dependency search stop there. Sage still builds without problems
so point 1 is covered. None of the stuff removed is needed to build sage
and can't be used at runtime.
Sage can interface directly with the singular executable , that's covered by
the interface test.
For the rest of point 2, we have to check if we depend on any the changed
libraries (libsingcf_g.a libsingfac.a). The answer to that is nope. If you
want to make real sure, here are all libraries shipped with singular:
sage/local/lib/omalloc_debug.o
sage/local/lib/p_Procs_FieldGeneral.so
sage/local/lib/omalloc.o
sage/local/lib/p_Procs_FieldZp.so
sage/local/lib/p_Procs_FieldIndep.so
sage/local/lib/dbmsr.so
sage/local/lib/p_Procs_FieldQ.so
sage/local/lib/libsingular.so
sage/local/lib/libomalloc_p.a
sage/local/lib/libcfmem.a
sage/local/lib/libsingfac_g.a
sage/local/lib/libomalloc_ndebug.a
sage/local/lib/libomalloc.a
sage/local/lib/libfac.a
sage/local/lib/libsingcf.a
sage/local/lib/libsingfac_p.a
sage/local/lib/libcf.a
sage/local/lib/libsingcf_p.a
In short testing is just make triple sure at this stage (but ok it will be
done eventually).
Francois
Martin Albrecht
> removed were all static libraries:
> <<< obj /opt/sage/local/lib/libfac.a
> <<< obj /opt/sage/local/lib/libcfmem.a
> <<< obj /opt/sage/local/lib/libcf.a
> Some other may have slightly changed has the build procedure in the spkg
> was actually building them twice.
Different build options are needed for linking against libcf and libfac
directly than for including them in Singular as a stand-alone program. We used
to link against libfac and libcf directly but this is long gone. So I think
you are probably safe here.
Martin
--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
_www: http://www.informatik.uni-bremen.de/~malb
_jab: martinr...@jabber.ccc.de