Add 'gcc' libraries to Sage binaries (< 0.5% bloat)
Dr. David Kirkby
thread on it.
I'd propose that we include in any binary distribution gcc's C, C++ and Fortran
shared libraries. They would be placed in $SAGE_LOCAL/lib. Then we can ensure
that people will run Sage with what libraries Sage was built with, rather than
what versions they may or may not have lying around.
The amount of bloat this would add to the binary would be very small. For
Solaris, the compressed sizes of the files are:
-rwxr-xr-x 1 drkirkby staff 1.5M Feb 22 10:10 libstdc++.so.6.0.10.gz
-rwxr-xr-x 1 drkirkby staff 717K Feb 22 10:10 libgfortran.so.3.0.0.gz
-rw-r--r-- 1 drkirkby staff 80K Feb 22 10:10 libgcc_s.so.1.gz
So adding all 3 adds 2.3 MB of extra code to the binary. But given the binary is
500 MB (not untypical), that is less than 0.5% of bloat.
By doing this, we ensure that people
* Always have the libraries.
* Always have the exact same versions Sage was built with.
I believe the Fortran library might already be included for Linux (I have not
checked), but I'd suggest all 3 were added to binaries.
The C library is the one people most likely will have, but given it is by far
the smallest, we might as well include it to be 100% sure.
Comments?
Dave
Bill Hart
kernel is installed, etc, etc?
I'm completely ignorant on this, so may be talk out my proverbial.
Bill.
Ross Kyprianou
people with less Linux experience
Whatever we can do to make Sage "work out of the box" is good
(i.e. I know its 99% there but it will be even better if we can avoid
asking people to ensure certain things are installed and are certain
versions - even when people choose to build)
So, in a nutshell, its a +1 from me FWIW
(Of course this is subject to any vetos from anyone saying why this
wont work under certain circumstances)
Volker Braun
(contains standard math library) as well. While a viable possibility,
there are two obvious arguments against it:
On Feb 22, 11:27 am, "Dr. David Kirkby" <david.kir...@onetel.net>
wrote:
> * Always have the libraries.
Instead, check on first startup that all required packages are
installed and print an error message instead ('Please "yum install
libgfortran"').
> * Always have the exact same versions Sage was built with.
Downside: if your distribution comes out with a bugfix you don't
automatically profit from it. Also, if your distribution keeps
breaking the C++ ABI then maybe its time to switch.
Dr. David Kirkby
> On 22 Feb, 11:27, "Dr. David Kirkby" <david.kir...@onetel.net> wrote:
>> This came up on the thread "mercurial on t2" but I thought I'd start a new
>> thread on it.
>>
>> I'd propose that we include in any binary distribution gcc's C, C++ and Fortran
>> shared libraries. They would be placed in $SAGE_LOCAL/lib. Then we can ensure
>> that people will run Sage with what libraries Sage was built with, rather than
>> what versions they may or may not have lying around.
>> Comments?
>>
>> Dave
>
> Are we sure this would work? Won't those libraries depend on what
> kernel is installed, etc, etc?
>
> I'm completely ignorant on this, so may be talk out my proverbial.
>
> Bill.
>
I would not have thought so, though I don't claim to be an expert on this.
I just checked the sage-4.3.2-linux-64bit-ubuntu_9.10-x86_64-Linux binary, and
find the fortran library libgfortran.so is already there, as otherwise one would
need to have gfortran installed to run Sage. Since that is not causing any
particular kernel issues, I doubt the C or C++ libraries would either. (As
always, there can always be unwelcome surprises.)
The file COPYING.LIB in the gcc source distribution specifically allows copying
the libraries for the purposes of running code on other systems, without
including the source code for your proprietary program. I don't think the gcc
developers would bother if the version of the library depended on the kernel
version in use on the other system.
Of course, we already have to produce different binaries for different Linux
distributions. So I'd suggest just copying the gcc libraries from whatever
system one built Sage on.
The libraries I put in the Solaris binary were copied from a Sun running the
very first Solaris 10 release (03/2005). The kernel on both t2 and my other
Solaris box are both late 2009. So even with 4.5 years of kernel updates, the
gcc libraries still work. (I know Linux tends to have poorer backward
compatibility than Solaris, but I doubt that would be an issue even on Linux).
Dave
Willem Jan Palenstijn
> This came up on the thread "mercurial on t2" but I thought I'd start a
> new thread on it.
>
> I'd propose that we include in any binary distribution gcc's C, C++ and
> Fortran shared libraries. They would be placed in $SAGE_LOCAL/lib. Then
> we can ensure that people will run Sage with what libraries Sage was
> built with, rather than what versions they may or may not have lying
> around.
>
> The amount of bloat this would add to the binary would be very small. For
> Solaris, the compressed sizes of the files are:
>
> -rwxr-xr-x 1 drkirkby staff 1.5M Feb 22 10:10 libstdc++.so.6.0.10.gz
> -rwxr-xr-x 1 drkirkby staff 717K Feb 22 10:10 libgfortran.so.3.0.0.gz
> -rw-r--r-- 1 drkirkby staff 80K Feb 22 10:10 libgcc_s.so.1.gz
>
>
> The C library is the one people most likely will have, but given it is by
> far the smallest, we might as well include it to be 100% sure.
Note that libgcc_s is not the C library at all, it is a gcc library containing
some support infrastructure for certain features, such as C++ exception
handling. (In particular, it is used for ensuring C++ exceptions can cross library
boundaries.)
This text describes RedHat's policy on libgcc_s and libstdc++:
http://kbase.redhat.com/faq/docs/DOC-8313
It seems to suggest that if we want to include libstdc++ or libgcc_s, we should
include both.
-Willem Jan
Dr. David Kirkby
> If you want to go that route you probably want to include glibc
> (contains standard math library) as well.
> While a viable possibility,
> there are two obvious arguments against it:
>
> On Feb 22, 11:27 am, "Dr. David Kirkby" <david.kir...@onetel.net>
> wrote:
>> * Always have the libraries.
>
> Instead, check on first startup that all required packages are
> installed and print an error message instead ('Please "yum install
> libgfortran"').
I assume one would need root to do that, but perhaps not.
In any case, we already include the fortran library. The question is whether to
extend that to C and C++ too.
>> * Always have the exact same versions Sage was built with.
>
> Downside: if your distribution comes out with a bugfix you don't
> automatically profit from it.
True, though the Sage developers tend to build the binaries with late gcc
versions. In which case bug fixes in the libraries are more likely to be
included in the Sage binary than they are on the users system unless that user
has kept more up to date with gcc than the Sage developers.
Problems are more likely to occur if the user has an older buggy gcc, but the
Sage binary was built and tested using a newer gcc.
> Also, if your distribution keeps
> breaking the C++ ABI then maybe its time to switch.
Well, it was annoying that simply updating gcc to 4.4.3 from a slightly earlier
release caused problems. But including the libraries should avoid that.
At least with the libraries included, the user gets to use Sage with libraries
where Sage has been tested.
I know Mathematica on Solaris ships with libraries that are included on Solaris.
I assume Wolfram Research feel happier knowing that Mathematica runs with
libraries which it was tested on, rather than what the end user may have.
Dave
Volker Braun
> This text describes RedHat's policy on libgcc_s and libstdc++:http://kbase.redhat.com/faq/docs/DOC-8313
> It seems to suggest that if we want to include libstdc++ or libgcc_s, we should
> include both.
Well it literally says:
"Both libgcc and libstdc++ are core libraries, applications should
never bundle their own versions of these libraries."
Volker
Robert Bradshaw
Are there issues if the gcc and core C libraries don't match? For
example, If a user uses cython in the notebook, will it pick up the
users g++ and Sage's libstdc++? What about someone who starts with a
binary then starts developing with it or installs optional packages?
I guess a lot of binaries statically link the core C libraries in,
which of course doesn't make sense for us. On the other hand, a quick
google search seems to indicate that not even Firefox ships with
libstdc++, and the link at http://kbase.redhat.com/faq/docs/DOC-8313
seems to indicate that shipping core libraries is rarely the right
thing to do, and great care is made to keep them backwards compatible.
Perhaps we should check to make sure the libraries are at least
installed at first startup, but is this really a common occurrence? If
not, I'm not convinced we should do something that might cause weird
issues for everyone to cater to the 1% of users who have a strange,
custom *nix build on odd hardware.
- Robert
Georg S. Weber
Hi Dave,
this only is reasonable, if for building Sage we do not use the (or
"a") "system's default compiler". So on GNU/Linux, or Mac OS X, this
is almost a non-issue --- it's clear what this default compiler is,
and Sage uses it.
It's far less clear on Solaris, or e.g. on Windows. If some future
Sage is built with some Microsoft Visual C++ compiler, we will have to
tell the casual user exactly which additional msvc libraries (for msvc
2008, or 2010, or ...) need to be installed on a "virgin" Windows in
order for Sage to be able to run. (These additional libraries are free
for download, but not free to distribute by others than Microsoft.)
But then, these libraries will be installed in some system location.
Back to the Solaris case --- as long as we do not have some "GCC
spkg", eventually optional, that installs the full GCC, I'm opposed to
install its compiler specific libraries under $SAGE_ROOT. Instead,
Sage should rely on them to be found in the standard system locations
--- or in the ones the user explicitly communicates to Sage via
environment variables.
Of course it would make sense to provide for Solaris some "minimal GCC
library install" specifically adapted for the need of potential Sage
users.
(Just like in the case of Microsoft not urging users to install the
full msvc, but providing some library-only package.) But IMHO, this is
outside the scope of our current model of fully relocatable yet
"open" (Sage) installs, so that would need to be offered somewhat in
parallel. (Otherwise, Sage would need to be yet far more paranoid
about taking in which dynamic libraries and how than today, see also
Robert's remark.)
Cheers,
Georg
Bill Hart
Q1. Are there any other well-known packages which do this? If not, it
is not a standard thing to do, probably for good reason.
Cython uses the C compiler (if I understand correctly). I think this
kills the idea dead.
Q2. Would building Sage with the Sun CC compiler remove the need to
have standard GNU libraries accessible somewhere? If so, how hard
would it be to have Sage build with the Sun CC?
Bill.
Bill Hart
for Solaris which are distributed with binaries and which need these
libraries. How do they deal with this issue?
Is there a package installation system like apt-get or yum or rpm on
Solaris? They must surely deal with this issue somehow?
Bill.
Dr. David Kirkby
> Actually, this is really silly. The must be a multitude of packages
> for Solaris which are distributed with binaries and which need these
> libraries. How do they deal with this issue?
>
> Is there a package installation system like apt-get or yum or rpm on
> Solaris? They must surely deal with this issue somehow?
>
> Bill.
Yes - it called 'pkgadd'
Blastwave is a site where you pick a package you want (e.g. python) and it
downloads that along with any dependences. Here are those for phthon
http://www.blastwave.org/jir/pkgcontents.ftd?software=python&style=brief&state=5&arch=i386
One of which is "GNU Compiler Collection Version 4.3.3 Core RunTime" which is
basically the libraries.
Sunfreeware (run by a consultant for Wolfram Reserarch)
has loads of packages in binary format, though the site has stagnated.
Let's pick the first package (a2ps-4.14),
a2ps-4.14
"Having libgcc_s.so.1 in /usr/local/lib is required. This can be done by
installing libgcc-3.4.6 or gcc-3.4.6 or higher."
So it basically boils down to one of two choices.
1) Install gcc.
2) Install just the gcc libraries.
When I created the binary which runs on 't2nb' (which is a zone on Solaris)
http://t2nb.math.washington.edu:8000
I included the libraries. There is no gcc on that Solaris zone, so Sage would
not run without them.
In contrast, if you have gcc installed, and the libraries can be found via
LD_LIBRARY_PATH or some other means, then those libraries are not needed.
I personally think it is a bit safer to include them, but I can see counter
arguments. The most convincing one is from Robert Bradsure who said:
-------------------------------
Are there issues if the gcc and core C libraries don't match? For example, If a
user uses cython in the notebook, will it pick up the users g++ and Sage's
libstdc++? What about someone who starts with a binary then starts developing
with it or installs optional packages?
------------------------------
The truth is, I don't know the answer to that one.
It's clear the current system does cause some problems (not just on Solaris),
but I must admit Robert's argument may be valid.
Dave
Kasper Peeters
> shared libraries.
I personally think that this is a _very_ bad idea. As others have
emphasised, most
systems out there have a proper package management tool, which can
moreover
take care of dependencies. By doing all that yourself, you also burden
yourself with
the task of keeping the Sage-packaged external programs and libraries
up to date.
While on this topic, can anyone point me to a good read on why Sage
includes every
known piece of software under the sun in its distribution? I would
personally prefer to
get rid of that _all_ and instead use the energy to support deb/rpm/
pkg/... maintainers.
Cheers,
Kasper
Kasper Peeters
> example, If a user uses cython in the notebook, will it pick up the
> users g++ and Sage's libstdc++? What about someone who starts with a
> binary then starts developing with it or installs optional packages?
On OS X, the MacPorts folks have just been bitten by exactly these
kind
of problems. They ship everything themselves, including gcc and the
libraries. Unfortunately, Apple does not always use the same compiler
flags as MacPorts, and there are thus incompatible libraries floating
around
on those systems (which means: if you use MacPorts, and it links to a
standard Apple library which is not present in MacPorts, your program
will crash).
Packing your own versions of system libraries is _never_ a good idea,
and bound to lead to all sorts of problems in the long run.
Cheers,
Kasper
Dr. David Kirkby
>> I'd propose that we include in any binary distribution gcc's C, C++ and Fortran
>> shared libraries.
>
> I personally think that this is a _very_ bad idea. As others have
> emphasised, most
> systems out there have a proper package management tool, which can
> moreover
> take care of dependencies. By doing all that yourself, you also burden
> yourself with
> the task of keeping the Sage-packaged external programs and libraries
> up to date.
I'm not saying there are not ways of doing this. But a small, though not
insignificant number of people seem to be getting bitten by a failure to have
the required libraries. In fact, after a fortran compiler was removed from Sage
on Linux (there is still one for OS X), there were instances of those without
the Fortran library, so the addition of the Fortran library was made a 'blocker'
by William.
http://trac.sagemath.org/sage_trac/ticket/8049
So the Fortran library is shipped.
I just looked at an install of Mathematica on Solaris and notice they ship
'libzip.so' and 'libsqlite.so' as well as the java runtime environment
libmawt.so. In fact, there are tons of libraries which could be found on some
systems, but Wolfram Research obvious feel the desire to include them.
> While on this topic, can anyone point me to a good read on why Sage
> includes every
> known piece of software under the sun in its distribution? I would
> personally prefer to
> get rid of that _all_ and instead use the energy to support deb/rpm/
> pkg/... maintainers.
>
> Cheers,
> Kasper
The reason is most of the packages in Sage have small modifications. To suggest
to users that they download X, but modifications A, B and C, then download Y,
but make modifications D, E and F would put off too many people. Have a look in
$SAGE_ROOT/spkg/standard, then look for the packages that end in .p0.spkg,
.p1.spkg, .p2.spkg. All of them have had some changes. Often that are
modifications of the source code. ATLAS has modifications to take care of a bug
in a memset() in some older Solaris releases. libz has changes to allow a 64-bit
build on OS X. Those are two I can think of, but there are tons of them.
To be honest, as Sage consists of about 100 items, it would be hard to know
where to start with some bug reports when you don't know precisely what versions
of the libraries people have, and whether those libraries have been updated to a
version Sage is not tested with.
Also, some of the libraries might change to GPL 3. If those libraries did not
form part of the core operating system, Sage should not link to them.
As much as I can see why people do not like this, I can see a lot of logic in
William's approach.
Dave
Nick Alexander
> logic in William's approach.
To add to what David said, some of this is historical. The 100
packages sage builds now are unwieldy, but it was not always so.
Earlier distributions contained much less, and shipping them all
seemed like a reasonable trade for "build anywhere". It's worked
surprisingly well, and we are attached to it now.
A thought occurs -- perhaps we could have sage .spkg packages that in
fact are just dummies that really use the system libraries. That
might provide a transition path to mixed .spkg/system libraries.
Nick
Volker Braun
Specialist software receives much less testing, especially on exotic
architectures. You can easily be the first one who builds X on Y and
run into some obscure bug. So it is valuable to collect mathematical
programs and make sure that they build on a variety of systems and
collect the patches to do so.
System libraries, on the other hand, receive much more attention. Just
building a whole distribution with a given libstdc++, say, is a great
testcase. Since you mention zlib, Fedora applies 2 patches to that
library. Does the zlib spkg also include those? Compiler and more
general system libraries are even worse. A lot of work goes into core
libraries, and it would be a total waste to duplicate that effort.
The default should be to not duplicate system libraries. Only in those
(few) cases where the OS does not have a usable packaging system /
ships a broken library / etc, the sage binary distribution (for that
OS) should include the library as a workaround.
William Stein
This GPL3 comment is not really an issue.
> As much as I can see why people do not like this, I can see a lot of logic
> in William's approach.
I'm going to add a few comments supporting this.
The goal of Sage is to compete with the likes of all of Magma,
Mathematica and Matlab, and as such it is a necessarily a complicated
piece of software, being more complicated than any one of those Ma's.
Most people don't appreciate how complicated Sage is. At this point,
I don't think I even do.
It would be better for end users if we built standard rpm/deb/etc.
packages that integrate well with the rest of each Linux, OS X,
Solaris, Windows, etc., operating system, and of course regularly
tested that the full test suite passes on each system, and when
packages on those systems get old or too new, deal with those
problems. That would be wonderful. Unfortunately, we have to be
realistic, given the resources that we have available.
How many major software projects with a similar level of complexity to
Sage actually do this? I can think of exactly one: Open Office. That
project is at least as complicated as Sage (maybe more). Who creates
all of their packages? Do the Open Office developers? Or the
projects such as MacPorts, Debian, Ubuntu, Mandriva, RedHat, etc.?
Are there any Open Office devs reading this? Doesn't the
OpenOffice project have dozens of fulltime developers, employed by IBM
and Sun? Sage still has 0 fulltime devs.
-- William
Nils Bruin
builds the libraries (especially libstdc++) used during building would
be *available* (not necessarily linked to) by default. Several times I
ran into a situation where I quickly wanted to test something with a
new version of sage on a (redhat or fedora) machine for which no
precisely matching binary was available. libstdc++ seems to be a
particularly volatile library (i.e., gets upgraded with pretty much
every OS release), and hence binaries that were one version off would
not work. After tracking down the appropriate version of libstdc++ and
putting that in $SAGE_ROOT/local/lib, everything worked fine.
In other situations I gave up on finding the appropriate library and
submitted to a 3 hour compile from source or abandoned the project. I
would have much preferred to have done something like
ln -s $SAGE_ROOT/buildtime/lib/* $SAGE_ROOT/local/lib
and got on with the job. I do not control which version of the OS
these machines are running and do not have administrative permissions
on them.
Dr. David Kirkby
One obvious advantage of shipping all the code is that you know more precisely
what the person is using.
I believe the thread below indicates one of issues. (Read the full thread if you
wish, but here is a summary).
1) Alex Ghitza builds an Arch Linux binary using gcc 4.4.2 and puts it on the
Sage web site. It works for Alex
2) Akbar Hidaya downloads that binary and finds it does not work for him.
3) The binary stops working for Alex Ghitza once he updates his gcc from 4.4.2
to 4.4.3.
4) Alex correctly guess that the user has updated his gcc.
http://groups.google.com/group/sage-support/browse_thread/thread/aba48495d9c09e03
I think this is an rare example, but it shows how just updating gcc can cause a
failure. I'm sure having a version of gcc too old will also cause problems,
which is why I suggested including the gcc libraries.
Now consider what happens when the end user can have loads of different
libraries at different versions. Debugging problems would become much more
problematic.
Simply insisting that the user upgrade their libraries will not be practical if
the user does not have root access.
Many system admins will be unwilling to update system libraries for users. They
simply can't take the risk the updated libraries will not cause problems for
other uses.
In an ideal world, updated libraries would be backwards compatible and introduce
no new bugs. In practice, neither are always true.
I'm also unaware of any way of determining library versions in general.
Dave
Robert Bradshaw
> While on this topic, can anyone point me to a good read on why Sage
> includes every
> known piece of software under the sun in its distribution? I would
> personally prefer to
> get rid of that _all_ and instead use the energy to support deb/rpm/
> pkg/... maintainers.
This issue has come up several times before, I don't have a specific
thread to point to, but I would recommend searching the mailing list
archives. There are projects underway to split Sage into separate
packages, but it's a massive effort to do so and no one's successfully
gotten a fully working Sage that way (though we're closer than we've
ever been). I'm unconvinced it would be less effort on our part
(maintaing and up-streaming all of deb/rpm/ports/cygwin/??? rather
than just our own extremely simple setup) as well.
- Robert
Peter Jeremy
>This came up on the thread "mercurial on t2" but I thought I'd start
>a new thread on it. I'd propose that we include in any binary
>distribution gcc's C, C++ and Fortran shared libraries. They would be
>placed in $SAGE_LOCAL/lib. Then we can ensure that people will run
>Sage with what libraries Sage was built with, rather than what
>versions they may or may not have lying around.
This is a very difficult question since there are strong pros and
cons. My suspicion is that this might wind up needing to be answered
on a case-by-case basis.
On 2010-Feb-22 05:19:01 -0800, Volker Braun <vbrau...@gmail.com> wrote:
>If you want to go that route you probably want to include glibc
>(contains standard math library) as well. While a viable possibility,
>there are two obvious arguments against it:
Whilst the math parts of glibc would definitely help, actually
including the kernel interface parts of glibc is likely to cause
more problems than it solves due to subtle differences in the
kernel ABI across different kernels.
On 2010-Feb-22 10:46:38 -0800, Robert Bradshaw <robe...@math.washington.edu> wrote:
>Perhaps we should check to make sure the libraries are at least
>installed at first startup, but is this really a common occurrence? If
>not, I'm not convinced we should do something that might cause weird
>issues for everyone to cater to the 1% of users who have a strange,
>custom *nix build on odd hardware.
It's not just that the libraries are installed but that the installed
libraries match Sage. This latter point is somewhat more difficult -
especially given the fluidity of the g++/libstdc++ ABI.
On 2010-Feb-23 00:48:51 -0800, Kasper Peeters <kasper....@googlemail.com> wrote:
>I personally think that this is a _very_ bad idea. As others have
>emphasised, most systems out there have a proper package management
>tool, which can moreover take care of dependencies.
Note that Solaris is one of the systems which doesn't come with
a proper package management system.
> By doing all that
>yourself, you also burden yourself with the task of keeping the
>Sage-packaged external programs and libraries up to date.
Agreed - and Sage has that issue now with the variety of applications
it ships with. But by shipping the libraries, you also get rid of a
lot of support questions caused by the the installed libraries not
being suitable for Sage. And in the specific case of the compiler
libraries, there's no additional support burden - Sage just ships
whatever it was compiled with.
On 2010-Feb-23 09:48:33 -0800, William Stein <wst...@gmail.com> wrote:
>How many major software projects with a similar level of complexity to
>Sage actually do this? I can think of exactly one: Open Office. That
>project is at least as complicated as Sage (maybe more).
OOo does a mix of requiring dependencies to be pre-installed and
embedding them in its own build environment. System dependencies
include Xorg, perl, gmake, bzip2, jdk, bison, tcsh, zip, unzip, GNU
coreutils, GNU patch, bash, imake, GNU tar, gperf, Apache ANT and,
optionally, seamonkey, KDE, Gnome, freetype, icu, cups. It embeds
things like agg, berkeleydb, beanshell, boost, cairo, curl, dmake,
epm, expat, icu, jpeg, libxml2, lucene, neon, nss, openssl, python,
saxon, stl, Apache tomcat, vigra, xmerge, xpdf, zlib. Despite (or
maybe because of) this combination, OOo is very fragile to build.
For several years, OOo on FreeBSD has it's own gcc port because it
was so sensitive to compiler changes.
>Are there any Open Office devs reading this? Doesn't the
>OpenOffice project have dozens of fulltime developers, employed by IBM
>and Sun? Sage still has 0 fulltime devs.
OOo is a opensource version of StarOffice - which is a commercial
product. I'm not sure how much paid developer effort goes into OOo.
--
Peter Jeremy
Kasper Peeters
> packages that integrate well with the rest of each Linux, OS X,
> Solaris, Windows, etc., operating system, and of course regularly
> tested that the full test suite passes on each system, and when
> packages on those systems get old or too new, deal with those
> problems. That would be wonderful. Unfortunately, we have to be
> realistic, given the resources that we have available.
The problem is that, by relying on your own versions of many tools
and libraries, you essentially prevent other people from helping
you with packaging for those rpm/deb/etc. systems. Because those
other people are now burdened with the task of separating those
libraries and patching things on the Sage side. Which is harder for
them,
because they do not know what to watch out for. As a result, you are
burdened with more work, because now _you_ have to take care of making
Sage installation/compilation easy, instead of having distribution
packagers
take care of that.
The consequence is clearly visible: there are no up-to-date packages
of Sage for any of the Linux distributions. Which means that for many
users,
installing Sage is actually _harder_ than installing most other
software.
But don't get me wrong: I am definitely not saying that it is easy to
make Sage deal with distribution-supplied libraries, and I fully
understand the historical reasons for shipping Sage with such a large
number of spkg's.
What I am saying is that, in the long run, it is much better to steer
clear of that.
And therefore, that it would be unwise to add more of your own
versions
of libraries, _especially_ when it concerns libraries which every
distribution
already ships with (like gcc-lib). I can see that Macports is having
problems
with the 'we ship everything ourselves' logic, and I don't see how
Sage is
going to avoid that.
Cheers,
Kasper
Robert Bradshaw
>> It would be better for end users if we built standard rpm/deb/etc.
>> packages that integrate well with the rest of each Linux, OS X,
>> Solaris, Windows, etc., operating system, and of course regularly
>> tested that the full test suite passes on each system, and when
>> packages on those systems get old or too new, deal with those
>> problems. That would be wonderful. Unfortunately, we have to be
>> realistic, given the resources that we have available.
>
> The problem is that, by relying on your own versions of many tools
> and libraries, you essentially prevent other people from helping
> you with packaging for those rpm/deb/etc. systems. Because those
> other people are now burdened with the task of separating those
> libraries and patching things on the Sage side. Which is harder for
> them, because they do not know what to watch out for. As a result,
> you are
> burdened with more work, because now _you_ have to take care of making
> Sage installation/compilation easy, instead of having distribution
> packagers take care of that.
We would have to take care of making Sage installation/compilation
easy anyways, as many of the packages we rely on (often the most
fragile ones) are rather specialized math programs that aren't always
in all the standard distributions, and it would fall on us to do all
the work to get them into all the packaging systems. And we would
probably have to roll our own for Solaris, OS X, or any other platform
that doesn't yet have a large packaging distribution infrastructure.
It would also make Sage releases much more dependent on outside people
and projects--right now if one finds a bug in an upstream package one
can get it into the next sage release rather than wait for upstream to
accept and push it. Also if there were one package repository/system
that all supported platforms used, things would be much simpler.
I have some personal experience with Cython. I'm very greatful for
whoever makes .dep and .rpm packages of Cython, but I do have to admit
that they're not always up to date and I'm glad I can get the new
Cython into Sage before waiting for every packaging system we would
rely on to finally upgrade to the latest.
> The consequence is clearly visible: there are no up-to-date packages
> of Sage for any of the Linux distributions.
I'm playing devils advocate here a bit, but there are up-to-date
binaries of Sage for many Linux distributions, so it would seem that's
(for the moment) a more effective way to go. Note that if they
accepted monolithic packages this would be much easier.
> Which means that for many
> users, installing Sage is actually _harder_ than installing most other
> software.
That is sad but true.
> But don't get me wrong: I am definitely not saying that it is easy to
> make Sage deal with distribution-supplied libraries, and I fully
> understand the historical reasons for shipping Sage with such a large
> number of spkg's.
> What I am saying is that, in the long run, it is much better to steer
> clear of that.
> And therefore, that it would be unwise to add more of your own
> versions
> of libraries, _especially_ when it concerns libraries which every
> distribution
> already ships with (like gcc-lib). I can see that Macports is having
> problems
> with the 'we ship everything ourselves' logic, and I don't see how
> Sage is
> going to avoid that.
Despite my defense of the way things are, I am very excited about the
recent push for the Gentoo port, and I'm curious to see how it will be
maintaining it over, say, the next year. We will see if, for example,
library versioning issues are a major headache or non-issue. Perhaps
eventually this will even become the default way of building/
distributing Sage for Linux. I'm not sure how we would move away from
the monolithic source and binary model for other platforms though,
especially given our mission of being a viable alternative to the
commercial offerings.
- Robert
Robert Bradshaw
> building/distributing Sage for Linux. I'm not sure how we would move
> away from the monolithic source and binary model for other platforms
> though, especially given our mission of being a viable alternative
> to the commercial offerings.
[pressed send too soon]
Another thing I really like about the monolithic approach is that it
makes it really easy to install Sage as non-root, as many times as you
want, and to move and delete them. Of course some users could care
less about this.
- Robert