Re: [sage-devel] Standard Sage Components (was Re: Solaris - what do we expect?)
William Stein
<david....@onetel.net> wrote:
>> If the spkg fixes this problem and doesn't make things *worse* on
>> Solaris, it absolutely should get a positive review. Note that the
>> assuming "CC=gcc" was already in the original cliquer spkg. It is not
>> something added by that ticket.
>
> Fair enough.
>
> In fact, cliquer presents problems on Solaris too. See the ticket I
> created a couple of weeks ago.
>
> http://sagetrac.org/sage_trac/ticket/6852
>
> "cliquer-1.2 fails to build as it cant find Sun compiler (SCons issue)"
>
>
>> If we were discussing including cliquer in the first place, I might
>> have a different opinion.
>
>
> So what would your thoughts be, if someone one to propose package X is
> added, despite the fact it will not build on all of the following?
>
> 1) Build as 32-bit gcc on SPARC
> 2) Build as 64-bit gcc on SPARC
> 3) Build as 32-bit with Sun's compiler on SPARC
> 4) Build as 64-bit with Sun's compiler on SPARC
>
> 5) Build as 32-bit gcc on x64
> 6) Build as 64-bit gcc on x64
> 7) Build as 32-bit with Sun's compiler on x86
> 8) Build as 64-bit with Sun's compiler on x64
I'm pretty much by default against adding any new standard packages to
Sage anytime soon. I can't think of anything even on the horizon.
Maybe some sort of linear programming code is being proposed. Can
anybody else think of anything?
The last two packages that we added to Sage -- cliquer and ratpoints
-- both caused a lot of trouble. I will be much more careful in the
future about adding spkg's.
-- William
David Joyner
>
...
>
>
> I'm pretty much by default against adding any new standard packages to
> Sage anytime soon. I can't think of anything even on the horizon.
> Maybe some sort of linear programming code is being proposed. Can
> anybody else think of anything?
William Stein
>
> On Sun, Sep 13, 2009 at 8:29 PM, William Stein <wst...@gmail.com> wrote:
>>
>
> ...
>
>>
>>
>> I'm pretty much by default against adding any new standard packages to
>> Sage anytime soon. I can't think of anything even on the horizon.
>> Maybe some sort of linear programming code is being proposed. Can
>> anybody else think of anything?
>
>
> PIL and glpk are on the horizon. PIL has already recieved a positive vote.
>
I had forgotten about them -- thanks for pointing them out.
Fortunately, those are both mainstream widely mature packages -- note
that both cliquer and ratpoints are highly specialized with (probably)
one developer. In contrast PIL is the standard Python imaging library
and glpk the standard GPL'd linear programming library. So that
helps. But this time, let's be very careful about how these actually
get included in Sage.
Who is actually doing the work to include either of these? I think
ncohen is working on glpk (?). I don't know who is working on PIL. I
know who proposed it, but not if anybody is doing the actual work.
William
Dr. David Kirkby
>>> I'm pretty much by default against adding any new standard packages to
>>> Sage anytime soon. I can't think of anything even on the horizon.
>>> Maybe some sort of linear programming code is being proposed. Can
>>> anybody else think of anything?
>>
>> PIL and glpk are on the horizon. PIL has already recieved a positive vote.
>>
>
> I had forgotten about them -- thanks for pointing them out.
> Fortunately, those are both mainstream widely mature packages -- note
> that both cliquer and ratpoints are highly specialized with (probably)
> one developer. In contrast PIL is the standard Python imaging library
> and glpk the standard GPL'd linear programming library. So that
> helps. But this time, let's be very careful about how these actually
> get included in Sage.
If something is obscure and presenting problems, should it be moved out
of the standard packages and to optional? Certainly cliquer has issues
on Solaris, with it forcing the use of gcc, then at another point
failing to build it if can't find 'cc' (that was on the first release of
Solaris 10 I would add - it is less problematic on later releases).
Dave
David Joyner
>
> On Sun, Sep 13, 2009 at 5:50 PM, David Joyner <wdjo...@gmail.com> wrote:
>>
>>
>
> I had forgotten about them -- thanks for pointing them out.
> Fortunately, those are both mainstream widely mature packages -- note
> that both cliquer and ratpoints are highly specialized with (probably)
> one developer. In contrast PIL is the standard Python imaging library
> and glpk the standard GPL'd linear programming library. So that
> helps. But this time, let's be very careful about how these actually
> get included in Sage.
>
> Who is actually doing the work to include either of these? I think
> ncohen is working on glpk (?). I don't know who is working on PIL. I
> know who proposed it, but not if anybody is doing the actual work.
http://www.pythonware.com/products/pil/. There is a simpler interface to pil at
http://trac.sagemath.org/sage_trac/ticket/6741 (which I wrote this summer and
possibly Jason might review at some point), but it is not needed for PIL to
be included. What else is needed?
I'm rather short on time, now that teaching has started but I can try
to help out
if any is needed.
>
> William
>
> >
>
William Stein
<david....@onetel.net> wrote:
>
> William Stein wrote:
>
>>>> I'm pretty much by default against adding any new standard packages to
>>>> Sage anytime soon. I can't think of anything even on the horizon.
>>>> Maybe some sort of linear programming code is being proposed. Can
>>>> anybody else think of anything?
>>>
>>> PIL and glpk are on the horizon. PIL has already recieved a positive vote.
>>>
>>
>> I had forgotten about them -- thanks for pointing them out.
>> Fortunately, those are both mainstream widely mature packages -- note
>> that both cliquer and ratpoints are highly specialized with (probably)
>> one developer. In contrast PIL is the standard Python imaging library
>> and glpk the standard GPL'd linear programming library. So that
>> helps. But this time, let's be very careful about how these actually
>> get included in Sage.
>
> If something is obscure and presenting problems, should it be moved out
> of the standard packages and to optional?
others have been actively working on resolving all problems. If
there were an spkg that was causing issues, and nobody were working on
it, and we could remove it without too much trouble, that would be
different (quadpack comes to mind, by the way).
William
> Certainly cliquer has issues
> on Solaris, with it forcing the use of gcc, then at another point
> failing to build it if can't find 'cc' (that was on the first release of
> Solaris 10 I would add - it is less problematic on later releases).
>
> Dave
>
>
>
>
>
>
> >
>
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Jason Grout
> The optional package is pil-1.1.6, which is the current version according to
> http://www.pythonware.com/products/pil/. There is a simpler interface to pil at
> http://trac.sagemath.org/sage_trac/ticket/6741 (which I wrote this summer and
> possibly Jason might review at some point), but it is not needed for PIL to
> be included. What else is needed?
I'm short on time at the moment as well, so if anyone else wants to
review it, go right ahead!
Jason
Tom Boothby
while now, but it would be nice to have that cython interface working.
Michelle Callaghan - Sun Microsystems
Sorry for crashing your thread, but I was just searching around to see
if anyone was running Sage on Solaris and I came upon your dicussions,
I just wondered if there is a specific customer requirement that you
know of for Sage on Sun as we would love to work with Sage, if there
is a need for this.
If there is anything I can do to help let me know and I will see what
I can do.
Cheers
Michelle
On Sep 14, 1:29 am, William Stein <wst...@gmail.com> wrote:
> On Sun, Sep 13, 2009 at 5:16 PM, Dr. David Kirkby
>
>
>
> <david.kir...@onetel.net> wrote:
> >> If the spkg fixes this problem and doesn't make things *worse* on
> >>Solaris, it absolutely should get a positive review. Note that the
> >> assuming "CC=gcc" was already in the original cliquer spkg. It is not
> >> something added by that ticket.
>
> > Fair enough.
>
Minh Nguyen
<SNIP>
>> So what would your thoughts be, if someone one to propose package X is
>> added, despite the fact it will not build on all of the following?
>>
>> 1) Build as 32-bit gcc on SPARC
>> 2) Build as 64-bit gcc on SPARC
>> 3) Build as 32-bit with Sun's compiler on SPARC
>> 4) Build as 64-bit with Sun's compiler on SPARC
>>
>> 5) Build as 32-bit gcc on x64
>> 6) Build as 64-bit gcc on x64
>> 7) Build as 32-bit with Sun's compiler on x86
>> 8) Build as 64-bit with Sun's compiler on x64
Working on platforms with GCC installed means that I don't need to
first compile GCC myself. I have had relatively little trouble with
doing porting work and testing on sage.math, bsd.math, virtualized
Linux guests on boxen.math, t2.math using GCC, and various Fedora, Red
Hat and SUSE machines on SkyNet.
The trouble comes in when I want to test or do porting work on a SPARC
machine with Solaris and Sun's compiler. At the moment, t2.math is
equipped with both GCC and Sun compiler. I can set GCC as the default
compiler with these export lines by David Kirkby:
export PATH=/usr/local/gcc-4.4.1-sun-linker/bin:/usr/local/bin2:/usr/bin:/usr/ccs/bin:/usr/local/bin:/usr/sfw/bin:/bin:/usr/sbin
export LD_LIBRARY_PATH=/usr/local/gcc-4.4.1-sun-linker/lib:/usr/local/lib
export SAGE_FORTRAN=/usr/local/gcc-4.4.1-sun-linker/bin/gfortran
export SAGE_FORTRAN_LIB=/usr/local/gcc-4.4.1-sun-linker/lib/libgfortran.so
If I also want to do testing or porting with the Sun compiler on
t2.math, are there equivalent export commands I can use? Of course it
would be nice to have a box with just the Sun compiler, linker and
library paths all setup whenever one login, and another box with GNU
equivalent. In the absense of that, switching between compilers,
linkers and library paths on the same box can be confusing.
> The last two packages that we added to Sage -- cliquer and ratpoints
> -- both caused a lot of trouble.
The building problem with cliquer is pretty much solved I think. As
for ratpoint, the issue is that it should build OK with GCC 3.4.x. Is
there a machine somewhere with GCC 3.4.x that I can use to have a shot
at this problem?
--
Regards
Minh Van Nguyen
Pat LeSmithe
> Sorry for crashing your thread, but I was just searching around to see
> if anyone was running Sage on Solaris and I came upon your dicussions,
> I just wondered if there is a specific customer requirement that you
> know of for Sage on Sun as we would love to work with Sage, if there
> is a need for this.
>
> If there is anything I can do to help let me know and I will see what
> I can do.
It would be interesting and perhaps very useful to hear from the Project
Fortress team about the prospects for high-performance computing (HPC)
with Sage.
Fortress is actually a new programming language, distinct from Python:
http://projectfortress.sun.com/Projects/Community
http://research.sun.com/projects/plrg/Publications/
It has at least one goal I think the Sage community would appreciate,
namely, that programs should have a mathematical representation:
http://projectfortress.sun.com/Projects/Community/wiki/MathSyntaxInFortress
Implicit parallelism is also appealing. Does Fortress have a foreign
function interface?
Disclaimer: I'm not familiar with Fortress. I just discovered it
recently, during a digression.
Dr. David Kirkby
I created '/opt/t2/bin/sun-studio' You can source that instead of
whatever you use for gcc. It does NOT set SAGE_FORTRAN, so you might
want to add something like
export SAGE_FORTRAN=/opt/SUNWspro/bin/f95
but I'm not exactly sure what compiler will work best, so you will have
to try.
>> The last two packages that we added to Sage -- cliquer and ratpoints
>> -- both caused a lot of trouble.
>
> The building problem with cliquer is pretty much solved I think. As
> for ratpoint, the issue is that it should build OK with GCC 3.4.x. Is
> there a machine somewhere with GCC 3.4.x that I can use to have a shot
> at this problem?
/usr/sfw/bin/gcc is version 3.4.3, but from what I gather, flint will
not build with an old compiler either, so we are sort of needing
something more modern. It would have been good if Sage would have built
with the compiler supplied with Solaris, but it will not.
The more immediate problem is that Sage's configure script will not
allow you to use the Sun compiler at all. I manually hacked the
configure script to get around that, but I am looking at a better
solution - see my next post.
Dave
Michelle Callaghan - Sun Microsystems
On Sep 17, 9:39 pm, Pat LeSmithe <qed...@gmail.com> wrote:
> Michelle Callaghan - Sun Microsystems wrote:
>
> > Sorry for crashing your thread, but I was just searching around to see
> > if anyone was running Sage on Solaris and I came upon your dicussions,
> > I just wondered if there is a specific customer requirement that you
> > know of for Sage on Sun as we would love to work with Sage, if there
> > is a need for this.
>
> > If there is anything I can do to help let me know and I will see what
> > I can do.
>
> It would be interesting and perhaps very useful to hear from the Project
> Fortress team about the prospects for high-performance computing (HPC)
> with Sage.
>
> Fortress is actually a new programming language, distinct from Python:
>
> It has at least one goal I think the Sage community would appreciate,
> namely, that programs should have a mathematical representation:
>
> Implicit parallelism is also appealing. Does Fortress have a foreign
> function interface?
>
> Disclaimer: I'm not familiar with Fortress. I just discovered it
> recently, during a digression.
too new to have any real news on this yet, but it will be something to
consider in the future. Our HPC team is in discussion with Sage about
Solaris and fixing these issues, they are making headway and we hope
to see some developments soon.
Sorry I can't be more specific at this time, as the work is still
underway.
Dr. David Kirkby
>
> Sorry for crashing your thread, but I was just searching around to see
> if anyone was running Sage on Solaris and I came upon your dicussions,
> I just wondered if there is a specific customer requirement that you
> know of for Sage on Sun as we would love to work with Sage, if there
> is a need for this.
>
> If there is anything I can do to help let me know and I will see what
> I can do.
>
> Cheers
> Michelle
Hi,
I'm one of the people working on the port of Sage to Sun's Solaris. I'm
mainly working on SPARC, but intend doing likewise on x86. Sage does
run, but building it is not straightforward.
A Sun T5240 was donated by Sun to the Sage project. That is now running
Solaris 10 update 7. We wish to produce binaries which will run on any
Solaris 10 release, so we should build them on a machine running the
first release of Solaris 10 (03/2005 if I recall correctly). If you are
aware of any way we can get access to such a machine, it would be useful.
I had suggested to William Stein that he purchase an older machine (the
T5240 is not supported on the first release of Solaris 10), but he has
issues with rack space, cooling and power, so it is not even possible to
get a small 1U machine. I have some older SPARCs at home, but don't
really have the bandwidth to make creating huge binaries and uploading
them. I also have issues with power and cooling, in that the power costs
come directly from my own pocket!
If you are aware of anyone who has some spare time, and would like to
help on the project, we could do with more Solaris developers. Currently
we have only build Sage as 32-bit on gcc. We intend ultimately to create
a 64-bit version with the Sun compiler, but there are several issues
of GNUisms, which mean Sage will not build with Sun's compiler. I am
sorting some of those out. In the latest alpha release, some of the
changes were:
#6759: David Kirkby: Update sqlite to latest release - needed for
Sun's compiler [Reviewed by Minh Van Nguyen]
#6609: David Kirkby: GNUism in lcalc-20080205.p2 passing GNU flags
directly to the Sun assembler. [Reviewed by Minh Van Nguyen]
Dave
Marshall Hampton
Abshoff then critiqued the optional spkg and gave me a list of things
I needed to do. I think I have done all of them, and I would very
much like to see lrs made standard to move the polytope functionality
forward. I really doubt that lrs will cause significant problems,
even on Solaris (I can compile it on t2, and I know nothing about
Solaris).
The only reason I have waited to propose lrs as standard is that I
found my last attempt quite depressing. Actually, I think our voting
"system" isn't really formal enough. To be perfectly honest, I think
the reality of it is that I need to convince you (William Stein), the
current release manager, and then make a reasonable sounding case to
everyone else.
I would also like to propose biopython as standard too. Its a very
mature project, almost entirely in python, and in the last year they
have really cleaned things up (in terms of dependencies and removing
deprecated code, and doing some nice refactoring of heavily used
pieces). I have been waiting to have time to really focus on it; I
think the only thing needed from a Sage perspective is some tests (it
has its own very extensive test suite, but they aren't doctests).
-Marshall
William Stein
>
> I proposed making the lrs spkg standard about a year ago; Micheal
> Abshoff then critiqued the optional spkg and gave me a list of things
> I needed to do. I think I have done all of them, and I would very
> much like to see lrs made standard to move the polytope functionality
> forward. I really doubt that lrs will cause significant problems,
> even on Solaris (I can compile it on t2, and I know nothing about
> Solaris).
>
> The only reason I have waited to propose lrs as standard is that I
> found my last attempt quite depressing. Actually, I think our voting
> "system" isn't really formal enough. To be perfectly honest, I think
> the reality of it is that I need to convince you (William Stein), the
> current release manager, and then make a reasonable sounding case to
> everyone else.
-- William
>
> I would also like to propose biopython as standard too. Its a very
> mature project, almost entirely in python, and in the last year they
> have really cleaned things up (in terms of dependencies and removing
> deprecated code, and doing some nice refactoring of heavily used
> pieces). I have been waiting to have time to really focus on it; I
> think the only thing needed from a Sage perspective is some tests (it
> has its own very extensive test suite, but they aren't doctests).
>
> -Marshall
>
> On Sep 13, 7:29 pm, William Stein <wst...@gmail.com> wrote:
>
>>
>> I'm pretty much by default against adding any new standard packages to
>> Sage anytime soon. I can't think of anything even on the horizon.
>> Maybe some sort of linear programming code is being proposed. Can
>> anybody else think of anything?
>>
>> The last two packages that we added to Sage -- cliquer and ratpoints
>> -- both caused a lot of trouble. I will be much more careful in the
>> future about adding spkg's.
>>
>> -- William
> >
>
Dr. David Kirkby
> I proposed making the lrs spkg standard about a year ago; Micheal
> Abshoff then critiqued the optional spkg and gave me a list of things
> I needed to do. I think I have done all of them, and I would very
> much like to see lrs made standard to move the polytope functionality
> forward. I really doubt that lrs will cause significant problems,
> even on Solaris (I can compile it on t2, and I know nothing about
> Solaris).
My belief is that it should build with Sun's compiler, and in 64-bit
mode before being made a standard package. Otherwise a port of Sage to
64-bit will just get more and more difficult.
Have you tried with SAGE64=yes on Solaris? There are many things in Sage
which will not build in 64-bit mode on Solaris with gcc, but I'm really
keen that list is not increased.
I intend updating prereq so Sage can be tested with the Sun compiler. At
present that fails very early, as the 'configure' script in prereq exits
with the Sun compiler. However, it is not hard to hack that yourself -
just remove the line that causes it to exit when it finds the gcc
version is not suitable. I hope to have a better solution soon.
William Stein
> On Tue, Sep 22, 2009 at 4:53 AM, Marshall Hampton <hamp...@gmail.com> wrote:
>>
>> I proposed making the lrs spkg standard about a year ago; Micheal
>> Abshoff then critiqued the optional spkg and gave me a list of things
>> I needed to do. I think I have done all of them, and I would very
>> much like to see lrs made standard to move the polytope functionality
>> forward. I really doubt that lrs will cause significant problems,
>> even on Solaris (I can compile it on t2, and I know nothing about
>> Solaris).
>>
>> The only reason I have waited to propose lrs as standard is that I
>> found my last attempt quite depressing. Actually, I think our voting
>> "system" isn't really formal enough. To be perfectly honest, I think
>> the reality of it is that I need to convince you (William Stein), the
>> current release manager, and then make a reasonable sounding case to
>> everyone else.
>
> Start convincing us! I don't even know what "lrs" is.
Oh, you're right, I think our voting procedure is not formal enough.
I've been thinking this for quite a while. I would definitely be
interested in having a good discussion about how to make it more
formal. Could we do that first before lrs?
First, some obvious questions:
1. How does *Python* do package inclusion in their standard
library? How is it decided? All I know is that Guido has final say,
but I'm sure there is much more to it than that. I think at a bare
minimum they require a formal PEP. If so, maybe we should require
a formal SEP (=Sage enhancement proposal) before even considering
including a new package with Sage.
2. What about other projects? E.g., how does Ubuntu, Debian, etc.
decide what is in their standard distribution (the one that comes on
CD)?
If anybody reading this has experience with any of the above projects
or related projects, please share your experience. Once that is done,
I think we'll be able to have a productive discussion.
-- William
William Stein
<david....@onetel.net> wrote:
>
> Marshall Hampton wrote:
>> I proposed making the lrs spkg standard about a year ago; Micheal
>> Abshoff then critiqued the optional spkg and gave me a list of things
>> I needed to do. I think I have done all of them, and I would very
>> much like to see lrs made standard to move the polytope functionality
>> forward. I really doubt that lrs will cause significant problems,
>> even on Solaris (I can compile it on t2, and I know nothing about
>> Solaris).
>
> My belief is that it should build with Sun's compiler, and in 64-bit
> mode before being made a standard package. Otherwise a port of Sage to
> 64-bit will just get more and more difficult.
>
> Have you tried with SAGE64=yes on Solaris? There are many things in Sage
> which will not build in 64-bit mode on Solaris with gcc, but I'm really
> keen that list is not increased.
You have a very good point.
We should also consider having two Sage versions -- one with *all*
optional free packages, and one without. That way optional packages
would get much much better testing, and be on a far better footing
than they are now. We should really learn from projects like R, Perl,
Ubuntu, etc. that all have extremely robust well tested optional
package systems.
-- William