Vote on bugs to be fixed for sage-4.4 "stabilization release".
William Stein
Lately it seems like Sage has gotten more bugs rather than less. I
think it's time for a stabilization release -- say Sage-4.4 -- that
fixes the absolutely most annoying of these bugs.
What are *your* top 4 bugs that are in sage-4.3.2 that you really wish
were fixed?? If you care, respond to this thread with a list of 4
bugs which *must* have corresponding trac tickets. We'll take the
most popular reported bugs, and the ones that aren't impossible will
all be upgraded to blockers and fixed for sage-4.4, which will be a
bug fix release.
My top 4:
1. Right now in the Sage notebook the cells lose focus in many situations:
http://trac.sagemath.org/sage_trac/ticket/8231
2. Defect: tab completion broken for many parent objects (new)
http://trac.sagemath.org/sage_trac/ticket/8223
3. Sage takes too long to startup still:
http://trac.sagemath.org/sage_trac/ticket/8254
4. %time broken in the notebook:
http://trac.sagemath.org/sage_trac/ticket/8225
PLEASE: No discussion about releases numbers, the GPL, etc., in this thread!
-- William
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Dr. David Kirkby
> Hi,
>
> Lately it seems like Sage has gotten more bugs rather than less. I
> think it's time for a stabilization release -- say Sage-4.4 -- that
> fixes the absolutely most annoying of these bugs.
I think your idea is excellent.
> What are *your* top 4 bugs that are in sage-4.3.2 that you really wish
> were fixed??
I'd like to see the Solaris build get back on track. Two recent changes in Sage
broke this. In each case, Sage is released without having sufficient time for
testing IMHO.
My top 3 bugs - in order of annoyance. (I've not really got a fourth.)
1) http://trac.sagemath.org/sage_trac/ticket/7867
"GCC reports incorrect flags compiling descent_two_isogeny.c on Solaris 10"
This bug was introduced when #6583 got merged. It is a direct result of that, as
Minh has found. I'm not implying that #6583 is necessarily bad code, as I don't
fully understand the issues. But the addition of the elliptic curves code in
#6583 broke the Solaris build.
2) http://trac.sagemath.org/sage_trac/ticket/7932
"_Complex_I undeclared - a new bug totally stops a Solaris 10 build."
This is fixed by Roberts changes, but those changes are not integrated into Sage.
3) http://trac.sagemath.org/sage_trac/ticket/8191
"Add iconv need for Solaris, and possibly Cygwin too."
This is needed for R to build on Solaris, but you might wish to avoid adding a
new standard package in a stabilisation release. I think it might be wise to
leave this, but I'll leave that decision to you. I can create the iconv.spkg,
but someone else would need to make the necessary changes so that iconv gets built.
I believe first two really do need to be fixed. The third is less important, and
perhaps a bit too far for a bug-fix release.
Let me know what you think about adding iconv in 4.4.
Dave
Jason Grout
> Hi,
>
> Lately it seems like Sage has gotten more bugs rather than less. I
> think it's time for a stabilization release -- say Sage-4.4 -- that
> fixes the absolutely most annoying of these bugs.
>
Just curious---do you see this as the next release, building on the
alpha that was just released?
I'd say the cookie bug(s?) in discussion on another thread is in my top
4 annoying bugs: #8249. I haven't seen this recently, but others say it
is cropping up a lot, and I have seen it a lot in the past (maybe 6
months ago).
Thanks,
Jason
William Stein
<jason...@creativetrax.com> wrote:
> On 02/12/2010 06:32 PM, William Stein wrote:
>>
>> Hi,
>>
>> Lately it seems like Sage has gotten more bugs rather than less. I
>> think it's time for a stabilization release -- say Sage-4.4 -- that
>> fixes the absolutely most annoying of these bugs.
>>
>
>
> Just curious---do you see this as the next release, building on the alpha
> that was just released?
No. Sage-4.3.3 will be released as normal. However, I think there is
great value in doing a extra-doubly-stable super-tested sage-4.4,
which fixes a "top 10 list" of very annoying issues.
> I'd say the cookie bug(s?) in discussion on another thread is in my top 4
> annoying bugs: #8249. I haven't seen this recently, but others say it is
> cropping up a lot, and I have seen it a lot in the past (maybe 6 months
> ago).
Thank you for your suggestion. Thanks to David Kirkby also for his.
William
John Cremona
polynomials over number fields too often gives stupid results
(reducible factors, factors whose product is not the original, etc).
I am still not clear to what extent this is possible before we upgrade
the pari library (we use pari-2.3.3, the latest so-called stable
release is 2.3.5 and the current so-called development version is
2.4.2).
John
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>
Florent Hivert
> Lately it seems like Sage has gotten more bugs rather than less. I
> think it's time for a stabilization release -- say Sage-4.4 -- that
> fixes the absolutely most annoying of these bugs.
> What are *your* top 4 bugs that are in sage-4.3.2 that you really wish
> were fixed?? If you care, respond to this thread with a list of 4
> bugs which *must* have corresponding trac tickets. We'll take the
> most popular reported bugs, and the ones that aren't impossible will
> all be upgraded to blockers and fixed for sage-4.4, which will be a
> bug fix release.
My top 4:
Doctests drive me crazy:
1. whitespace error in doctest causes A Mysterious Error.
http://trac.sagemath.org/sage_trac/ticket/7993
The documentation is not in a very good shape:
2. Reference manual: class hierarchy, inherited members, underscore variables
http://trac.sagemath.org/sage_trac/ticket/7549
Which could be used to solve the following issues
2+1/2 Improve sphinx rendering of categories in reference manual.
http://trac.sagemath.org/sage_trac/ticket/7448
2+2/3 Annoying warnings when building the HTML reference manual
http://trac.sagemath.org/sage_trac/ticket/8244
2+3/4 Build the reference manual incrementally
http://trac.sagemath.org/sage_trac/ticket/6495
3. Sage takes too long to startup still:
http://trac.sagemath.org/sage_trac/ticket/8254
As usual, there are three kinds of people in the world: those who can count,
and those who cannot.
Cheers,
Florent
Simon King
On 13 Feb., 01:32, William Stein <wst...@gmail.com> wrote:
> Lately it seems like Sage has gotten more bugs rather than less. I
> think it's time for a stabilization release -- say Sage-4.4 -- that
> fixes the absolutely most annoying of these bugs.
Good idea!
> What are *your* top 4 bugs that are in sage-4.3.2 that you really wish
> were fixed??
There are two bugs in conversion (not coercion) for polynomial rings.
They gave me a very hard time when I tried to improve infinite
polynomial rings:
1. http://trac.sagemath.org/sage_trac/ticket/5917 Failing conversions
for multipolynomial rings over fraction fields
2. http://trac.sagemath.org/sage_trac/ticket/7654 Conversion bug in
MPolynomialRing_libsingular
Then there are situations where polynomial computations segfault,
which I find very bad:
3. http://trac.sagemath.org/sage_trac/ticket/7795 MPolynomialRing
segfaults when getting high exponents
4. http://trac.sagemath.org/sage_trac/ticket/7794
PolynomialRing_integral_domain ignores Ctrl-C and segfaults
I hope other people like these bugs as well. The problem is that there
are many developers with many different interests. So, I wonder if
there will really be bugs that get more than one or two votes.
Cheers,
Simon
Georg S. Weber
1. http://trac.sagemath.org/sage_trac/ticket/8258
get "make documentation" relocation-safe:
The very annoying "unnecessarily rebuilding docs" topic
2. http://trac.sagemath.org/sage_trac/ticket/7005
singular -- port to cygwin:
actually, this should be a very easy one
3. http://trac.sagemath.org/sage_trac/ticket/7987
Most extensions don't need to be listed in module_list:
Already with patch and for review. I still essentially dislike
"#clib" and friends, but the topic of this ticket certainly is a step
in the right direction of storing the build information of .pyx files
more locally
4. http://trac.sagemath.org/sage_trac/ticket/5296
Update the OS X Readme:
IMHO, the current OS X Readme is getting embarrassing
Cheers,
Georg
Robert Miller
can remember this getting fixed three, maybe four times. I fixed it
myself once, and refereed at least one other fix.
Looking on trac, you'd think this was fixed, but in fact it took me
four seconds to come up with an example where it's still an issue:
sage: G = posets.IntegerPartitions(5).hasse_diagram()
sage: G.show()
The "axes_pad" solution is not a viable one, as changing vertex size
and proportions can change the amount of padding needed. Someone needs
to solve this right, and we need a way of keeping it from regressing!
So I guess my second issue would be to have doctests which produce
plots to actually verify that they're not getting garbage.
Relevant tickets to this issue:
1) http://trac.sagemath.org/sage_trac/ticket/8210
2) http://trac.sagemath.org/sage_trac/ticket/7299
3) http://trac.sagemath.org/sage_trac/ticket/7004
4) http://trac.sagemath.org/sage_trac/ticket/5938
--
Robert L. Miller
http://www.rlmiller.org/
Harald Schilly
> Graphs plot with their most outward vertices chopped off.
* I don't know if there is a ticket for this, but i hate the "error
code: -7 ... and now it's getting ugly." of jsMath. There are really
many user who are confused by that. Just add a string to that: "You
have to install some fonts that you can find <here>".
* startup time
* cvxopt (i don't know how to do a proper spkg for that, test it on
all systems if it works, there were also patches for solaris, etc...)
http://trac.sagemath.org/sage_trac/ticket/6456
And i think we do not need to talk about the tab completion. That's
really a blocker because it seriously degrades Sage's usability.
H
David Joyner
Jason Grout
> On Feb 13, 11:43 pm, Robert Miller<r...@rlmiller.org> wrote:
>> Graphs plot with their most outward vertices chopped off.
> +1 vote
>
> * I don't know if there is a ticket for this, but i hate the "error
> code: -7 ... and now it's getting ugly." of jsMath. There are really
> many user who are confused by that. Just add a string to that: "You
> have to install some fonts that you can find<here>".
>
OR you have to complain to your Sage administrator to do
sage -i jsmath-image-fonts-1.3p3.spkg (or whatever the current spkg is).
Jason
Jason Grout
> Graphs plot with their most outward vertices chopped off. I think I
> can remember this getting fixed three, maybe four times. I fixed it
> myself once, and refereed at least one other fix.
>
> Looking on trac, you'd think this was fixed, but in fact it took me
> four seconds to come up with an example where it's still an issue:
>
> sage: G = posets.IntegerPartitions(5).hasse_diagram()
> sage: G.show()
>
> The "axes_pad" solution is not a viable one, as changing vertex size
> and proportions can change the amount of padding needed. Someone needs
> to solve this right, and we need a way of keeping it from regressing!
> So I guess my second issue would be to have doctests which produce
> plots to actually verify that they're not getting garbage.
>
I agree that the fix in 4.3.2(?) is a bandaid that solves most common cases.
I think maybe a call to axis('tight') at the right place might be the
right thing to do. See http://sagenb.org/home/pub/1595/ for the effect.
Thanks,
Jason
Jason Grout
For context, here is the post to the matplotlib list when I was trying
to deal with these issues originally:
http://www.mail-archive.com/matplotl...@lists.sourceforge.net/msg13298.html
In the end, I decided that expanding the axes beyond what the user asked
for (via axes_pad) was better than just expanding the clipping box,
since just expanding the clipping box would look really funny when
frame=True (i.e., things would be hanging outside of the frame).
I don't know why I didn't see axis('tight') before, or maybe saw it and
didn't end up using it. It looks like the option has some side effects
that we probably at least want to think about, though:
http://matplotlib.sourceforge.net/api/pyplot_api.html#matplotlib.pyplot.axis
Thanks,
Jason
dmharvey
interrupts my workflow (trying to find the broken whitespace...):
http://trac.sagemath.org/sage_trac/ticket/7993
Then a few p-adic ones:
http://trac.sagemath.org/sage_trac/ticket/8240
http://trac.sagemath.org/sage_trac/ticket/8181
http://trac.sagemath.org/sage_trac/ticket/8260
david
Dan Drake
> What are *your* top 4 bugs that are in sage-4.3.2 that you really wish
> were fixed??
I would like Sage to build with SAGE_CHECK=yes, on Linux at least.
Right now, pyprocessing fails this check; removing it is #6503.
SQLAlchemy also fails; fixing that is #7091.
I also found a tiny problem with SageTeX's spkg-check, which has been
fixed and awaits review: #8255.
Dan
--
--- Dan Drake
----- http://mathsci.kaist.ac.kr/~drake
-------
Nicolas M. Thiery
> really a blocker because it seriously degrades Sage's usability.
Arr, my patch has been up since 6 days, but I had forgotten to set it
as "needs review". Done. #8233.
Best,
Nicolas
--
Nicolas M. Thi�ry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/
Georg S. Weber
Christopher Schwan:
- update cvxopt, ticket #6456
- remove pyprocessing, ticket #6503
- update networkx, ticket #7608
- patch combinat, ticket #7803
Robert Bradshaw
Though as mentioned on the other thread, not for a stabilization
release, right?
- Robert
Robert Bradshaw
They are part of Cython 0.12.1, which went into the latest release.
> 3) http://trac.sagemath.org/sage_trac/ticket/8191
> "Add iconv need for Solaris, and possibly Cygwin too."
>
> This is needed for R to build on Solaris, but you might wish to
> avoid adding a new standard package in a stabilisation release. I
> think it might be wise to leave this, but I'll leave that decision
> to you. I can create the iconv.spkg, but someone else would need to
> make the necessary changes so that iconv gets built.
>
> I believe first two really do need to be fixed. The third is less
> important, and perhaps a bit too far for a bug-fix release.
>
> Let me know what you think about adding iconv in 4.4.
>
> Dave
>
>
Robert Bradshaw
> I hope other people like these bugs as well. The problem is that there
> are many developers with many different interests. So, I wonder if
> there will really be bugs that get more than one or two votes.
I'm voting my favorites from others out of this list, so at least all
of mine will have at least two votes :).
1. Sage startup time: http://trac.sagemath.org/sage_trac/ticket/
8254
2. Mysterious error in doctest: http://trac.sagemath.org/sage_trac/ticket/7993
3. Notebook cookes error: http://trac.sagemath.org/sage_trac/ticket/8249
4. jsMath's "error code: -7 ... and now it's getting ugly"
- Robert
Bjarke Hammersholt Roune
wrong papers, so that would be my top pick
http://trac.sagemath.org/sage_trac/ticket/6472
and I agree on the startup time
Dr. David Kirkby
I just put a comment on the second trac ticket about the startup time on a quite
old Sun Blade 1000. Sage is only taking 8 seconds to start on that. Although the
processors are not that quick (dual 900 MHz), the disks are 15,000 rpm and use a
2 Gbit/s fibre channel interface. (Though it is call fibre channel, it actually
uses copper!) So the disks are local, and disks fast.
It is running a UFS file system on Solaris 10.
It suggests to me the time to read data from disk might be more of an issue than
raw CPU power.
Dave
William Stein
I've created this trac wiki page with a subset of the 10 most
important current bug/issues in Sage, according to votes in this
thread:
http://trac.sagemath.org/sage_trac/wiki/stab1
These are all bugs/issues that many people care about. None are
highly specialized.
So if anybody has some time to donate to working on Sage, and you want
to make an impact that people will definitely appreciate, work on one
of the bugs listed at http://trac.sagemath.org/sage_trac/wiki/stab1
After seeing the actual bugs, I no longer see the wisdom of making a
specific release targeting them. The problem is that they are all
potentially hard, and there is no telling when they will all be
resolved. However, listing them all together and seeing that a lot of
people care about them will help encourage people to work on them.
As people make progress on any of these, feel free to respond to this
thread with an update.
It still makes sense to have the next Sage release be Sage-4.3.4.
Note that nobody has stepped up to be release manager yet, so that
release is currently *NOT* being made by anyone, even though if
somebody did the work, we could easily have an alpha in a day, as
there are 52 tickets with positive review right now:
http://trac.sagemath.org/sage_trac/report/11
A lot are in combinatorics and graph theory, so if somebody in the
sage-combinat group wants to be the release manager for sage-4.3.4,
that could make a lot of sense.
-- William
Robert Bradshaw
> Hi,
>
> I've created this trac wiki page with a subset of the 10 most
> important current bug/issues in Sage, according to votes in this
> thread:
>
> http://trac.sagemath.org/sage_trac/wiki/stab1
>
> These are all bugs/issues that many people care about. None are
> highly specialized.
>
> So if anybody has some time to donate to working on Sage, and you want
> to make an impact that people will definitely appreciate, work on one
> of the bugs listed at http://trac.sagemath.org/sage_trac/wiki/stab1
>
> After seeing the actual bugs, I no longer see the wisdom of making a
> specific release targeting them. The problem is that they are all
> potentially hard, and there is no telling when they will all be
> resolved. However, listing them all together and seeing that a lot of
> people care about them will help encourage people to work on them.
Just a thought, would knocking out this important list of bugs be a
good goal for Sage 5.0?
- Robert
William Stein
That's an AWESOME idea! Plus some coverage goal, e.g., 85%.
mhampton
-Marshall
On Mar 2, 1:01 pm, Robert Bradshaw <rober...@math.washington.edu>
wrote:
> On Mar 1, 2010, at 4:26 AM, William Stein wrote:
>
>
>
> > Hi,
>
> > I've created this trac wiki page with a subset of the 10 most
> > important current bug/issues in Sage, according to votes in this
> > thread:
>
> > http://trac.sagemath.org/sage_trac/wiki/stab1
>
> > These are all bugs/issues that many people care about. None are
> > highly specialized.
>
> > So if anybody has some time to donate to working on Sage, and you want
> > to make an impact that people will definitely appreciate, work on one
> > of the bugs listed athttp://trac.sagemath.org/sage_trac/wiki/stab1
David Kirkby
> Just a thought, would knocking out this important list of bugs be a good
> goal for Sage 5.0?
>
> - Robert
It is certainly unusual the way Sage version numbers go. In just about
any other software project Assuming the version is of the form X.Y.Z,
X is incremented if there is major new functionality
Y is incremented if there is added, but less major functionality
Z is incremented when bug fixes are made.
I think someone might be a bit disappointed if they update Sage to
5.0.0, without seeing some tangible new functionality.
(I did think of requesting it was called 5.0 when the Solaris port was
finished, but guess that would not be too popular!)
Of course, software called be called by the names of birds, plants,
big cats (as Apple do), or anything else you choose. But the way most
software numbering works is that X in incremented when there is
significant new functionality added, and Z incremented for bug fixes.
Dave
William Stein
> On 2 March 2010 19:01, Robert Bradshaw <robe...@math.washington.edu> wrote:
>
>> Just a thought, would knocking out this important list of bugs be a good
>> goal for Sage 5.0?
>>
>> - Robert
>
> It is certainly unusual the way Sage version numbers go. In just about
> any other software project Assuming the version is of the form X.Y.Z,
>
> X is incremented if there is major new functionality
> Y is incremented if there is added, but less major functionality
> Z is incremented when bug fixes are made.
>
> I think someone might be a bit disappointed if they update Sage to
> 5.0.0, without seeing some tangible new functionality.
I'll give disappointed customers a full money-back refund.
By the way, OS X 10.6 was a major new release of OS X, and the big
claim that Jobs made when announcing it was: "no new features!" It
was all about optimization all over the place, better 64-bit support
under the hood, etc. My impression is that this sort of "quality
improvement under the hood" -- not necessarily new features -- is
something a lot of users really, really value, especially in free
software.
> (I did think of requesting it was called 5.0 when the Solaris port was
> finished, but guess that would not be too popular!)
What's wrong with that goal? It was one of the main goals for
Sage-4.0, after all :-). The goals for Sage-4.0 were something like:
* 75% doctest coverage
* OS X 64-bit port
* Solaris 10 support
* Switch from maxima to Pynac for symbolics.
The above goals were I think great motivation for sage-4.0, and really
helped focus and drive development.
So again, I don't think Solaris 10 support being a goal for 5.0 is at
all unreasonable. And now are chances of success are even high, due
to all the hard work everybody has done.
One worry about that laundry list being the goals for 5.0 is that it
is too complicated and hard to remember. It's pretty cool that I can
so easily just remember what the goal list was for sage-4.0. It
would be nice if the 5.0 goal list was similarly comprehensible. How
about:
1. 85% doctest coverage score
2. Official Solaris 10 support
3. Build with SAGE_CHECK=yes works on all platforms, *and* every
package has at least some spkg-check in it, that checks something.
4. Greatly improve the Sage startup time. This needs to be made
precise, e.g., the following should take < 0.5 seconds when run
repeatedly from a scratch disk on sage.math:
time sage -c "print factor(2010)"
Right now it takes over 1.5 seconds every time.
wstein@sage:~$ time sage -c "print factor(2010)"
2 * 3 * 5 * 67
real 0m1.535s
user 0m1.140s
sys 0m0.460s
The above spreads the goals around nicely. 1 is about user
documentation and better library testing. 2 is about better platform
support. 3 is about better quality in our build system, and will pay
many rewards later as we upgrade packages (finding bugs as early as
possible in the build cycle), and 4 is something every single user
will really notice. Note that 4 is probably quite a lot of work all
over the place, and its difficulty is not to be underestimated.
Thoughts about goals 1-4 above? Are they something you could remember
a year from now?
William
Dr. David Kirkby
> By the way, OS X 10.6 was a major new release of OS X, and the big
> claim that Jobs made when announcing it was: "no new features!"
Marketing comes into play a lot here. I think there were good reasons, because
10.5 was highly criticised as buggy.
> It
> was all about optimization all over the place, better 64-bit support
> under the hood, etc. My impression is that this sort of "quality
> improvement under the hood" -- not necessarily new features -- is
> something a lot of users really, really value, especially in free
> software.
I agree.
>> (I did think of requesting it was called 5.0 when the Solaris port was
>> finished, but guess that would not be too popular!)
>
> What's wrong with that goal?
> It
> would be nice if the 5.0 goal list was similarly comprehensible. How
> about:
>
> 1. 85% doctest coverage score
> 2. Official Solaris 10 support
Could full Solaris support for Sage not be sufficient for a 5.0 release, without
the other requirements? A build on a new platform would be a reason for a major
release. As would it be when the Cygwin port is working.
If so, 5.0 could be out very soon indeed, as I can now build Sage with every
doctest passing (including all the long ones).
> 3. Build with SAGE_CHECK=yes works on all platforms, *and* every
> package has at least some spkg-check in it, that checks something.
> 4. Greatly improve the Sage startup time. This needs to be made
> precise, e.g., the following should take < 0.5 seconds when run
> repeatedly from a scratch disk on sage.math:
> time sage -c "print factor(2010)"
> Right now it takes over 1.5 seconds every time.
> wstein@sage:~$ time sage -c "print factor(2010)"
> 2 * 3 * 5 * 67
> real 0m1.535s
> user 0m1.140s
> sys 0m0.460s
Personaly I don't find that too excessive for a large tool. How long does Gimp
take to start?
Dtrace might be a very useful tool to find out what is using the time up. Dtrace
was developed by Sun, but Apple use it on OS X. I believe Apple have wrapped it
in a GUI called 'Instruments'.
It was using dtrace that I realised that Sage constantly calling 'top' was
bringing my system to its knees. The load average was huge, but no processes
appeared to be running.
'top' is called in an infinite loop, as you can see at:
http://trac.sagemath.org/sage_trac/attachment/ticket/8391/top-to-prstat.patch
If 'top' is not installed, so the Sage library just keeps calling it until the
doctest times out. It's next to impossible to see that sort of thing with ps or
top.
Dave
Dr. David Kirkby
> Dtrace might be a very useful tool to find out what is using the time
> up. Dtrace was developed by Sun, but Apple use it on OS X. I believe
> Apple have wrapped it in a GUI called 'Instruments'.
I should point out that
* You need to be root to use Dtrace
* I'm not aware of any common linux ditro with Dtrace
That rather cuts down the number of people that have access to it.
Dave
William Stein
<david....@onetel.net> wrote:
>> Right now it takes over 1.5 seconds every time.
>> wstein@sage:~$ time sage -c "print factor(2010)"
>> 2 * 3 * 5 * 67
>> real 0m1.535s
>> user 0m1.140s
>> sys 0m0.460s
>
> Personaly I don't find that too excessive for a large tool. How long does
> Gimp take to start?
That's irrelevant. What matters is how long Maple, Mathematica,
Matlab, Maxima, Pari, and Magma take to start.
After repeatedly running the command on sage.math, this is how things stabilize:
Pari 0.030s
Python 0.046s
Maple 0.111s
Maxima 0.456s
Mathematica 0.524s
Matlab 0.844s
Magma 0.971s
Sage 1.658s
This is probably the only benchmark that involves a "function" that
*everybody* uses -- starting up the program. Sage is currently dead
last, and by a lot. Python and Pari are both by far the fastest to
startup, so at least it isn't Python's fault :-).
LOG:
wstein@sage:~$ time echo "2+2;" | sage -python
real 0m0.046s
wstein@sage:~$ time echo "2+2;" | magma
Magma V2.14-9 Wed Mar 3 2010 05:40:30 on sage [Seed = 2126205915]
Type ? for help. Type <Ctrl>-D to quit.
4
Total time: 0.570 seconds, Total memory usage: 6.94MB
real 0m0.971s
--
wstein@sage:~$ time echo "2+2;" | maple
|\^/| Maple 12 (X86 64 LINUX)
._|\| |/|_. Copyright (c) Maplesoft, a division of Waterloo Maple Inc. 2008
\ MAPLE / All rights reserved. Maple is a trademark of
<____ ____> Waterloo Maple Inc.
| Type ? for help.
> 2+2;
4
> quit
memory used=0.8MB, alloc=0.7MB, time=0.01
real 0m0.111s
---
wstein@sage:~$ time echo "2+2;" | math
Mathematica 6.0 for Linux x86 (64-bit)
Copyright 1988-2007 Wolfram Research, Inc.
In[1]:=
In[2]:=
real 0m0.524s
--
wstein@sage:~$ time echo "2+2;" | matlab
Warning: Unable to open display , MATLAB is starting without a display.
You will not be able to display graphics on the screen.
Warning:
MATLAB is starting without a display, using internal event queue.
You will not be able to display graphics on the screen.
< M A T L A B >
Copyright 1984-2006 The MathWorks, Inc.
Version 7.2.0.283 (R2006a)
January 27, 2006
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
>> >>
real 0m0.844s
---
wstein@sage:~$ time echo "2+2;" | sage -maxima
;;; Loading #P"/home/wstein/build/production/sage/local/lib/ecl/defsystem.fas"
;;; Loading #P"/home/wstein/build/production/sage/local/lib/ecl/cmp.fas"
;;; Loading #P"/home/wstein/build/production/sage/local/lib/ecl/sysfun.lsp"
Maxima 5.20.1 http://maxima.sourceforge.net
using Lisp ECL 9.10.2
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1)
(%o1) 4
(%i2)
real 0m0.456s
---
wstein@sage:~$ time echo "2+2;" | sage -gp
GP/PARI CALCULATOR Version 2.3.3 (released)
amd64 running linux (x86-64/GMP-4.2.1 kernel) 64-bit version
compiled: Feb 25 2010, gcc-4.2.4 (Ubuntu 4.2.4-1ubuntu4)
(readline v6.0 enabled, extended help available)
Copyright (C) 2000-2006 The PARI Group
PARI/GP is free software, covered by the GNU General Public License, and
comes WITHOUT ANY WARRANTY WHATSOEVER.
Type ? for help, \q to quit.
Type ?12 for how to get moral (and possibly technical) support.
parisize = 8000000, primelimit = 500000
Goodbye!
real 0m0.030s
---
wstein@sage:~$ time echo "2+2;" | sage
----------------------------------------------------------------------
| Sage Version 4.3.3, Release Date: 2010-02-21 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
sage: sage:
Exiting SAGE (CPU time 0m0.05s, Wall time 0m0.06s).
real 0m1.658s
Paulo César Pereira de Andrade
A suggestion to debug these issues is to use oprofile. A cheat sheet from a
procedure I used some times in the past to debug performance of multi
programs/modules/libraries is:
# rm -fr /var/lib/oprofile/samples/current/; opcontrol --init;
opcontrol --start; <sage-command-here>; opcontrol --dump; opcontrol
--stop; opcontrol --deinit
# opreport --symbols
In my tests, I would run as root, have the computer as idle as possible, and
also do the start/stop of oprofile logging to avoid as much as possible any
noise.
This of couse is Linux specific, but similar tools should exist in other OSes.
An example in the current computer I am, but not logging kernel time
(otherwise, need an uncompressed kernel image):
$ sudo su
# urpmi python-debug
# opcontrol --no-vmlinux
# rm -fr /var/lib/oprofile/samples/current/; opcontrol --init;
opcontrol --start; echo "2+2" | sage; opcontrol --dump; opcontrol
--stop; opcontrol --deinit
Using default event: CPU_CLK_UNHALTED:100000:0:1:1
Using 2.6+ OProfile kernel interface.
Using log file /var/lib/oprofile/samples/oprofiled.log
Daemon started.
Profiler running.
----------------------------------------------------------------------
| Sage Version 4.3.3, Release Date: 2010-02-21 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
sage: 4
sage:
Exiting SAGE (CPU time 0m0.05s, Wall time 0m0.05s).
Stopping profiling.
Killing daemon.
Unloading oprofile module
# opreport --symbols | head -23
samples % image name app name symbol name
13489 22.8685 no-vmlinux no-vmlinux /no-vmlinux
3195 5.4166 libpython2.6.so.1.0 libpython2.6.so.1.0
PyEval_EvalFrameEx
3099 5.2539 libpython2.6.so.1.0 libpython2.6.so.1.0
lookdict_string
1340 2.2718 libpython2.6.so.1.0 libpython2.6.so.1.0
PyDict_GetItem
1268 2.1497 libpython2.6.so.1.0 libpython2.6.so.1.0
__i686.get_pc_thunk.bx
1166 1.9768 libpython2.6.so.1.0 libpython2.6.so.1.0
PyParser_AddToken
1001 1.6970 libpython2.6.so.1.0 libpython2.6.so.1.0 visit_decref
971 1.6462 libpython2.6.so.1.0 libpython2.6.so.1.0 PyDict_Next
959 1.6258 libpython2.6.so.1.0 libpython2.6.so.1.0
visit_reachable
802 1.3597 libc-2.11.so libc-2.11.so memcpy
736 1.2478 ld-2.11.so ld-2.11.so do_lookup_x
707 1.1986 libpython2.6.so.1.0 libpython2.6.so.1.0
_PyType_Lookup
680 1.1528 libpython2.6.so.1.0 libpython2.6.so.1.0
PyObject_Malloc
644 1.0918 ld-2.11.so ld-2.11.so strcmp
595 1.0087 libpython2.6.so.1.0 libpython2.6.so.1.0
tupletraverse
588 0.9969 libpython2.6.so.1.0 libpython2.6.so.1.0 _PyString_Eq
567 0.9613 libpython2.6.so.1.0 libpython2.6.so.1.0
PyString_FromFormatV
540 0.9155 libc-2.11.so libc-2.11.so _int_malloc
531 0.9002 libpython2.6.so.1.0 libpython2.6.so.1.0
PyObject_Free
497 0.8426 bash bash /bin/bash
[I needed to save this email and do a run without firefox running, because it
wants at leat 10% cpu, and would polute the results from inside xulrunner]
This also using mandriva sagemath rpm.
I installed python-debug rpm or it would display all python timings
in a single
chunk... And stopped at the first bash entry. For source build, probably would
need a non stripped build, other distros may have different debug package
formats.
Paulo
Pat LeSmithe
> Pari 0.030s
> Python 0.046s
> Maple 0.111s
> Maxima 0.456s
> Mathematica 0.524s
> Matlab 0.844s
> Magma 0.971s
> Sage 1.658s
>
> This is probably the only benchmark that involves a "function" that
> *everybody* uses -- starting up the program. Sage is currently dead
> last, and by a lot. Python and Pari are both by far the fastest to
> startup, so at least it isn't Python's fault :-).
On a somewhat(?) related note: How difficult is it to implement
analogues of the suspend, resume, and snapshot features that VirtualBox,
VMWare, etc., have?
Dr. David Kirkby
> On Tue, Mar 2, 2010 at 7:56 PM, Dr. David Kirkby
> <david....@onetel.net> wrote:
>>> Right now it takes over 1.5 seconds every time.
>>> wstein@sage:~$ time sage -c "print factor(2010)"
>>> 2 * 3 * 5 * 67
>>> real 0m1.535s
>>> user 0m1.140s
>>> sys 0m0.460s
>> Personaly I don't find that too excessive for a large tool. How long does
>> Gimp take to start?
>
> That's irrelevant. What matters is how long Maple, Mathematica,
> Matlab, Maxima, Pari, and Magma take to start.
> After repeatedly running the command on sage.math, this is how things stabilize:
>
>
> Pari 0.030s
> Python 0.046s
> Maple 0.111s
> Maxima 0.456s
> Mathematica 0.524s
> Matlab 0.844s
> Magma 0.971s
> Sage 1.658s
Fair point.
Personally I have a bit of a problem understanding why I need to worry about a
program starting up in less than 2 s, when I might run something on it which
will take at least one order of magnitude longer, and probably several order of
magnitudes longer.
> wstein@sage:~$ time echo "2+2;" | math
> Mathematica 6.0 for Linux x86 (64-bit)
> Copyright 1988-2007 Wolfram Research, Inc.
>
> In[1]:=
> In[2]:=
>
> real 0m0.524s
Either my Sun Ultra 27 is significantly quicker than sage.math, or Wolfram
Research have made significant improvements, since 6.0, as version 7.0 computes
that in less than half the time.
drkirkby@hawk:~$ time echo "2+2;" | math
Mathematica 7.0 for Sun Solaris x86 (64-bit)
Copyright 1988-2009 Wolfram Research, Inc.
In[1]:=
In[2]:=
real 0m0.251s
user 0m0.166s
sys 0m0.121s
drkirkby@hawk:~$
For something a bit more taxing:
drkirkby@hawk:~$ time echo "Factorial[10000000];" | math
Mathematica 7.0 for Sun Solaris x86 (64-bit)
Copyright 1988-2009 Wolfram Research, Inc.
In[1]:=
In[2]:=
real 0m20.699s
user 0m20.491s
sys 0m0.207s
Martin Rubey
> Personally I have a bit of a problem understanding why I need to
> worry about a program starting up in less than 2 s, when I might run
> something on it which will take at least one order of magnitude
> longer, and probably several order of magnitudes longer.
I can only say why it matters for me:
martin@rubey-laptop:~/lib/fricas/target/i686-pc-linux/bin$ time echo "2+2" | ./AXIOMsys
Checking for foreign routines
AXIOM=NIL
spad-lib="/lib/libspad.so"
FriCAS (AXIOM fork) Computer Algebra System
Version: FriCAS 2010-01-08
Timestamp: Monday February 22, 2010 at 13:22:25
-----------------------------------------------------------------------------
Issue )copyright to view copyright notices.
Issue )summary for a summary of useful system commands.
Issue )quit to leave FriCAS and return to shell.
-----------------------------------------------------------------------------
(1) -> Warning: HyperTeX macro table not found
(1) 4
Type: PositiveInteger
(2) ->
real 0m0.797s
user 0m0.404s
sys 0m0.076s
martin@rubey-laptop:~/Documents/sage-4.1.1-linux-Ubuntu_9.04-i686-Linux$ time echo "2+2;" | ./sage
----------------------------------------------------------------------
| Sage Version 4.1.1, Release Date: 2009-08-14 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
Loading Sage library. Current Mercurial branch is: combinat
sage: sage:
Exiting SAGE (CPU time 0m0.26s, Wall time 0m0.84s).
real 0m18.403s
user 0m5.144s
sys 0m1.304s
Martin
Dr. David Kirkby
>> Personally I have a bit of a problem understanding why I need to
>> worry about a program starting up in less than 2 s, when I might run
>> something on it which will take at least one order of magnitude
>> longer, and probably several order of magnitudes longer.
>
> I can only say why it matters for me:
<SNIP>
> martin@rubey-laptop:~/Documents/sage-4.1.1-linux-Ubuntu_9.04-i686-Linux$ time echo "2+2;" | ./sage
> ----------------------------------------------------------------------
> | Sage Version 4.1.1, Release Date: 2009-08-14 |
> | Type notebook() for the GUI, and license() for information. |
> ----------------------------------------------------------------------
> Loading Sage library. Current Mercurial branch is: combinat
> sage: sage:
> Exiting SAGE (CPU time 0m0.26s, Wall time 0m0.84s).
>
> real 0m18.403s
> user 0m5.144s
> sys 0m1.304s
>
>
> Martin
OK, 18 s is a bit more significant. 1.5 s I have a bit more of a problem with
worrying about.
I'm going to start another thread on this specific issue of startup time.
Dave
Florent Hivert
On Wed, Mar 03, 2010 at 05:48:28AM -0800, William Stein wrote:
> On Tue, Mar 2, 2010 at 7:56 PM, Dr. David Kirkby
> <david....@onetel.net> wrote:
> >> � � �Right now it takes over 1.5 seconds every time.
> >> wstein@sage:~$ time sage -c "print factor(2010)"
> >> 2 * 3 * 5 * 67
> >> real � �0m1.535s
> >> user � �0m1.140s
> >> sys � � 0m0.460s
> >
> > Personaly I don't find that too excessive for a large tool. How long does
> > Gimp take to start?
>
> That's irrelevant. What matters is how long Maple, Mathematica,
> Matlab, Maxima, Pari, and Magma take to start.
> After repeatedly running the command on sage.math, this is how things stabilize:
I'm not sure your benchmark correctly reflect what happens. Here are three
consecutive launches on a relatively fast idle machine:
tomahawk-~ $ time sage -c "print factor(2010)"
2 * 3 * 5 * 67
sage -c "print factor(2010)" 2,32s user 1,00s system 14% cpu 22,943 total
tomahawk-~ $ time sage -c "print factor(2010)"
2 * 3 * 5 * 67
sage -c "print factor(2010)" 1,34s user 0,34s system 101% cpu 1,648 total
tomahawk-~ $ time sage -c "print factor(2010)"
2 * 3 * 5 * 67
sage -c "print factor(2010)" 1,33s user 0,32s system 97% cpu 1,686 total
The difference is enormous: more than one order of magnitude. IMHO the main
problem is that at startup sage must access a *lot* of files and that access
disk is the bottleneck. Once all those file are in the buffer/cache, the speed
is completely different.
I'm always surprised that they are many thing that I won't ever use that are
imported at the top level of sage. Shouldn't we clean up what is imported and
what is not imported by default and make it easy to import bunch of thing. For
example, Many people certainly don't care about many stuff in
combinatorics. Shouldn't we import by default very basic thing (permutations
and cartesian_product) and let the user who is interested in combinatorics
write the from sage.combinat import * ?
My 2 cents, which has probably already been discussed before...
Cheers,
Florent
William Stein
<florent...@univ-rouen.fr> wrote:
> Hi William,
>
> On Wed, Mar 03, 2010 at 05:48:28AM -0800, William Stein wrote:
>> On Tue, Mar 2, 2010 at 7:56 PM, Dr. David Kirkby
>> <david....@onetel.net> wrote:
>> >> Right now it takes over 1.5 seconds every time.
>> >> wstein@sage:~$ time sage -c "print factor(2010)"
>> >> 2 * 3 * 5 * 67
>> >> real 0m1.535s
>> >> user 0m1.140s
>> >> sys 0m0.460s
>> >
>> > Personaly I don't find that too excessive for a large tool. How long does
>> > Gimp take to start?
>>
>> That's irrelevant. What matters is how long Maple, Mathematica,
>> Matlab, Maxima, Pari, and Magma take to start.
>> After repeatedly running the command on sage.math, this is how things stabilize:
>
> I'm not sure your benchmark correctly reflect what happens.
My benchmark correctly reflects what I wanted to have it reflect. It
would also be of interest to consider issues of disk speed and total
reads, but I would like to very clearly separate that out by making a
precisely stated benchmark, which I did.
> Here are three
> consecutive launches on a relatively fast idle machine:
>
> tomahawk-~ $ time sage -c "print factor(2010)"
> 2 * 3 * 5 * 67
> sage -c "print factor(2010)" 2,32s user 1,00s system 14% cpu 22,943 total
> tomahawk-~ $ time sage -c "print factor(2010)"
> 2 * 3 * 5 * 67
> sage -c "print factor(2010)" 1,34s user 0,34s system 101% cpu 1,648 total
> tomahawk-~ $ time sage -c "print factor(2010)"
> 2 * 3 * 5 * 67
> sage -c "print factor(2010)" 1,33s user 0,32s system 97% cpu 1,686 total
>
> The difference is enormous: more than one order of magnitude. IMHO the main
> problem is that at startup sage must access a *lot* of files and that access
> disk is the bottleneck. Once all those file are in the buffer/cache, the speed
> is completely different.
And, once those files are in the buffer/cache... the speed is still
LAME (compared to any other MA's under identical circumstanced).
Which is my point.
William
Florent Hivert
tomahawk-~ $ time mupkern input
*----* MuPAD Pro 4.5.0 -- The Open Computer Algebra System
/| /|
*----* | Copyright (c) 1997 - 2007 by SciFace Software
| *--|-* All rights reserved.
|/ |/
*----* Licensed to: Combinat devel
2 3 5 67
mupkern input 0,77s user 0,14s system 21% cpu 4,238 total
Next consecutive launches:
mupkern input 0,45s user 0,05s system 97% cpu 0,512 total
mupkern input 0,45s user 0,04s system 95% cpu 0,515 total
mupkern input 0,44s user 0,05s system 96% cpu 0,508 total
mupkern input 0,45s user 0,04s system 92% cpu 0,534 total
mupkern input 0,46s user 0,04s system 95% cpu 0,526 total
Cheers,
Florent
Robert Bradshaw
I often run things that take an order of magnitude less time to run--
e.g. I'm reading a paper and want to try out a quick example to get a
feel for something, or to factor (or even multiply) several digit
numbers. It also makes it prohibitive to be used as a (non-persistent)
web-service backend. I think partially it's a perception thing--one
could argue the same about a web page--why should I worry about it
taking 2 seconds to load/render if I'm going to spend an order of
magnitude more time reading it? Also, on that note, people's very
first exposure to Sage is waiting for it to start up--we don't want
that to be memorable :)
On Mar 3, 2010, at 9:12 AM, Florent Hivert wrote:
> I'm always surprised that they are many thing that I won't ever use
> that are
> imported at the top level of sage. Shouldn't we clean up what is
> imported and
> what is not imported by default and make it easy to import bunch of
> thing.
Once things are in the global namespace, it'll break peoples code to
pull it back out. I don't think we should just dump everything we can
there though.
> For example, Many people certainly don't care about many stuff in
> combinatorics. Shouldn't we import by default very basic thing
> (permutations
> and cartesian_product) and let the user who is interested in
> combinatorics
> write the from sage.combinat import * ?
>
> My 2 cents, which has probably already been discussed before...
I think we can have the names there without importing all the code
behind everything. With tab completion, a huge global namespace isn't
that bad.
- Robert
Jason Grout
>
> I often run things that take an order of magnitude less time to
> a feel for something, or to factor (or even multiply) several digit
> numbers. It also makes it prohibitive to be used as a (non-persistent)
> web-service backend. I think partially it's a perception thing--one
> could argue the same about a web page--why should I worry about it
> taking 2 seconds to load/render if I'm going to spend an order of
> magnitude more time reading it? Also, on that note, people's very first
> exposure to Sage is waiting for it to start up--we don't want that to be
> memorable :)
>
This last point is particularly noticeable when using Sage from the
notebook. A student will start up a Sage worksheet, type "1+1", and
then evaluate. Then the student has to wait the startup Sage time +
network transmission time + sometimes maxima startup time (depending on
the initial computation), and it feels like Sage is taking *way* too
long to answer a simple query.
Jason
John Cremona
Could that be solved by doing that startup as soon as the person logs
in? Or as soon as they open the worksheet (before they do the first
evaluate)?
John
> Jason
Dr. David Kirkby
Yes, I'd agree there. In fact, when I've tried this on my own Solaris machines,
where Sage is less well tested, I've often wondered if the server is not working.
Could a solution be to load all parts of Sage that are currently loaded, but to
load most of them in the background, after someone gets a Sage prompt?
If you could find the 10% of the code most frequently used by Sage users, then
load that before giving a prompt.
Then the other more obscure and less frequently used code silently loads. There
must be loads of Sage code that is used by only a small fraction of Sage users,
so it is less important if that takes longer to load.
Dave
Jason Grout
> On 4 March 2010 09:46, Jason Grout<jason...@creativetrax.com> wrote:
>> On 03/04/2010 02:35 AM, Robert Bradshaw wrote:
>>>
>>> I often run things that take an order of magnitude less time to
>>> run--e.g. I'm reading a paper and want to try out a quick example to get
>>> a feel for something, or to factor (or even multiply) several digit
>>> numbers. It also makes it prohibitive to be used as a (non-persistent)
>>> web-service backend. I think partially it's a perception thing--one
>>> could argue the same about a web page--why should I worry about it
>>> taking 2 seconds to load/render if I'm going to spend an order of
>>> magnitude more time reading it? Also, on that note, people's very first
>>> exposure to Sage is waiting for it to start up--we don't want that to be
>>> memorable :)
>>>
>>
>> This last point is particularly noticeable when using Sage from the
>> notebook. A student will start up a Sage worksheet, type "1+1", and then
>> evaluate. Then the student has to wait the startup Sage time + network
>> transmission time + sometimes maxima startup time (depending on the initial
>> computation), and it feels like Sage is taking *way* too long to answer a
>> simple query.
>>
>
> Could that be solved by doing that startup as soon as the person logs
> in? Or as soon as they open the worksheet (before they do the first
> evaluate)?
Yes. If the worksheet is a new worksheet, this would make a lot of
sense, so +1 in that case. However, for a worksheet that is reopened,
that places a lot of demand on the server if you are just opening the
sheet to see what it is.
Thanks,
Jason
Pat LeSmithe
> Could that be solved by doing that startup as soon as the person logs
> in? Or as soon as they open the worksheet (before they do the first
> evaluate)?
We already do the latter (though not for doc worksheets). From
sagenb.notebook.twist, around line 1515:
class Worksheet(WorksheetResource, resource.Resource):
addSlash = True
def render(self, ctx):
self.worksheet.sage()
s = notebook.html(worksheet_filename = self.name,
username=self.username)
return HTMLResponse(stream=s)
Worksheet.sage starts the worksheet process, if it's not already started.
Of course, it's possible to attempt *several* cell operations in the
time it might take for startup and other delays to pass.
Is memory use a problem, particularly on busy servers?
Jason Grout
> Is memory use a problem, particularly on busy servers?
It definitely could be an issue on my campus server. I have 3GB in a
virtual machine right now (I'm writing an internal school grant for more
memory soon). Fortunately (?!), I haven't been using Sage in my classes
as much as I intended, so usage is light right now.
This might be a place for a setting in the admin page, with it
defaulting to "start up Sage automatically". That would let people with
limited resources turn that off.
Thanks,
Jason
Simon King
On Mar 4, 8:35 am, Robert Bradshaw <rober...@math.washington.edu>
wrote:
[...]
> I think we can have the names there without importing all the code
> behind everything. With tab completion, a huge global namespace isn't
> that bad.
How would this be possible, technically? I mean, is there a technical
solution that does not require python to be patched?
Perhaps modifying the sage preparser:
- At startup, virtually nothing of sage.all is imported, but the
names of things in sage.all are known, and it is known where to import
them from. Let SageAllNames be a dictionary that associates the name
with the location for import.
- The preparser could detect whether a given command line contains an
identifier N outside an import or assign statement, with N in
SageAllNames.keys(). If not globals().has_key(N), the preparser would
prepend "from %s import %s"%(SageAllNames[N],N) to the given command
line.
- It would be easy to modify tab completion so that
SageAllNames.keys() is accessible to tab completion.
Would that be feasible?
I think it could reduce the startup time considerably. A user wouldn't
notice any change during an interactive session. And since it only
concerns the preparser, it would not break any code.
Cheers,
Simon
Paulo César Pereira de Andrade
After reading the continuation of the thread, the first thing that come
to my mind was unexec.
A quick google search showed a few matches, and an attempt from
2003, that apparently had a few regressions (in signal and thread tests),
and also, in the message I found, the fact that the unexec code was
gpl was also considered a problem.
I know only as from overview what unexec does; used by emacs
and xemacs; basically, a way to dump a running program, and reload
it at a later stage, much faster.
Does anybody know if there are any new approachs in it (I only found
some emails from 2003 in a google search...) ?
I believe it may require significant work if starting from emacs/xemacs
unexelf.c to work correctly with shared modules/libraries, etc. But, it
could be a way to to drastically reduce sage load time, but possibly,
by generating a very huge image file.
Paulo
Robert Bradshaw
> Hi!
>
> On Mar 4, 8:35 am, Robert Bradshaw <rober...@math.washington.edu>
> wrote:
> [...]
>> I think we can have the names there without importing all the code
>> behind everything. With tab completion, a huge global namespace isn't
>> that bad.
>
> How would this be possible, technically? I mean, is there a technical
> solution that does not require python to be patched?
See, for example, lazy import at http://trac.sagemath.org/sage_trac/ticket/7502
- Robert
Simon King
On 4 Mrz., 19:21, Robert Bradshaw <rober...@math.washington.edu>
wrote:
[...]
> See, for example, lazy import athttp://trac.sagemath.org/sage_trac/ticket/7502
Thank you very much, that was almost what I was hoping for.
What I don't like in that solution:
If you lazily import, say, QQ, then QQ will forever be a LazyImport
object -- there will only be an attribute QQ._object eventually
getting assigned to the rational field. Hence, whenever the user wants
to do something with QQ (and this may be very often) then there would
be the attribute lookup. This may be a small overhead, but wouldn't it
be noticable, on the long run?
It seems frigthening to me to have every single thing from sage.all
wrapped up into a LazyImport object
I would prefer that a LazyImport object has an argument G that
provides the name space into which the import shall eventually take
place. In the beginning, the LazyImport object would be written into G
under a certain name. As soon as anything is done with the object, it
does a *real* import, replaces itself in G with the real thing, and
since the reference from G is gone, the LazyImport object would
eventually be garbage collected.
At least to me, this seems slightly nicer.
Example session:
# lazily import QQ under the name MyQQ
sage: LazyImport(globals(),'QQ','sage.rings.rational_field','MyQQ')
lazily imported QQ
# MyQQ is known in the global name space, but it is not properly
imported yet:
sage: MyQQ
lazily imported QQ
# Now do anything with MyQQ: Introspection, or calling:
sage: MyQQ(3)
3
# In the preceding line, MyQQ imported QQ and replaces itself with it
in globals().
# So, by now, there is absolutely no overhead in calling MyQQ:
sage: MyQQ
Rational Field
What do you think about it?
Cheers,
Simon
Robert Bradshaw
> Hi Robert!
> On 4 Mrz., 19:21, Robert Bradshaw <rober...@math.washington.edu>
> wrote:
> [...]
>> See, for example, lazy import athttp://trac.sagemath.org/sage_trac/ticket/7502
>
> Thank you very much, that was almost what I was hoping for.
>
> What I don't like in that solution:
> If you lazily import, say, QQ, then QQ will forever be a LazyImport
> object -- there will only be an attribute QQ._object eventually
> getting assigned to the rational field. Hence, whenever the user wants
> to do something with QQ (and this may be very often) then there would
> be the attribute lookup. This may be a small overhead, but wouldn't it
> be noticable, on the long run?
> It seems frigthening to me to have every single thing from sage.all
> wrapped up into a LazyImport object
I hope the overhead is small, but it will certainly not be zero. I'm
not imagining that everything be wrapped, only less-commonly used,
expensive stuff. QQ would be imported by default for sure. For
example, EllipticCurve could be wrapped. The schemes directory takes .
3 seconds to load--I use it all the time, but that won't be a big hit
the first time I use it, and the overhead of the wrapper for
constructing an elliptic curve will be (relatively) negligible.
> I would prefer that a LazyImport object has an argument G that
> provides the name space into which the import shall eventually take
> place. In the beginning, the LazyImport object would be written into G
> under a certain name.
This is how it works now. You don't even have to provide globals(), as
it deduces it from the call stack. (This won't work from Cython, if
anyone knows how to do that I'd be glad to know ;) Probably should be
an optional argument.)
> As soon as anything is done with the object, it
> does a *real* import, replaces itself in G with the real thing, and
> since the reference from G is gone, the LazyImport object would
> eventually be garbage collected.
I've actually intended to do this as well, it'd be easy, but I just
haven't had time to do it. Note, however, that this is not transitive,
so the lazy objects may still be around. (This is less of an issue for
the global namespace, but if someone does "from sage.all import foo"
they'll have the lazy version forever.) There's some other
improvements that I'd like to make too. You willing to referee a patch
in this direction? :)
- Robert
Simon King
On Mar 5, 12:42 am, Robert Bradshaw <rober...@math.washington.edu>
wrote:
[...]
> > As soon as anything is done with the object, it
> > does a *real* import, replaces itself in G with the real thing, and
> > since the reference from G is gone, the LazyImport object would
> > eventually be garbage collected.
>
> I've actually intended to do this as well, it'd be easy, but I just
> haven't had time to do it. Note, however, that this is not transitive,
> so the lazy objects may still be around.
You mean, if you have
sage: imp =
LazyImport(globals(),'QQ','sage.rings.rational_field','MyQQ')
then "imp" would remain lazy even if at some point it injects QQ into
globals()? Sure, this would be difficult to circumvent, but that way
of usage is certainly discouraged.
> (This is less of an issue for
> the global namespace, but if someone does "from sage.all import foo"
> they'll have the lazy version forever.)
Why? If you start with a lazy import of "foo" then foo will be a
LazyImport object, to start with. If you then do "from sage.all import
foo" then the reference "foo" to the LazyImport object is replaced by
a reference to what was just imported; as there is no pointer to the
LazyImport object, garbage collection will work.
> There's some other
> improvements that I'd like to make too. You willing to referee a patch
> in this direction? :)
Sure! Where is the ticket?
Best regards,
Simon
Robert Bradshaw
What I was referring to is the lack of transitivity. If in sage.all
you have
lazy_import("sage.rings.all", "foo")
than anything that does "from sage.all import foo" before foo is
looked up will have the lazy version, even if sage.all's reference is
updated.
>> There's some other
>> improvements that I'd like to make too. You willing to referee a
>> patch
>> in this direction? :)
>
> Sure! Where is the ticket?
I'll get a patch up later today (I think all that's missing is some
doctests, but I'll have to see).
- Robert
William Stein
Goals for Sage-5.0:
* 90% doctest coverage score (=write about 1500 doctests)
* Official Solaris 10 support (all tests pass)
* Official Cygwin support (all tests pass)
* Close _all_ tickets listed at
http://trac.sagemath.org/sage_trac/wiki/stab1
TARGET DATE: Sometime in May.
-- William
Dr. David Kirkby
> Hi,
>
> Goals for Sage-5.0:
>
> * 90% doctest coverage score (=write about 1500 doctests)
Hopefully with some justification of why the expected result is what it is. Not
magic numbers - see
http://groups.google.co.uk/group/sage-devel/browse_thread/thread/90d933ea2881cbf8
> * Official Solaris 10 support (all tests pass)
I personally can't understand why official support for a completely new
operating system is not sufficient justification for incrementing the major
version.
Micheal was paid full-time to work on the Solaris port. With his salary, the
hardware costs (admittedly some donated by Sun), overheads etc, this port must
have cost close to $100,000. I would have thought that time to crack open a
bottle of champagne and increment the major release number!
> * Official Cygwin support (all tests pass)
Again, I would see the addition of that alone as being sufficient to increment
the major release number.
> * Close _all_ tickets listed at
> http://trac.sagemath.org/sage_trac/wiki/stab1
>
> TARGET DATE: Sometime in May.
I can't speak for the other goals, but I believe the Solaris one is probably
doable, though I have got some failures in 4.4.4.alpha0 which are not as easily
fixed as those in 4.3.3.
The problem with Solaris 10 support is that people can bring out incompatible
changes faster than I can fix them.
I have a set of changes sufficient to make 4.3.3 pass all tests.
http://trac.sagemath.org/sage_trac/ticket/8409
Now on 4.3.4.alpha0, the following fail (this includes the long doctests).
sage -t -long
"local/lib/python2.6/site-packages/sagenb-0.7.5.1-py2.6.egg/sagenb/misc/sageinspect.py"
sage -t -long
"local/lib/python2.6/site-packages/sagenb-0.7.5.1-py2.6.egg/sagenb/notebook/interact.py"
sage -t -long "devel/sage/sage/categories/finite_semigroups.py"
sage -t -long "devel/sage/sage/categories/examples/finite_semigroups.py"
sage -t -long "devel/sage/sage/combinat/posets/posets.py"
sage -t -long "devel/sage/sage/plot/colors.py"
sage -t -long "devel/sage/sage/homology/delta_complex.py"
sage -t -long "devel/sage/sage/homology/cubical_complex.py"
sage -t -long "devel/sage/sage/homology/examples.py"
sage -t -long "devel/sage/sage/homology/cell_complex.py"
sage -t -long "devel/sage/sage/homology/chain_complex.py"
sage -t -long "devel/sage/sage/homology/simplicial_complex.py"
Total time for all tests: 42573.4 seconds
Dave
Nick Alexander
On 5-Mar-10, at 5:26 PM, Dr. David Kirkby wrote:
> William Stein wrote:
>> Hi,
>> Goals for Sage-5.0:
>> * 90% doctest coverage score (=write about 1500 doctests)
>
> Hopefully with some justification of why the expected result is what
> it is. Not magic numbers - see
>
> http://groups.google.co.uk/group/sage-devel/browse_thread/thread/90d933ea2881cbf8
>
>> * Official Solaris 10 support (all tests pass)
>
> I personally can't understand why official support for a completely
> new operating system is not sufficient justification for
> incrementing the major version.
Isn't the Sage-5.0 incrementing the major version number?
Nick
William Stein
David is trying to argue that the goals for Sage-5.0 should be
* Official Solaris 10 support (all tests pass)
TARGET DATE: Sometime in March?
*instead* of the following:
* 90% doctest coverage score (=write about 1500 doctests)
* Official Solaris 10 support (all tests pass)
* Official Cygwin support (all tests pass)
* Close _all_ tickets listed at
http://trac.sagemath.org/sage_trac/wiki/stab1
TARGET DATE: Sometime in May.
-- William
Nick Alexander
>
> * Official Solaris 10 support (all tests pass)
>
> TARGET DATE: Sometime in March?
>
> *instead* of the following:
>
> * 90% doctest coverage score (=write about 1500 doctests)
> * Official Solaris 10 support (all tests pass)
> * Official Cygwin support (all tests pass)
> * Close _all_ tickets listed at
> http://trac.sagemath.org/sage_trac/wiki/stab1
>
> TARGET DATE: Sometime in May.
I vote -1 to this. As a user, I expect a major release (defined as a
version number bump) to include exciting new toys. Solaris support
does not *in my opinion* qualify as an exciting new toy. I anticipate
a Slashdot story announcing the new release, followed by a surge of
interest and downloads. It is *my opinion* that Solaris support is
not an exciting new feature that the resulting publicity should be
promoting. If anything, it is *my opinion* that Cygwin support is
much more likely to appeal to these potential new users, and would
warrant the publicity.
Nick
PS. Emphasis added because I do not want to fan any flames -- please
respond accordingly.
Dr. David Kirkby
Part of my logic for this is that given Sun have
* Donated hardware (t2) worth around $30,000 - $40,000 for this,
* Have supplied other hardware heavily discounted. (sage.math and most of the
disks are I believe Suns)
* Are asking William for a release where they can point customers at,
then those that supplied a lot of money probably do consider it quite important.
You personally might not, but a major investor does.
If Sun (now Oracle) did not consider it important, I doubt they would have
supplied the hardware, and I doubt they would be asking William questions about
the Solaris port.
The only person paid full time to work on Sage was paid to do the Solaris port.
A lot of time, effort and money has gone into it.
I agree with you about Cygwin too.
I also think reaching a specific level of doc tests is a bit irrelevant in
determining when to increment the major release number. Perhaps if the doc tests
reached 100%, then I might agree.
I think with the very fast release cycle of Sage, no one release is likely to
have many exciting new toys.
Dave
Robert Bradshaw
Robert Bradshaw
Don't get me wrong, some people will be very excited about the Solaris
port. However, I bet at least 90% of users couldn't care less. If
release numbers have anything to do with marketing, making this the
only goal is not a good idea.
> The only person paid full time to work on Sage was paid to do the
> Solaris port.
That is a gross simplification of the history here.
> A lot of time, effort and money has gone into it.
The same could be said about many, many components of Sage.
> I agree with you about Cygwin too.
>
> I also think reaching a specific level of doc tests is a bit
> irrelevant in determining when to increment the major release
> number. Perhaps if the doc tests reached 100%, then I might agree.
These are all goals for 5.0, not necessarily what's going to be new in
5.0. I agree that 90% doctest coverage is not something that *makes* a
release, but it's a good thing to shoot for by a certain point in
time. And if (when) any of the above goals are accomplished early (and
I expect some of them to be so, including Solaris) we're not going to
be holding them out of the 4.x series.
> I think with the very fast release cycle of Sage, no one release is
> likely to have many exciting new toys.
Isn't it great--we get new, shiny toys all year round :)
- Robert
William Stein
Thanks everybody for all the discussion of sage-5.0 goals. I've made
a new sage-5.0 milestone
http://trac.sagemath.org/sage_trac/milestone/sage-5.0
and I've made a list of our goals. I set the release goal date at
June 1, 2010, which gives us a full 3 months to meet the given goals.
--William
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Dr. David Kirkby
> Hi,
>
> Thanks everybody for all the discussion of sage-5.0 goals. I've made
> a new sage-5.0 milestone
>
> http://trac.sagemath.org/sage_trac/milestone/sage-5.0
>
> and I've made a list of our goals. I set the release goal date at
> June 1, 2010, which gives us a full 3 months to meet the given goals.
>
> --William
One item on that list is:
"Official 32-bit Solaris 10 support on sparc and x86 (automatic build, all tests
pass)"
I think Solaris 10 on x86 is being a bit optimistic. SPARC yes, but there may be
other issues on x86.
If there are issues on Solaris 10 x86, then I don't know who is going to work on
them, but probably not me too much. I'd rather work on the OpenSolaris 64-bit on
x86 than Solaris 10 32-bit.
With some changes to the 4.3.3 source, I was able to get 100% of the tests pass,
but with 4.3.4.alpha0 things have gone very much backwards.
There's a full log of the long doctests in the directory.
http://boxen.math.washington.edu/home/kirkby/doctest-results/2/
There's so many issues, I've not even created tickets for them all.
Dave
William Stein
<david....@onetel.net> wrote:
> William Stein wrote:
>>
>> Hi,
>>
>> Thanks everybody for all the discussion of sage-5.0 goals. I've made
>> a new sage-5.0 milestone
>>
>> http://trac.sagemath.org/sage_trac/milestone/sage-5.0
>>
>> and I've made a list of our goals. I set the release goal date at
>> June 1, 2010, which gives us a full 3 months to meet the given goals.
>>
>> --William
>
> One item on that list is:
>
>
> "Official 32-bit Solaris 10 support on sparc and x86 (automatic build, all
> tests pass)"
>
> I think Solaris 10 on x86 is being a bit optimistic. SPARC yes, but there
> may be other issues on x86.
>
> If there are issues on Solaris 10 x86, then I don't know who is going to
> work on them, but probably not me too much. I'd rather work on the
> OpenSolaris 64-bit on x86 than Solaris 10 32-bit.
OK, I've removed x86 Solaris support as a goal.
> With some changes to the 4.3.3 source, I was able to get 100% of the tests
> pass, but with 4.3.4.alpha0 things have gone very much backwards.
>
> There's a full log of the long doctests in the directory.
>
> http://boxen.math.washington.edu/home/kirkby/doctest-results/2/
>
> There's so many issues, I've not even created tickets for them all.
It doesn't look soooo bad to me.
-- William
Robert Bradshaw
For the first alpha, I would actually say that looks pretty good:
----------------------------------------------------------------------
The following tests failed:
sage -t -long "local/lib/python2.6/site-packages/sagenb-0.7.5.1-
py2.6.egg/sagenb/misc/sageinspect.py"
sage -t -long "local/lib/python2.6/site-packages/sagenb-0.7.5.1-
py2.6.egg/sagenb/notebook/interact.py"
sage -t -long "devel/sage/sage/categories/finite_semigroups.py"
sage -t -long "devel/sage/sage/categories/examples/
finite_semigroups.py"
sage -t -long "devel/sage/sage/combinat/posets/posets.py"
sage -t -long "devel/sage/sage/plot/colors.py"
sage -t -long "devel/sage/sage/homology/delta_complex.py"
sage -t -long "devel/sage/sage/homology/cubical_complex.py"
sage -t -long "devel/sage/sage/homology/examples.py"
sage -t -long "devel/sage/sage/homology/cell_complex.py"
sage -t -long "devel/sage/sage/homology/chain_complex.py"
sage -t -long "devel/sage/sage/homology/simplicial_complex.py"
Total time for all tests: 42573.4 seconds
A 30-second skim through the list gives me the impression that there
are probably 3 or 4 issues total that are causing all of these
failures. Of course I could be wrong, and who knows how hard it will
be to fix those homology ones (= chomp didn't build correctly?). This
one jumped out at me though:
File "/export/home/drkirkby/32/sage-4.3.4.alpha0/devel/sage/sage/
categories/finite_semigroups.py", line 232:
sage: sorted(S.j_transversal_of_idempotents())
Expected:
['a', 'ab', 'ac', 'acb', 'b', 'bc', 'c']
Got:
['a', 'ab', 'ac', 'acb', 'b', 'c', 'cb']
How could it get that wrong?
The only long term solution I see is a continuous build bot that runs
on a Solaris box (among others) and bounces tickets that fail. I plan
on putting one together if no one beats me to it, but not until this
summer.
- Robert
John H Palmieri
wrote:
> On Mar 6, 2010, at 4:06 PM, Dr. David Kirkby wrote:
[snip]
> A 30-second skim through the list gives me the impression that there
> are probably 3 or 4 issues total that are causing all of these
> failures. Of course I could be wrong, and who knows how hard it will
> be to fix those homology ones (= chomp didn't build correctly?).
That one is easy -- it's actually related to what I mentioned in the
Sage seminar on Friday: on Solaris, "which program" apparently exits
with a return code of 0 even if "program" is not found. So the
problem is that CHomP is not installed on t2 but because of this
"which" issue, Sage thinks it is.
I have a patch; once I run doctests, I'll post it at #8463.
--
John
Dr. David Kirkby
I don't think it is that John. I am really tied up today, and don't have chance
to check this over in detail. But in summary.
* 'which' is not part of POSIX so I believe it is not a good solution.
* 'which' exists on Solaris. It is documented to return an exit code of 0 if
the command is found, and >0 of the command is not found. I've checked it, and
found it behaves as expected.
* The output from 'which' is quite different from the output of the 'type'
command your ticket uses.
* IMHO, the best way to approach this is to use 'command -v' as that is
* Documented in POSIX to work the same way as 'which'
* Works on every system I've tried it on - Linux, HP-UX, Solaris, OSX and
FreeBSD
It really can't understand why 'which' is not workking, despite I think its use
should be discouraged.
I've put a lot more information on the ticket.
But I don't have time to do much on this today. My wife has gone away for a week
so I can get on with some decorating. I want to use daylight hours for that, and
leave Sage until it is too dark to do other more pressing tasks.
Dave
Florent Hivert
> A 30-second skim through the list gives me the impression that there are
> probably 3 or 4 issues total that are causing all of these failures. Of
> course I could be wrong, and who knows how hard it will be to fix those
> homology ones (= chomp didn't build correctly?). This one jumped out at me
> though:
I don't understand why it jumped to you.
> File
> "/export/home/drkirkby/32/sage-4.3.4.alpha0/devel/sage/sage/categories/finite_semigroups.py",
> line 232:
> sage: sorted(S.j_transversal_of_idempotents())
> Expected:
> ['a', 'ab', 'ac', 'acb', 'b', 'bc', 'c']
> Got:
> ['a', 'ab', 'ac', 'acb', 'b', 'c', 'cb']
>
> How could it get that wrong?
What to you mean by that ? As indicated by the name, the function compute a
transversal of an equivalence relation (the j-relation) so the choice is not
canonical at all. It actually depends on the implementation of Cayley
graphs. Please see #8445 which should fix this issue.
> The only long term solution I see is a continuous build bot that runs on a
> Solaris box (among others) and bounces tickets that fail. I plan on putting
> one together if no one beats me to it, but not until this summer.
+10 to that but not only for Solaris. I mean, neither authors of patches nor
reviewers have the time/will/possibility to check on all the possible
architectures. This should be automated. My dream solution (and I can tell you
I'm not the only one dreaming about that) is the following:
When you put a ticket on trac, you must clearly indicate in a field (not in
the text) which patches should be applied in which order. A sensible default
may be helpful here. You must also indicate on which other tickets this patch
depend. Then in a fully automated manner a few time after, either:
- a green light turn on indicating that all tests passed on all the architecture.
- a red light turn on with a link to the log of the failure.
Note that I've no idea how hard it is to implement in trac, neither if we
have the necessary hardware to support this load.
Cheers,
Florent
Robert Bradshaw
I was reading that last line as [..., 'c', 'bc']. Makes more sense now.
>> The only long term solution I see is a continuous build bot that
>> runs on a
>> Solaris box (among others) and bounces tickets that fail. I plan on
>> putting
>> one together if no one beats me to it, but not until this summer.
>
> +10 to that but not only for Solaris.
Yep, that's my intent.
> I mean, neither authors of patches nor
> reviewers have the time/will/possibility to check on all the possible
> architectures. This should be automated.
Yep. Also, it shifts the review focus away from making sure the patch
applies and passes tests to actually refereeing the code,
documentation, and examples themselves.
> My dream solution (and I can tell you
> I'm not the only one dreaming about that) is the following:
>
> When you put a ticket on trac, you must clearly indicate in a field
> (not in
> the text) which patches should be applied in which order. A sensible
> default
> may be helpful here. You must also indicate on which other tickets
> this patch
> depend. Then in a fully automated manner a few time after, either:
>
> - a green light turn on indicating that all tests passed on all the
> architecture.
> - a red light turn on with a link to the log of the failure.
>
> Note that I've no idea how hard it is to implement in trac,
I think it's totally doable, though it might be worth switching to
something like Reitveld. As part of my dream solution a patched copy
of Sage would be left lying around for any potential referees to try
out (and in such a way that they could easily add their own examples
to the doctests).
> neither if we have the necessary hardware to support this load.
Collectively we probably do--or at least the architectures that are in
common use will be more completely tested.
- Robert
Florent Hivert
> > Note that I've no idea how hard it is to implement in trac, neither if we
> > have the necessary hardware to support this load.
>
> >From reading the Sage merge script, I think one could use that script
> or write something along similar lines to implement a (simple)
> proof-of-concept continuous buildbot. That is, without adding any new
> fields to trac. I just want to point out that the current number of
> fields on a trac ticket can overwhelm a beginner, indeed anyone. More
> fields added would mean more time spent thinking about what
> information to add to which field. In many cases, one could easily
> write a lot of information clearly and concisely as a ticket comment.
> If the information is critical for reviewing a ticket, such
> information could be written in the ticket description. If you have
> been following how David Kirkby writes his descriptions for Solaris
> and portability tickets, you would see what I mean.
I surely agree with all of that. However, I'm not sure it would be easy to
write a sufficiently clever buildbot that is able to automatically find the
necessary information from the ticket description. Hence my suggestion to add
more field. Another maybe simpler solution is to be able to launch the
buildbot by hands say from a trac webpage, after giving him the lists of
patches. This should not be a time consuming task for the author/reviewer.
As Robert very nicely pointed out, the ultimate goal is that
"[...], the review focus away from making sure the patch
"applies and passes tests to actually refereeing the code, documentation,
"and examples themselves."
Cheers,
Florent
Robert Bradshaw
Single ticket patches (the majority of them) are easy to handle. If
there is more than one ticket, it can try applying them all, and that
failing, applying only the last one, noting that it needs more
information on failure. As for specifying more complicated orders, we
were talking about this during the last Sage days and the best idea
was that it would look for the last bulleted list that consisted of
patch file names. Thus the author would write something like
Apply the patches in this order:
* integrate-fix.patch
* referee-comments.patch
* doctest-fix.patch
And both the automated system and build bot could figure this out.
- Robert
William Stein
And humans can easily read this too. This could just be the last
comment on the ticket (of that form).
William
Jason Grout
I'd rather have a field, with a default (if the field is empty) to try
applying all patches.
Otherwise, I have to remember that I need to make a list of the patches
to apply as a bulleted list, and make sure that list is the last such
list of filenames, etc. I think it's better to be explicit in a field,
rather than rely on parsing comments intended primarily for humans.
Jason