doctest coverage to 75%
William Stein
In order to get Sage's doctest coverage to 75%, which is one of the
main goals for Sage-4.0, it is necessary to doctest about 1900
functions in the Sage library. I just did a test (see
http://trac.sagemath.org/sage_trac/ticket/5724 and
http://trac.sagemath.org/sage_trac/ticket/5725) and it took me on
average about 2-3 minutes per doctest, including fixing a couple of
bugs I found. Estimating 3 minutes each, that is about 95 hours of
nonstop work.
It is very easy to find files that are full of code that isn't
doctested and *must* be doctested. It was trivial for me to find the
sequences and quotient_ring_element had very bad coverage. In both
cases, I found *several* small bugs in both files. Probably, any
random file with low coverage has several bugs in it. That's why
getting the doctest coverage in Sage up is critical to Sage becoming a
truly high quality system.
I hope those of you who feel qualified to write doctests, will help
out. It's nearly impossible for one person to do all 1900 of those
doctests in the next month. Writing doctests is not easy and it
absolutely requires experience experience with the Sage library, and
good understanding of the code you are testing.
Thanks,
William
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
John Cremona
included in the Reference manual? Although that manual is long, there
is a whole lot of stuff not mentioned in it, which is bad PR.
What I mean in practice is, if someone is going through a source file
whose doctests are incomplete and also old-style, then they convert
each to new-style as they go, as well as (at the same time as) adding
the new doctests.
John
2009/4/9 William Stein <wst...@gmail.com>:
Robert Bradshaw
> Can we at the same time convert more files to ReST and hence get them
> included in the Reference manual? Although that manual is long, there
> is a whole lot of stuff not mentioned in it, which is bad PR.
>
> What I mean in practice is, if someone is going through a source file
> whose doctests are incomplete and also old-style, then they convert
> each to new-style as they go, as well as (at the same time as) adding
> the new doctests.
That seems like a good idea to me.
> 2009/4/9 William Stein <wst...@gmail.com>:
>>
>> Hi,
>>
>> In order to get Sage's doctest coverage to 75%, which is one of the
>> main goals for Sage-4.0, it is necessary to doctest about 1900
>> functions in the Sage library. I just did a test (see
>> http://trac.sagemath.org/sage_trac/ticket/5724 and
>> http://trac.sagemath.org/sage_trac/ticket/5725) and it took me on
>> average about 2-3 minutes per doctest, including fixing a couple of
>> bugs I found. Estimating 3 minutes each, that is about 95 hours of
>> nonstop work.
>>
>> It is very easy to find files that are full of code that isn't
>> doctested and *must* be doctested. It was trivial for me to find the
>> sequences and quotient_ring_element had very bad coverage. In both
>> cases, I found *several* small bugs in both files. Probably, any
>> random file with low coverage has several bugs in it. That's why
>> getting the doctest coverage in Sage up is critical to Sage
>> becoming a
>> truly high quality system.
>>
>> I hope those of you who feel qualified to write doctests, will help
>> out. It's nearly impossible for one person to do all 1900 of those
>> doctests in the next month. Writing doctests is not easy and it
>> absolutely requires experience experience with the Sage library, and
>> good understanding of the code you are testing.
+1. I often end up writing a lot of doctests for surrounding code
when converting things to the new coercion model.
How about another Sage doc day in the coming month? I bet with enough
of us we could at least make a sizeable dent in the 1900 tests needed
in one big shot.
- Robert
David Roe
That seems like a good idea to me.
On Apr 9, 2009, at 1:49 AM, John Cremona wrote:
> Can we at the same time convert more files to ReST and hence get them
> included in the Reference manual? Although that manual is long, there
> is a whole lot of stuff not mentioned in it, which is bad PR.
>
> What I mean in practice is, if someone is going through a source file
> whose doctests are incomplete and also old-style, then they convert
> each to new-style as they go, as well as (at the same time as) adding
> the new doctests.
I've seen the guide to ReST, but is there a document somewhere summarizing what usually needs to be done to Sage functions? Is changing EXAMPLES: to EXAMPLES:: and adding a newline after it usually enough?
Sounds like a good idea to me. I'll try to show up and work on p-adics. :-)
How about this Saturday?
David
Jason Grout
> On Thu, Apr 9, 2009 at 5:19 AM, Robert Bradshaw <
> robe...@math.washington.edu> wrote:
>
>> On Apr 9, 2009, at 1:49 AM, John Cremona wrote:
>>
>>> Can we at the same time convert more files to ReST and hence get them
>>> included in the Reference manual? Although that manual is long, there
>>> is a whole lot of stuff not mentioned in it, which is bad PR.
>>>
>>> What I mean in practice is, if someone is going through a source file
>>> whose doctests are incomplete and also old-style, then they convert
>>> each to new-style as they go, as well as (at the same time as) adding
>>> the new doctests.
>> That seems like a good idea to me.
>
>
> I've seen the guide to ReST, but is there a document somewhere summarizing
> what usually needs to be done to Sage functions? Is changing EXAMPLES: to
> EXAMPLES:: and adding a newline after it usually enough?
I thought Mike Hansen had some tools to (largely) automate the
conversion. Were those merged into Sage, or are they available somewhere?
Jason
davidloeffler
On Apr 9, 9:45 am, William Stein <wst...@gmail.com> wrote:
>
> I hope those of you who feel qualified to write doctests, will help
> out. It's nearly impossible for one person to do all 1900 of those
> doctests in the next month. Writing doctests is not easy and it
> absolutely requires experience experience with the Sage library, and
> good understanding of the code you are testing.
>
> Thanks,
>
coverage is 74.9% as of 3.4.1.rc1. OK, so this is perhaps one of the
best-doctested parts already, but it's also one of the few I know much
about. I've posted a patch at http://trac.sagemath.org/sage_trac/ticket/5727
which adds 28 new doctests (and fixes 2 small bugs) and adds another
module to the reference manual. More to follow (maybe).
David
William Stein
Great. I've refereed this (positively). You fixed *numerous* bugs in
the code, imho, when writing those 28 new doctests -- great work!
Let's keep the doctest patches coming.
I'm mostly available on Saturday for a doctest day... so mark your calendars:
SATURDAY, APRIL 11 -- Doctest Day.
-- William
mabshoff
On Apr 9, 5:53 pm, William Stein <wst...@gmail.com> wrote:
> On Thu, Apr 9, 2009 at 11:13 AM, davidloeffler <dave.loeff...@gmail.com> wrote:
<SNIP>
> Great. I've refereed this (positively). You fixed *numerous* bugs in
> the code, imho, when writing those 28 new doctests -- great work!
> Let's keep the doctest patches coming.
202 sage: cmp(G, 1)
203 -1
202 sage: cmp(G, 1) in [1,-1]
203 True
cmp($FOO,$BAR) returns -1,0 or 1 depending on the memory address, so
the version that William did is much cleaner since G != 1 in this
case. A lot of those cmp() tests were set to random, but it is bad
style to do that.
> I'm mostly available on Saturday for a doctest day... so mark your calendars:
>
> SATURDAY, APRIL 11 -- Doctest Day.
send out an official announcement.
> -- William
Cheers,
Michael
William Stein
Thanks! (It sounds to me like you just volunteered yourself.)
William
William Stein
WHERE DO WE STAND?
Here's the Sage doctest coverage status:
We need 296 more function to get to 68% coverage.
We need 756 more function to get to 70% coverage.
We need 1905 more function to get to 75% coverage.
We will have a docday this Saturday, with the insanely ambitious goal
of getting to 70%. This is probably not humanely possible for a
single person to do in 8 hours, so I hope I'm not the only one (I
estimate it takes *at least* 3 minutes per doctest, which comes to
37.8 hours for 756 doctests).
WHAT SHOULD WE DOCTEST?
I'm going to list some ideas for code to "hit", which have bad
coverage and for which there will be a big payoff.
schemes/generic/morphism.py: 2% (1 of 40)
modular/modsym/manin_symbols.py: 2% (2 of 68)
interfaces/kash.py: 3% (1 of 30)
probability/random_variable.py: 3% (1 of 30)
rings/extended_integer_ring.py: 4% (6 of 128)
rings/pari_ring.py: 4% (1 of 22) # could this
be deleted???
structure/formal_sum.py: 4% (1 of 22)
interfaces/cocoa.py: 4% (1 of 23) # delete this,
it doesn't work yet
interfaces/reduce.py: 4% (1 of 23) # delete; it doesn't work
plot/plot3d/base.pyx: 5% (4 of 78)
plot/tachyon.py: 8% (5 of 58)
rings/morphism.pyx: 8% (3 of 35)
structure/generators.pyx: 11% (5 of 45)
categories/pushout.py: 12% (9 of 73)
rings/laurent_series_ring.py: 13% (3 of 23)
plot/plot3d/shapes.pyx: 16% (5 of 31)
gsl/probability_distribution.pyx: 20% (4 of 20) # has a nodoctest at
the top of file. very scary!
schemes/elliptic_curves/monsky_washnitzer.py: 23% (25 of 107) #
doctests here find bad things often, e.g., a bug in FLINT!
modular/hecke/ambient_module.py: 26% (9 of 34)
modular/modsym/ambient.py: 26% (26 of 97)
The above would be 874 doctests, which would solidly get us over 70%.
I've avoided code above that is being refactored, or rumored to be
refactored, or rewritten or whatever. This is all
stuff that just needs to get coveraged.
Alex Ghitza
> We will have a docday this Saturday, with the insanely ambitious goal
> of getting to 70%. This is probably not humanely possible for a
> single person to do in 8 hours, so I hope I'm not the only one (I
> estimate it takes *at least* 3 minutes per doctest, which comes to
> 37.8 hours for 756 doctests).
This Saturday translates into this Sunday down under, which won't work
for me. But I'll get some stuff done before then.
>
> schemes/generic/morphism.py: 2% (1 of 40)
I already have some code for this file, so I'll try to add to it.
Also, if someone wants to finish reviewing #5631, there's a handful of
doctests there.
Best,
Alex
--
Alex Ghitza -- Lecturer in Mathematics -- The University of Melbourne
-- Australia -- http://www.ms.unimelb.edu.au/~aghitza/
William Stein
>
>>
>> We will have a docday this Saturday, with the insanely ambitious goal
>> of getting to 70%. This is probably not humanely possible for a
>> single person to do in 8 hours, so I hope I'm not the only one (I
>> estimate it takes *at least* 3 minutes per doctest, which comes to
>> 37.8 hours for 756 doctests).
>
I just noticed that doctesting extended_integer_ring.py would make a
really big dent -- it has 128 functions in it of which only 4% are
doctested now, most functions are very easy to doctest, *and*
extended_rational_field.py, which should be very similar, has 98%
coverage. If somebody wants to easily make a big dent in the
coverage problem, volunteer now to doctest extended_integer_ring.py,
and do it asap:
rings/extended_integer_ring.py: 4% (6 of 128)
rings/extended_rational_field.py: 98% (73 of 74)
-- William
John Cremona
aactually are, so it is hard to get started....
John
2009/4/10 William Stein <wst...@gmail.com>:
David Roe
David
William Stein
>
> For me the trouble with extended_*.py is that I have no idea what they
> aactually are, so it is hard to get started....
But aren't you just dying to find out what they are? :-)
William
John Cremona
the patch at #5685 (still awaiting review) and had to adjust a couple
of functions in one of those files. I can guess: it somehow adds
+infinity and -infinity to ZZ in a way that makes as much sense as
possible.
But search_src("ExtendedInteger") only picks up these two files, which
suggests to me (maybe wrongly) that they are not actually used
anywhere!
John
>
> William
>
> >
>
William Stein
They are used. ExtendedInteger is used by the extended_rational_field
file, and that is used in at least two places:
rings/rational_field.py: sage: E = ExtendedRationalField
rings/padics/valuation.py:QQe =
sage.rings.extended_rational_field.ExtendedRationalField
William
John Cremona
be infinite. (Though I notice that ZZ(0).valuation(3) is plain old
<class 'sage.rings.infinity.PlusInfinity'>).
Now the padics section is also seriously under-documented. Funny
coincidence, that...
John
>
> William
>
> >
>
David Roe
:-) There's a patch in trac (#4637) adding documentation to p-adics, and I'm going to write a bunch of doctests tomorrow.
And yes, I originally wrote Extended Integers and Extended Rationals when I was writing lazy p-adics. Since lazy p-adics aren't currently in good enough shape to be turned on, I don't think any part of the sage library uses Extended Integers or Extended Rationals. Upon further reflection, I'm not sure I even need them for lazy p-adics. I don't know if it's a good idea to just get rid of extended integers and rationals or not.
David
John
>
> William
>
> >
>
William Stein
+1 to getting rid of them. I say we just delete both files. Nobody
knows what they are really, and they aren't needed, and they are
probably partly broken given the bad coverage (at least of integer).
I made this #5735:
http://trac.sagemath.org/sage_trac/ticket/5735
-- William
Robert Bradshaw
I'll second this. If we need them someday, then we'll accept them
with full coverage. Actually, I think this could probably be better
done lazily and generically rather than re-defining every method that
exists in the integers/rationals.
- Robert
John Cremona
that they are redundant.
Deleting these two files will increase the overall coverage rather quickly ;)
John
>
> - Robert
>
>
> >
>
mabshoff
http://wiki.sagemath.org/doc4
If you are working on any coverage at the moment and/or have any
relevant tickets please add them there so we avoid duplication of
work.
Re p-adics: It would be nice if the doctesting issues at #5499 could
be resolved so that p-adic followup tickets, i.e. #4637 could be
resolved.
Cheers,
Michael
William Stein
>
> FYI: I have started a wiki page at
>
> http://wiki.sagemath.org/doc4
welcome too. Everyone welcome.
Login to sage.math.washington.edu with ssh
Run irssi
/server irc.freenode.net
/join #sage-devel
-- William
>
> If you are working on any coverage at the moment and/or have any
> relevant tickets please add them there so we avoid duplication of
> work.
>
> Re p-adics: It would be nice if the doctesting issues at #5499 could
> be resolved so that p-adic followup tickets, i.e. #4637 could be
> resolved.
>
> Cheers,
>
> Michael
> >
>
Nicolas M. Thiery
Please also exclude categories/pushout.py for that reason.
Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/