Evolution of the Overall Doctest Coverage of Sage since 3 years

[Sébastien Labbé]

Hi,

I just did a graph of the Evolution of the Overall Doctest Coverage of
Sage since 3 years. The png is attached.

The coverage evolved quiet regularly during the last 20 releases (=20
months) with an average of a bit less than 5 percent a year. If we
keep this rhythm, we should reach 90% coverage by March 2012 and 100%
by April 2014. Is this the rhythm we want?

Sébastien

PS : Can somebody indicate me how to stop the y-axis at 100 without
losing the title ?

[Jason Grout]

On 3/7/11 7:42 PM, S�bastien Labb� wrote:

>
> PS : Can somebody indicate me how to stop the y-axis at 100 without
> losing the title ?

Can you post your code?

One thought is that we really ought to use the matplotlib title
functionality. From the graph, it looks like Sage manually adds a text
item as the title.

Jason

[Tom Boothby]

That's not the trend I'd highlight in the code. Looks to me like
we're overdue for a surge in doctest coverage. Can we get up to 95%
in the next 3 releases?

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>

[slabbe]

> > PS : Can somebody indicate me how to stop the y-axis at 100 without
> > losing the title ?
>
> Can you post your code?

http://www.thales.math.uqam.ca/~labbes/Experimentations/coverage_evolution.py

> One thought is that we really ought to use the matplotlib title
> functionality.  From the graph, it looks like Sage manually adds a text
> item as the title.

I manually added the title. Moreover, I discovered a bug using
axis_coords=True which I usually use for manually adding a title. It
looks like it doesn't like the x-axis starting at 2008 instead of
zero.

Sébastien

[Jason Grout]

On 3/7/11 9:27 PM, slabbe wrote:
> Moreover, I discovered a bug using
> axis_coords=True which I usually use for manually adding a title. It
> looks like it doesn't like the x-axis starting at 2008 instead of
> zero.

Fix is up at #10889, ready for review.

Thanks for the bug report.

Jason

[kcrisman]

On Mar 7, 10:19 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
> That's not the trend I'd highlight in the code.  Looks to me like
> we're overdue for a surge in doctest coverage.  Can we get up to 95%
> in the next 3 releases?
>

Unlikely, I'd say. Much of the remaining doctest holes are ones that
require more specialized knowledge, or are modules that are woefully
underdocumented but the authors are not as active any more. The
reality is that there were some big pushes a year and a half or two
years ago, but since then energy for that has stalled. Also, if you
look at the coverage since about a year ago, the slope is much smaller
- and perhaps the least-squares fit is better?

That isn't to say that a big push couldn't happen again. But it would
take a concerted effort and leadership by someone to help pair talent
with missing doc, more than just outputs of grepping through sage -
coverage to identify where the coverage is missing. Probably there
are also some older sage notebook files (not located in sagenb) that
could be dumped now to help that out? #8803 is similar in vein, and
would be a nontrivial help.

- kcrisman

[Jason Grout]

On 3/7/11 8:05 PM, Jason Grout wrote:

> One thought is that we really ought to use the matplotlib title
> functionality. From the graph, it looks like Sage manually adds a text
> item as the title.

There is a patch now posted up at #10512 which adds support for the
matplotlib title functionality:

plot(sin(x^2), (x, -3, 3), title='Plot of $\sin(x)$')

However, there is a small issue that still needs to be solved (see the
ticket), so it is needs_work for now.

Thanks,

Jason

[Nathann Cohen]

Not to mention this may just be caused by a dilution of the undocumented code. New code -- always documented -- is being added, so the undocumented code disappear in proportion. Not in volume :-)

Nathann

[slabbe]

> That's not the trend I'd highlight in the code. Looks to me like
> we're overdue for a surge in doctest coverage. Can we get up to 95%
> in the next 3 releases?

Maybe we could fix an objective for each release instead of just
having the objective of 90% for sage-5.0. Is gaining 1% per release
too much?

Sébastien

[slabbe]

> Not to mention this may just be caused by a dilution of the undocumented
> code. New code -- always documented -- is being added, so the undocumented
> code disappear in proportion. Not in volume :-)

Right :

sage: (1 - 0.844) * 26816
4183.29600000000
sage: (1 - 0.848) * 27200
4134.40000000000
sage: __ - _
48.8960000000006

Hence, from 4.6.1 to 4.6.2, the number of undocumented functions
decreased by only 49 !! I think we could be able to reduce this by at
least 100 at every release, don't you think?

Sébastien, Montréal

[Andrey Novoseltsev]

On Mar 8, 9:45 am, slabbe <sla...@gmail.com> wrote:
> Maybe we could fix an objective for each release instead of just
> having the objective of 90% for sage-5.0. Is gaining 1% per release
> too much?

I think that any fixed objective for the overall coverage is too much
for "the next release." If sage-5.0 aims for 90%, then release
managers just will not make such a release until 90% are reached and
we will continue having 4.x.y. But if the next release must get 1% and
it gets only 0.3% what should the release manager do? Just wait until
enough patches will be merged to bump the coverage enough? That does
not sound like a good idea. Ask others to write doctests? Who will
actually do it and which ones? I think it is very good that 100%
coverage is required for the new code, and it is easy to enforce, but
who is going to document sage/structure/parent_old.pyx (currently
covered at 5%) if this file is eventually going to be deprecated?

I am not saying that there is no point in improving doctest coverage,
I just think that as with other good possible improvements it is
unlikely that there will be enough effort in this direction and it is
not good if it becomes an obstacle for other areas that do get done,
such as new releases...

Thank you,
Andrey

[kcrisman]

No, unless people are doing tickets specifically to reduce this. It
is very time-consuming, especially in 'internals' files that end-users
are unlikely to be browsing. On the plus side, there are often
(always?) easy-to-fix bugs that are caught this way! But I can think
of several people, including myself, who have spent a lot of time on
this sort of thing, and it becomes a question of whether you want to
spend your time bringing in improvements or documenting code you don't
use on a daily basis with limited time.

But having a concerted push would be a different thing, and that has
been successful a few times in the past.

- kcrisman

[Simon King]

Hi!

On 8 Mrz., 18:29, kcrisman <kcris...@gmail.com> wrote:
> > Hence, from 4.6.1 to 4.6.2, the number of undocumented functions
> > decreased by only 49 !! I think we could be able to reduce this by at
> > least 100 at every release, don't you think?
>
> No, unless people are doing tickets specifically to reduce this.  It
> is very time-consuming, especially in 'internals' files that end-users
> are unlikely to be browsing.  On the plus side, there are often
> (always?) easy-to-fix bugs that are caught this way!

I did such tickets a while ago: #8800 and #9235, which together add
tests for about 75 functions. You are right that in that way one finds
many bugs, which (in the case of #8800) are sometimes not so easy to
fix though...
And it *is* time consuming.

Anyway, #9235 needs review. The doctest failure reported by the
patchbot seems unrelated. I hope that both patches will make it into
sage-4.7.

Cheers,
Simon