doctest a single function or class

[TB]

Dear list,

Is there a way to test a single docstring of a function/method, or all
the docstrings in a specific class, and not an entire module?

This is useful when testing a long file, without copying the needed
docstring to a new file. This is also different than
TestSuite(obj).run(). Currently I could only find the run_doctests()
functions that accepts modules, but not objects. Should I open a ticket
to add support for objects, or a flag to the TestSuite to run the
doctest (but not recursively)?

The command-line interface of the built-in unittest module in Python
supports this usecase:
$ python -m unittest test_module.TestClass.test_method
to test a specific method.

Regards,
TB

[1] https://docs.python.org/3/library/unittest.html#command-line-interface

[Simon King]

Hi,

I am sorry for the late answer (and I am a bit surprised that nobody
else answered before).

I believe it is somehow useful to have such a feature when developing a
large piece of new code (so that one can do a relevant subset of tests
quickly). However, I am not so sure how that feature would be used once
the piece of code is published. I.e., I cannot see how that feature
would be useful in the Sage test suites, but I see potential use in
development.

Is that the use case you had in mind?

Best regards,
Simon

[E. Madison Bray]

On Tue, Aug 13, 2019 at 11:28 AM Simon King <simon...@uni-jena.de> wrote:
>
> Hi,
>
> I am sorry for the late answer (and I am a bit surprised that nobody
> else answered before).
>
> I believe it is somehow useful to have such a feature when developing a
> large piece of new code (so that one can do a relevant subset of tests
> quickly). However, I am not so sure how that feature would be used once
> the piece of code is published. I.e., I cannot see how that feature
> would be useful in the Sage test suites, but I see potential use in
> development.

It's useful to have in general, even after initial development of some
code. For example, sometimes if just one test is failing in some
particular case or another (i.e. a regression) it is very useful to be
able to easily run just that test.

For example, in pytest, it's possible to test not just a specific
file, but also drill down into specific test cases, by name (including
wildcards).

With Sage's extensive use of doctests this is a little trickier but
not impossible, and is something I've often wanted as well.

One approach which would be fairly easy would be to extend the doctest
interface to accept a line number, or even range of line numbers, on
which to run tests. At the command line this could be a syntax like:

./sage -t <filename>:<lineno>

This is imperfect of course. For example, what if you're actively
editing the file that contains the test? Then the line number will
move around and you'd have to keep changing it. If nothing else it
might be able to allow some fuzz (e.g. start on the nearest doctest
block to that line number).

It can also fail if the test you're running depends on some results
from a previous doctest block. Obviously in that case it is the
user's responsibility to make sure that the test they want to focus on
works in isolation (though this is also why allowing a range syntax
like <filename>:<start>-<end> is useful).

> On 2019-07-09, TB <math...@gmail.com> wrote:
> > Dear list,
> >
> > Is there a way to test a single docstring of a function/method, or all
> > the docstrings in a specific class, and not an entire module?
> >
> > This is useful when testing a long file, without copying the needed
> > docstring to a new file. This is also different than
> > TestSuite(obj).run(). Currently I could only find the run_doctests()
> > functions that accepts modules, but not objects. Should I open a ticket
> > to add support for objects, or a flag to the TestSuite to run the
> > doctest (but not recursively)?
> >
> > The command-line interface of the built-in unittest module in Python
> > supports this usecase:
> > $ python -m unittest test_module.TestClass.test_method
> > to test a specific method.
> >
> > Regards,
> > TB
> >
> > [1] https://docs.python.org/3/library/unittest.html#command-line-interface
> >
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qiu3ch%2430iu%241%40blaine.gmane.org.

[TB]

Yes, I was thinking about this feature for a faster edit-test
development loop. This should be useful also when touching existing
code, and not just when adding new code.

Usually I have another shell to run "sage -t /some/file" while editing
that file, and in most cases this is fast enough. But when used with
--long this gets longer, of course, and testing pieces of code that are
surely irrelevant during development.

With such a feature, it should be easy to have a key binding, in an
editor of choice, that would run just the relevant doctests. Looking at
the doctest module docs [1] there is the run_docstring_examples()
function [2] function and an unittest API section, but I do not know how
easy it will be to integrate with Sage's doctest.

Slightly off-topic, while searching for more info I stumbled across the
Manuel package [3]. It says "Manuel lets you build tested
documentation", which might not suit Sage, but can suit other projects.

Regards,
TB

[1] https://docs.python.org/3/library/doctest.html
[2]
https://docs.python.org/3/library/doctest.html#doctest.run_docstring_examples
[3] https://pypi.org/project/manuel/

[E. Madison Bray]

I would suggest opening an issue for it. Doesn't guarantee anyone
will bother to implement it, but if it's just left as a mailing list
thread it's guaranteed no one will remember.