+ Examples are priceless.

+ Examples that don't work are worse than worthless.

+ Examples that work eventually turn into examples that don't.

+ Docstrings too often don't get written.

+ Docstrings that do get written rarely contain those priceless examples.

+ The rare written docstrings that do contain priceless examples eventually
turn into rare docstrings with examples that don't work.  I think this one
may follow from the above ...

+ Module unit tests too often don't get written.

+ The best Python testing gets done in interactive mode, esp. trying
endcases that almost never make it into a test suite because they're so
tedious to code up.

+ The endcases that were tested interactively-- but never coded up --also
fail to work after time.

About a month ago, I tried something new:  take those priceless interactive
testing sessions, paste them into docstrings, and write a module to do all
the rest by magic (find the examples, execute them, and verify they still
work exactly as advertised).

Wow -- it turned out to be the only scheme I've ever really liked, and I
like it a lot!  With almost no extra work beyond what I was doing before,
tests and docstrings get written now, and I'm certain the docstring examples
are accurate.  It's also caught an amazing number of formerly-insidious
buglets in my modules, from accidental changes in endcase behavior, to hasty
but inconsistent renamings.

doctest.py is attached, and it's the whole banana.  Give it a try, if you
like.  After another month or so of ignoring your groundless complaints,
I'll upload it to the python.org FTP contrib site.  Note that it serves as
an example of its own use, albeit an artificially strained example.

winklessly y'rs  - tim

# Module doctest.
# Released to the public domain 06-Mar-1999,
# by Tim Peters (tim_...@email.msn.com).

# Provided as-is; use at your own risk; no warranty; no promises; enjoy!

"""Module doctest -- a framework for running examples in docstrings.

NORMAL USAGE

In normal use, end each module M with:

def _test():
    import M, doctest           # replace M with your module's name
    return doctest.testmod(M)   # ditto

if __name__ == "__main__":
    _test()

Then running the module as a script will cause the examples in the
docstrings to get executed and verified:

python M.py

This won't display anything unless an example fails, in which case
the failing example(s) and the cause of the failure(s) are printed
to stdout (why not stderr? because stderr is a lame hack <0.2 wink>),
and the final line of output is "Test failed.".

Run it with the -v switch instead:

python M.py -v

and a detailed report of all examples tried is printed to stdout, along
with assorted summaries at the end.

You can force verbose mode by passing "verbose=1" to testmod, or prohibit
it by passing "verbose=0".  In either of those cases, sys.argv is not
examined by testmod.

In any case, testmod returns a 2-tuple of ints, (f, t), where f is the
number of docstrings examples that failed and t is the total number of
docstring examples attempted.

WHICH DOCSTRINGS ARE EXAMINED?

In any case, M.__doc__ is searched for examples.

By default, the following are also searched:

+ All functions in M.__dict__.values(), except those whose names
  begin with an underscore.

+ All classes in M.__dict__.values(), except those whose names
  begin with an underscore.

By default, any classes found are recursively searched similarly, to
test docstrings in their contained methods and nested classes.  Pass
"deep=0" to testmod and *only* M.__doc__ is searched.

Warning:  imports can cause trouble; e.g., if you do

from XYZ import XYZclass

then XYZclass is a name in M.__dict__ too, and doctest has no way to
know that XYZclass wasn't *defined* in M.  So it may try to execute the
examples in XYZclass's docstring, and those in turn may require a
different set of globals to work correctly.  I prefer to do "import *"-
friendly imports, a la

import XYY
_XYZclass = XYZ.XYZclass
del XYZ

and then the leading underscore stops testmod from going nuts.  You may
prefer the method in the next section.

WHAT'S THE EXECUTION CONTEXT?

By default, each time testmod finds a docstring to test, it uses a
*copy* of M's globals (so that running tests on a module doesn't change
the module's real globals).  This means examples can freely use any
names defined at top-level in M.  It also means that sloppy imports (see
above) can cause examples in external docstrings to use globals
inappropriate for them.

You can force use of your own dict as the execution context by passing
"globs=your_dict" to testmod instead.  Presumably this would be a copy
of M.__dict__ merged with the globals from other imported modules.

WHAT IF I WANT TO TEST A WHOLE PACKAGE?

Piece o' cake, provided the modules do their testing from docstrings.
Here's the test.py I use for the world's most elaborate Rational/
floating-base-conversion pkg (which I'll distribute some day):

from Rational import Cvt
from Rational import Format
from Rational import machprec
from Rational import Rat
from Rational import Round
from Rational import utils

modules = (Cvt,
           Format,
           machprec,
           Rat,
           Round,
           utils)

def _test():
    import doctest
    import sys
    verbose = "-v" in sys.argv
    for mod in modules:
        doctest.testmod(mod, verbose=verbose, report=0)
    doctest.master.summarize()

if __name__ == "__main__":
    _test()

IOW, it just runs testmod on all the pkg modules.  testmod remembers the
names and outcomes (# of failures, # of tries) for each item it's seen,
and passing "report=0" prevents it from printing a summary in verbose
mode.  Instead, the summary is delayed until all modules have been
tested, and then "doctest.master.summarize()" forces the summary at the
end.

So this is very nice in practice:  each module can be tested individually
with almost no work beyond writing up docstring examples, and collections
of modules can be tested too as a unit with no more work than the above.

SO WHAT DOES A DOCSTRING EXAMPLE LOOK LIKE ALREADY!?

Oh ya.  It's easy!  In most cases a slightly fiddled copy-and-paste of an
interactive console session works fine -- just make sure there aren't any
tab characters in it, and that you eliminate stray whitespace-only lines.

    >>> # comments are harmless
    >>> x = 12
    >>> x
    12
    >>> if x == 13:
    ...     print "yes"
    ... else:
    ...     print "no"
    ...     print "NO"
    ...     print "NO!!!"
    no
    NO
    NO!!!
    >>>

Any expected output must immediately follow the final ">>>" or "..."
line containing the code, and the expected output (if any) extends
to the next ">>>" or all-whitespace line.  That's it.

Bummer:  only non-exceptional examples can be used.  If anything raises
an uncaught exception, doctest will report it as a failure.

The starting column doesn't matter:

and as many leading blanks are stripped from the expected output as
appeared in the code lines that triggered it.

If you execute this very file, the examples above will be found and
executed, leading to this output in verbose mode:

Running doctest.__doc__
Trying: # comments are harmless
Expecting: nothing
ok
Trying: x = 12
Expecting: nothing
ok
Trying: x
Expecting: 12
ok
Trying:
if x == 13:
    print "yes"
else:
    print "no"
    print "NO"
    print "NO!!!"
Expecting:
no
NO
NO!!!
ok
... and a bunch more like that, with this summary at the end:

2 items had no tests:
    doctest.run_docstring_examples
    doctest.testmod
4 items passed all tests:
   7 tests in doctest
   2 tests in doctest.TestClass
   2 tests in doctest.TestClass.get
   1 tests in doctest.TestClass.square
12 tests in 6 items.
12 passed and 0 failed.
Test passed.
"""

__version__ = 0, 0, 1

import types
_FunctionType = types.FunctionType
_ClassType    = types.ClassType
_ModuleType   = types.ModuleType
del types

import string
_string_find = string.find
del string

import re
PS1 = re.compile(r" *>>>").match
PS2 = "... "
del re

# Extract interactive examples from a string.  Return a list of string
# pairs, (source, outcome).  "source" is the source code, and ends
# with a newline iff the source spans more than one line.  "outcome" is
# the expected output if any, else None.  If not None, outcome always
# ends with a newline.

def _extract_examples(s):
    import string
    examples = []
    lines = string.split(s, "\n")
    i, n = 0, len(lines)
    while i < n:
        line = lines[i]
        i = i + 1
        m = PS1(line)
        if m is None:
            continue
        j = m.end(0)  # beyond the prompt
        if string.strip(line[j:]) == "":
            # a bare prompt -- not interesting
            continue
        assert line[j] == " "
        j = j + 1
        nblanks = j - 4  # 4 = len(">>> ")
        blanks = " " * nblanks
        # suck up this and following PS2 lines
        source = []
        while 1:
            source.append(line[j:])
            line = lines[i]
            if line[:j] == blanks + PS2:
                i = i + 1
            else:
                break
        if len(source) == 1:
            source = source[0]
        else:
            source = string.join(source, "\n") + "\n"
        # suck up response
        if  PS1(line) or string.strip(line) == "":
            expect = None
        else:
            expect = []
            while 1:
                assert line[:nblanks] == blanks
                expect.append(line[nblanks:])
                i = i + 1
                line = lines[i]
                if PS1(line) or string.strip(line) == "":
                    break
            expect = string.join(expect, "\n") + "\n"
        examples.append( (source, expect) )
    return examples

And I'm writing a paper on testing software using scripting languages.  This
technique is VERY similar to the one I'm recommending, and I'm going to
plagarize shamelessly.  It's more complicated, and doesn't apply to most
other languages, but has a lot of side benefits.

Great post, but I question your timing.  I can't help noticing
that we're only a few weeks away from the next Pythonic Award.

all-the-best-episodes-occur-during-sweeps-week-ly yr's,

Best regards,

i like your idea better for the reason that each example is located
near its code, so a failure could be detected and the user sent to
where the problem __may__ lie.

as for interactive testing being the best mode, i agree for short
pieces of code i go to the interpreter, hack around, and then paste a
goodie into my module.

but for long winded code with many dependent pieces (a ray tracer
firing and bouncing rays around reflecting and refracting surfaces) i
tend to test only some larger subset -- for example, the initial ray
and the final ray through a refracting prism -- by running a script
and examining output.

but i guess there is a whole art to figuring out at what granularity
one should be testing (and re-testing) pieces of code (functions, a
few lines, classes, etc). i am still learning about this after coding
for 26 years, and still feel i have too much to learn.

<rant> finally, what i like best about your idea and design is that it
further encourages documentation, visible examples, and testing right
within the code base. i feel so stringly taht in the age of
internetwide development of code, all these things must be integrated
INTO THE SOURCE FILE to be really useable.

sometimes i tire so of clicking over to the browser to read the
doc.html, clicking to the README to look at the example, clicking to
xemacs to look at the source, clicking to the xterm to look at a
python interpreter. etc. etc. just as structs (classes) were created
by divine inspiration (intervention?)  to collect different but
related values (and functions), so too should open source code gather
all its parts together.  </rant>

nice work tim.....

I could imagine one improvement:
Docstrings might get a little too much used for all
kinds of stuff.

Zope uses the existance of docstrings to decide wether a thing
will be published.

Others (was it J. Asbahr?) use docstrings to store regexen
for little languages compilers.

I'm working on type library support for Python Servers, and
docstrings will show up as popups form VB and colleagues.

This means to me that we need some way to distinguish
docstring meanings:
For typeinfo support, it would be bad to have the test code in
a multi-lined docstring.
For Zope, we need a way to split the meaning of the
docstring existence. Zope first look if there is an
object.__doc__ attribute and then for entry_name+"__doc__".
My point is to extend Zope's idea to __test__.

Kinda proposal:
If an object has an __test__ attribute, it is preferred to
__doc__. Currently this doesn't help with functions, so I'd
say we could use name mangling like function_name+"__test__",
with the same semantics as Zope has.
a) try to find a __test__ attribute. If not found,
b) try to find a global object_name+"__test__" attribute. I.n.f.,
c) try to find an object.__doc__ attribute. If not found,
d) try to find a global object_name+"__doc__" attribute.

I think, with this scheme, we can have tested Zope code which
is not published, also a short doc string for type libraries,
a longer alternative by object_name+"__doc__", and so on.

cheers - chris

if-tabs-were-called-hemorrhoids-as-god-intended-people-would-
    see-them-for-the-bloody-mess-they-are-ly y'rs  - tim

*** Old/doctest.py      Sat Mar 06 14:35:36 1999
--- New/doctest.py      Sat Mar 06 14:38:58 1999
***************
*** 1,4 ****
! # Module doctest.
  # Released to the public domain 06-Mar-1999,
  # by Tim Peters (tim_...@email.msn.com).

--- 1,4 ----
! # Module doctest version 0.0.2
  # Released to the public domain 06-Mar-1999,
  # by Tim Peters (tim_...@email.msn.com).

***************
*** 81,90 ****

  By default, each time testmod finds a docstring to test, it uses a
  *copy* of M's globals (so that running tests on a module doesn't change
! the module's real globals).  This means examples can freely use any
! names defined at top-level in M.  It also means that sloppy imports (see
! above) can cause examples in external docstrings to use globals
! inappropriate for them.

  You can force use of your own dict as the execution context by passing
  "globs=your_dict" to testmod instead.  Presumably this would be a copy
--- 81,91 ----

  By default, each time testmod finds a docstring to test, it uses a
  *copy* of M's globals (so that running tests on a module doesn't change
! the module's real globals, and so that one test in M can't leave behind
! crumbs that accidentally allow another test to work).  This means
! examples can freely use any names defined at top-level in M.  It also
! means that sloppy imports (see above) can cause examples in external
! docstrings to use globals inappropriate for them.

  You can force use of your own dict as the execution context by passing
  "globs=your_dict" to testmod instead.  Presumably this would be a copy
***************
*** 136,144 ****

  SO WHAT DOES A DOCSTRING EXAMPLE LOOK LIKE ALREADY!?

! Oh ya.  It's easy!  In most cases a slightly fiddled copy-and-paste of an
! interactive console session works fine -- just make sure there aren't any
! tab characters in it, and that you eliminate stray whitespace-only lines.

      >>> # comments are harmless
      >>> x = 12
--- 137,147 ----

  SO WHAT DOES A DOCSTRING EXAMPLE LOOK LIKE ALREADY!?

! Oh ya.  It's easy!  In most cases a copy-and-paste of an interactive
! console session works fine -- just make sure the leading whitespace
! is rigidly consistent (you can mix tabs and spaces if you're too lazy
! to do it right, but doctest is not in the business of guessing what
! you think a tab means).

      >>> # comments are harmless
      >>> x = 12
***************
*** 150,155 ****
--- 153,159 ----
      ...     print "no"
      ...     print "NO"
      ...     print "NO!!!"
+     ...
      no
      NO
      NO!!!
***************
*** 169,176 ****
              >>> math.floor(1.9)
              1.0

! and as many leading blanks are stripped from the expected output as
! appeared in the code lines that triggered it.

  If you execute this very file, the examples above will be found and
  executed, leading to this output in verbose mode:
--- 173,180 ----
              >>> math.floor(1.9)
              1.0

! and as many leading whitespace characters are stripped from the expected
! output as appeared in the initial ">>>" line that triggered it.

  If you execute this very file, the examples above will be found and
  executed, leading to this output in verbose mode:
***************
*** 212,218 ****
  Test passed.
  """

! __version__ = 0, 0, 1

  import types
  _FunctionType = types.FunctionType
--- 216,232 ----
  Test passed.
  """

! # 0,0,1    06-Mar-1999
! #    initial version posted
! # 0,0,2    06-Mar-1999
! #    loosened parsing:
! #        cater to stinkin' tabs
! #        don't insist on a blank after PS2 prefix
! #            so trailing "... " line from a compound stmt no longer
! #            breaks if the file gets whitespace-trimmed
! #    better error msgs for inconsistent leading whitespace
!
! __version__ = 0, 0, 2

  import types
  _FunctionType = types.FunctionType
***************
*** 225,232 ****
  del string

  import re
! PS1 = re.compile(r" *>>>").match
! PS2 = "... "
  del re

  # Extract interactive examples from a string.  Return a list of string
--- 239,249 ----
  del string

  import re
! PS1 = ">>>"
! PS2 = "..."
! _isPS1 = re.compile(r"(\s*)" + re.escape(PS1)).match
! _isPS2 = re.compile(r"(\s*)" + re.escape(PS2)).match
! _isEmpty = re.compile(r"\s*$").match
  del re

  # Extract interactive examples from a string.  Return a list of string
***************
*** 237,283 ****

  def _extract_examples(s):
      import string
      examples = []
      lines = string.split(s, "\n")
      i, n = 0, len(lines)
      while i < n:
          line = lines[i]
          i = i + 1
!         m = PS1(line)
          if m is None:
              continue
          j = m.end(0)  # beyond the prompt
!         if string.strip(line[j:]) == "":
              # a bare prompt -- not interesting
              continue
!         assert line[j] == " "
          j = j + 1
!         nblanks = j - 4  # 4 = len(">>> ")
!         blanks = " " * nblanks
          # suck up this and following PS2 lines
          source = []
          while 1:
              source.append(line[j:])
              line = lines[i]
!             if line[:j] == blanks + PS2:
                  i = i + 1
              else:
                  break
          if len(source) == 1:
              source = source[0]
          else:
              source = string.join(source, "\n") + "\n"
          # suck up response
!         if  PS1(line) or string.strip(line) == "":
              expect = None
          else:
              expect = []
              while 1:
!                 assert line[:nblanks] == blanks
                  expect.append(line[nblanks:])
                  i = i + 1
                  line = lines[i]
!                 if PS1(line) or string.strip(line) == "":
                      break
              expect = string.join(expect, "\n") + "\n"
          examples.append( (source, expect) )
--- 254,312 ----

  def _extract_examples(s):
      import string
+     isPS1, isPS2, isEmpty = _isPS1, _isPS2, _isEmpty
      examples = []
      lines = string.split(s, "\n")
      i, n = 0, len(lines)
      while i < n:
          line = lines[i]
          i = i + 1
!         m = isPS1(line)
          if m is None:
              continue
          j = m.end(0)  # beyond the prompt
!         if isEmpty(line, j):
              # a bare prompt -- not interesting
              continue
!         if line[j] != " ":
!             raise ValueError("line " + `i-1` + " of docstring lacks "
!                 "blank after " + PS1 + ": " + line)
          j = j + 1
!         blanks = m.group(1)
!         nblanks = len(blanks)
          # suck up this and following PS2 lines
          source = []
          while 1:
              source.append(line[j:])
              line = lines[i]
!             m = isPS2(line)
!             if m:
!                 if m.group(1) != blanks:
!                     raise ValueError("inconsistent leading whitespace "
!                         "in line " + `i` + " of docstring: " + line)
                  i = i + 1
              else:
                  break
          if len(source) == 1:
              source = source[0]
          else:
+             # get rid of useless null line from trailing empty "..."
+             if source[-1] == "":
+                 del source[-1]
              source = string.join(source, "\n") + "\n"
          # suck up response
!         if  isPS1(line) or isEmpty(line):
              expect = None
          else:
              expect = []
              while 1:
!                 if line[:nblanks] != blanks:
!                     raise ValueError("inconsistent leading whitespace "
!                         "in line " + `i` + " of docstring: " + line)
                  expect.append(line[nblanks:])
                  i = i + 1
                  line = lines[i]
!                 if isPS1(line) or isEmpty(line):
                      break
              expect = string.join(expect, "\n") + "\n"
          examples.append( (source, expect) )

It's well-known Python Corporate Policy never to admit that tabs can cause
problems.  So the Steering Committee never mentioned this in public:  Guido
was so despondent over all the problems caused by leading whitespace that he
had typed up a resignation letter, aiming to start a new life as a ferret
breeder (an avocation to which the Dutch psyche is particularly
well-attuned; from whence the charming English expression "like feeding
tulips to a ferret"; etc).

I had schemed to usurp Guido by solving this problem once & for all, and
with Gordon's goading produced the triumphant tabnanny.py (available in a
Tools/Scripts/ directory near you!).  But I hadn't sufficiently anticipated
Guido's fabled Dutch ferocity:  seeing that the problem was now well & truly
solved, the clever bastard used his time machine to introduce the
serviceable workalike tabpolice.py, some weeks *before* tabnanny.py.  Thus
my pioneering effort was made to appear as if the work of a madman,
laboriously elaborating a minor refinement beyond all reason, and--
indeed! --beyond even the bounds of decency.

The Steering Committee, quite familiar with Guido's mind-clouding tricks,
honored me with the Pythonic Award anyway, over Guido's objection that
tabnanny.py was clearly the work of a madman, laboriously etc.  This epic
battle for power continued, unseen by the public, for several more months,
until I discovered that Guido doesn't actually have any groupies!  Python
really isn't Perl.  So I figured "why bother?", and have been at peace ever
since.

Guido is still at it, though.  I suppose none of you remember my
introduction late last summer of GUINANNY, the Tk-based Python development
environment featuring rigorous treatment of leading whitespace in all cases.
This time Guido retroactively undid the postings and long praiseful
discussions, replacing them with his serviceable workalike IDLE instead.

Am I bitter?  You'd think so -- but, no.  Not at all!  You see, Pythonic
Award Winners *do* have groupies.

[Christian Tismer]

Besides, some well-chosen examples *belong* in docstrings, and this scheme
keeps them honest -- you can't object to putting documentation in docstrings
<wink>.  For hordes of unilluminating examples, I'm doing stuff like:

def for_testing():
    """
... hordes of lines deleted ...

... hordes of lines deleted ...
"""

The user never sees that unless they look at the source, but the framework
runs it along with everything else, and it is *so pleasant* to write tests
this way:  just paste a session into the string & add a few comments.

Python2 should allow setting attributes on any object (remind me to remind
Guido about that ...), and when I can set a __test__ attribute on functions
and class methods I'll be happy to do that.  As is, there's no (pleasant)
way to associate information with functions or class methods other than
docstrings, so of course they're going to get warped toward conflicting
purposes.

It would be very easy to add an optional __test__=0 arg to testmod, though,
which if true would look *only* in module.__test__ for something to test.

I don't want something fancier than that because I can't guess what people
might need; e.g., I'm already feeling a need to distinguish among "short"
tests and "long" tests and "overnight" tests.  This suggests making __test__
a flexible structure, like a dict mapping names to strings to be tested.
But then there's no builtin *policy* left, so insisting on calling it
__test__ and making it a dict just impose restrictions.  So I'd rather
expose the _Tester class, add new

    runstring(self, string, name, ...)
and
    runobject(self, obj [, name], ...) # run obj.__doc__

methods, and let the user build whatever policies they want out of those.

testmod's policy of running everything in sight will stay, though:  I'll be
damned if I change this into something I don't want to use myself <0.5
wink>.

Not doing so will confuse not only auto doc tools but also
programmers who are not aware of the special meaning of those
particular doc strings.

BTW: In 1.5.1 there's a new optimization -OO that will cause the
byte code compiler to not only perform the usual -O optimizations
but also have it ignore any doc strings found in the source code.
Programs relying on doc strings for whatever purpose will of course
break...

[Christian Tismer]

Not that I've actually written any (more than 100 lines)
of Python, nor used Zope (more than 10 minutes), but I
am following both, in anticipation of greater fluency.

Seems like Zope shouldn't be telling me I can't docstring
what I don't want published.

On the other hand, I'm not convinced that your use of
docstrings for testing fits my biases either.  I would
have hoped (pure imagination here) that docstrings would
be a suitable place for a sort of plain text man page,
telling how to use a method or class.  Tests would
quickly obscure such usage.

Maybe its time for a little xml-style markup.

As in surrounding your tests with <test> ... </test>.
And Zope using a <zope/> tag to mean publish.
And my imagined man page with <man> ... </man>

You could even imagine nested test tags, such as
<test> ... <overnight> ... </overnight> ... </test>

All in the docstring, along with some other untagged
or otherwise tagged stuff.

Oh well -- just fantasizing here, in the easy style
only available to the unwashed and uninitiated.

You're right, Tim, that if it doesn't lead to actual
usage, to a practical increase in testing, then it
doesn't matter how nice it sounds in theory.
--

The tests in the doc string should be somewhat illustrative, of course; but
they should cover the entire range of inputs (the way you're supposed to
write tests).  If a test gets too much in the way of clarity, then change it
into a function and call it from the doc.

Has anyone ever accidentally published Zope stuff?  I don't really think
it's that easy.  The __doc__ attribute is only part of what you need.

    def investigate_all_known_Y2K_problems(self):
        """This is a method."""

If you want add docstrings to non-publishable methods, you could
always declare them with leading underscores.  Another alternative
is to use comments for you non-publishable Zope methods.  For me,
this is sometimes even useful as a documentation aid.

    def charge_outrageous_Y2K_consulting_fee(self):
        # no docstring, this is not a publishable method.

requesting-a-'private'-keyword-is-not-an-option-ly yr's,

As for John Aycock's parsing framework, you subclass his Lexer /
Parser / Generator, and embed the rule in the docstring and the
action in the code of the methods you add. Both the rule and action
tend to be very simple, most of the behavior is in the framework. It
might sound perverse, but it's very natural in practice. With the
possible (and trivial) exception of the regexes in the Lexer, you
wouldn't be testing the individual methods, but the behavior of the
Lexer / Parser / Generator as a whole.

In all these cases, the major problem is that methods / functions
have only one user-assignable attribute. Christian, John, Tim and the
Zope folks all want to associate information with methods /
functions. Well, Tim has the additional problem that his usability
tester is (to be charitable) somewhat less than reasonable...

William "Billy" Tanksley writes, in reply to my:
|> >  Tests would quickly obscure such usage.

|> As a matter of fact, they don't.  The tests are VITAL documentation,

melarkey.  I write tests for (some of) what I code,
and I write man-page like interface description and
how to use commentary for (more of) what I code.

They are not the same thing.  Tests are not documentation.
One is a tool to find bugs, the other is a means to explain
to fellow humans something about the code in question.

|>  But Tim's tests already have markup - ... ">>>".

true - good point.
--

And who will be the "unfortunate successor"? Can there be any successor?
I mean Tim++ oh sorry: Tim=Tim+1 :-)

Well, they've been used that way before, but that's by no means what they're
for.  By and large, in the few timmes I've seen docstrings used, they've
contained fairly extensive documentation, usually of an entire module
(function docstrings get used less, and when they are used it is indeed more
often short).

In doctests, IF you put a docstring in AND it has the appropriate
interactive-like syntax in it at some point AND you call doctest on its
object, THEN the stuff will be executed.  All other stuff is ignored.

How is this -- what was your word -- "less than reasonable"?  Seems almost
boolean.

def _fat_test:
    """A really long test script"""
    whatever

_fattest = r"""\

A call to doctest.master.runstring(_fattest) will do the rest, merging the
_fat_test results into the rest of the module's test results.  I think I'll
graft in a simple version of Chris's __test__ idea too, so you can do e.g.

__test__ = {"fat_test": _fattest}

and have testmod run it by magic.

    """a, b -> greatest common divisor.

    This is > 0 unless a == b == 0, in which case return 0.

    >>> print gcd(3, 8), gcd(8, 3), gcd(2L**500, 2L**500 +1)
    1 1 1L
    >>> print gcd(6, 15), gcd(15, 6), gcd(15, -6), gcd(-6, -15)
    3 3 3 3
    >>> print gcd(-6, -6), gcd(0, 6), gcd(-6, 0), gcd(6, -12)
    6 6 6 6
    >>> gcd(0, 0)
    0
    """

I think the console session pasted in there is much clearer about endcase
behavior than would be an equally large pile of obscure technical prose.  I
don't consider those to be gcd tests so much as a vital part of the docs.
The strenuous but unilluminating gcd *tests* are hiding elsewhere in the
module from which this was taken.

BTW, while speeding the gcd function, on at least two occasions some of
those examples stopped working as advertised!

This is valuable in a different way in "tutorial mode", where actual
input/output illustrates points made in the prose.  For example, here's part
of a module's tutorial docstring:

    So what does this rounded guy really look like?  Here it is in binary:

    >>> print rounded.str(mode=Format.FIXED, base=2, prec=55)
    0.0001100110011001100110011001100110011001100110011001101_2
    >>>

    Since the machine value isn't exactly 1/10, it shouldn't be surprising
    that if we print out enough digits they won't be exactly "0.100..."
    either (.cvt_exact(base) attempts exact conversion to the given base,
    but returns None if that's impossible):

    >>> print rounded.cvt_exact(10)
    0.1000000000000000055511151231257827021181583404541015625
    >>>

    And that's the exact decimal value of the best possible IEEE double
    approximation to 1/10!

IOW, I'm aiming for illumination, not distraction.  And, BTW, one of those
examples stopped working during development too (change in keyword argument
name -- a million things can cause examples to break, and now I no longer
have to worry about letting one slip thru the cracks).

Besides, I won't like XML-style markup of docstrings until a std tool is
distributed for displaying docstrings with that crap weeded out.  I'll stick
to the "plain ASCII manpage" idea until then, and think this scheme isn't
stretching that even a teensy bit.

I will shortly be posting a short discussion (about 10-20pp, double spaced)
about testing software written in "scripting" languages.  Tim's method is,
of all of the things I've reasearched, by FAR the best by any number of
criteria.  I could earn a PhD with this writeup...  And Tim won't get a
THING!

In fact, the test has two purposes after that first passed test.

First of all, it tells the customer what the coder was planning when he
wrote the code.  If the tests don't match the requirements, the customer
should complain QUICK.  (If the customer is a coder, he can also use the
tests as a sample of use, but that's secondary.)

Second, it tells the maintainer briefly what the original coder intended the
limits of his code to be, and provides him with a quick way to affirm that
his code meets the original requirements.

Both of these jobs are primarily issues of human communication.

Guido, could the next version of IDLE support doctests in a graphical way,
perhaps tree-format, with percentage tested summaries and a way to activate
and view a single test with a click or two?

A brief description of an object's interface.

Neither a line like "I'm a method" nor a three page
description including several test sessions makes sense
in this context.

ciao - chris

...

Without giving a syntax proposal, it should be noted that
an initialization function of a module could do with the
doc strings whatever it wants to do. An initializer could
modify docstrings arbitrarily, depending of the actual purpose.

In the case of my COM stuff, I could cut the doc down to a
brief description, if the module is published by COM, otherwise
leave the longer docs intact.

Tim's code could stay as it is (since it is indeed tagged), or it
could be modified. Leaving some brief examples which are helpful
for the user, but removing more extensive testing stuff. This
would enable Tim to do even more in place, with his decision
wether to expose it to the user or not.

Another (less related) benefit of docstring modification code
could be this: People tend to write doc strings this way:

def func(someargs):
    """sometimes this is line 1
    and this is line 2
    """

or something similar. While this notation is readable in the
source code, the printout isn't perfect. A modification at
runtime which adjusts extra line feeds and indentation
can make sense.

At the risk that I might be called perverse (some of you
don't do so already :), I want to note that the __doc__
attribute can take any Python object, not just strings.
One *could* decide to execute a docstring and assign the
result back, or even replace it with an arbitrary object,
which might show runtime configurable doc text.

I just played with that. Wrote a doc class with a "brief" flag,
and wrapped instances into a module's object around all doc
strings. One can extend this idea and teach __doc__ to do a lot
more. For instance, and instance can show example code with
instance specific data filled in.

This was just to inject an idea. Doc strings are by no means
restrictive and can be extended as seems fit. The possibility
to assign arbitrary objects is very new to me and needs
some thought, sure.

cheers - chris

(In a clever diversionary tactic, Paul redirects the burden of proof
elsewhere...)

The *real* problem is that you can't assign information to functions.
If I could assign "publish_me=1" to my functions, then we wouldn't have
hijacked the docstring.  Also, this policy in Zope existed prior to
Python 1.5's _ policy.

Perhaps the truth is out there, looming in the interfaces
proposal...anyway, go start an uprising on the Zope list and demand the
dropping of the docstring requirement.  I know I'm in favor of dropping
it.

Not a too serious proposal, but there are some ways...

ciao - chris