how to inherit docstrings?
Eric Snow
of the subclass to get the docstring of the matching method in the
base class. You can do this with decorators (after the class
definition), with class decorators, and with metaclasses [1].
However, I was hoping for a way to do it with just function decorators
on the methods (no metaclass or class decorator). I am not sure if
this is doable. I realize now that this is exactly the reason I got
to thinking last week about objects being notified when they are bound
[2].
So, is there a way to do this with just decorators, or am I "stuck"
with the metaclass/class decorator route? (It's not all that bad :)
Thanks!
-eric
p.s. Am I missing something or can you really not change the docstring
of a class? I was thinking about the idea of inheriting class
docstrings too.
[1] http://code.activestate.com/recipes/577743-using-decorators-to-inherit-function-docstrings/
[2] http://mail.python.org/pipermail/python-ideas/2011-June/010446.html
Ben Finney
> p.s. Am I missing something or can you really not change the docstring
> of a class? I was thinking about the idea of inheriting class
> docstrings too.
The docstring of an object (whether function or class or module) is the
object's ‘__doc__’ attribute. Access that attribute to get the
docstring; re-bind that attribute to set a different docstring.
So, it's even possible to do what you ask without decorators at all:
class Foo(object):
def frob(self):
""" Frobnicate thyself. """
class Bar(Foo):
def frob(self):
pass
frob.__doc__ = Foo.frob.__doc__
Not very elegant, and involving rather too much repetition; but not
difficult.
--
\ “We are no more free to believe whatever we want about God than |
`\ we are free to adopt unjustified beliefs about science or |
_o__) history […].” —Sam Harris, _The End of Faith_, 2004 |
Ben Finney
Eric Snow
> Eric Snow <ericsnow...@gmail.com> writes:
>
>> p.s. Am I missing something or can you really not change the docstring
>> of a class? I was thinking about the idea of inheriting class
>> docstrings too.
>
> The docstring of an object (whether function or class or module) is the
> object's ‘__doc__’ attribute. Access that attribute to get the
> docstring; re-bind that attribute to set a different docstring.
>
Sorry, I should have been more clear:
>>> class X:
... "some doc"
...
>>> X.__doc__
'some doc'
>>> X.__doc__ = "another doc"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: attribute '__doc__' of 'type' objects is not writable
That is on 3.3.
> So, it's even possible to do what you ask without decorators at all:
>
> class Foo(object):
> def frob(self):
> """ Frobnicate thyself. """
>
> class Bar(Foo):
> def frob(self):
> pass
> frob.__doc__ = Foo.frob.__doc__
>
> Not very elegant, and involving rather too much repetition; but not
> difficult.
>
Yeah, definitely you can do it directly for each case. However, the
inelegance, repetition, and immodularity are exactly why I am pursuing
a solution. :) (I included a link in the original message to
examples of how you can already do it with metaclasses and class
decorators too.)
I'm just looking for a way to do it with decorators in the class body
without using metaclasses or class decorators.
Thanks
-eric
> --
> \ “We are no more free to believe whatever we want about God than |
> `\ we are free to adopt unjustified beliefs about science or |
> _o__) history […].” —Sam Harris, _The End of Faith_, 2004 |
> Ben Finney
Carl Banks
> On Thu, Jun 9, 2011 at 12:37 AM, Ben Finney <ben+p...@benfinney.id.au> wrote:
> > So, it's even possible to do what you ask without decorators at all:
> >
> > class Foo(object):
> > def frob(self):
> > """ Frobnicate thyself. """
> >
> > class Bar(Foo):
> > def frob(self):
> > pass
> > frob.__doc__ = Foo.frob.__doc__
> >
> > Not very elegant, and involving rather too much repetition; but not
> > difficult.
> >
>
> Yeah, definitely you can do it directly for each case. However, the
> inelegance, repetition, and immodularity are exactly why I am pursuing
> a solution. :) (I included a link in the original message to
> examples of how you can already do it with metaclasses and class
> decorators too.)
>
> I'm just looking for a way to do it with decorators in the class body
> without using metaclasses or class decorators.
The tricky part is that, inside the class body (where decorators are being evaluated) the class object doesn't exist yet, so the method decorator has no way to infer what the base classes are at that point. A class decorator or metaclass can operate after the class object is made, but a method decorator can't.
The best you could probably do with a method decorator is something like this:
def inherit_docstring(base):
def set_docstring(f):
f.__doc__ = getattr(base,f.func_name).__doc__
return f
return set_docstring
where you have to repeat the base class every time:
class Bar(Foo):
@inherit_docstring(Foo)
def somefunction(self):
pass
Carl Banks
Ben Finney
> AttributeError: attribute '__doc__' of 'type' objects is not writable
>
> That is on 3.3.
Well, that sucks :-(
Where can we see the discussion of that change before it was
implemented?
> I'm just looking for a way to do it with decorators in the class body
> without using metaclasses or class decorators.
Yes, that'd be nice. Do you have a specification in mind for how it
would work? Perhaps time to start a thread on the ‘python-ideas’ forum.
--
\ “Following fashion and the status quo is easy. Thinking about |
`\ your users' lives and creating something practical is much |
_o__) harder.” —Ryan Singer, 2008-07-09 |
Ben Finney
Duncan Booth
> Eric Snow <ericsnow...@gmail.com> writes:
>
>> AttributeError: attribute '__doc__' of 'type' objects is not writable
>>
>> That is on 3.3.
>
> Well, that sucks :-(
>
> Where can we see the discussion of that change before it was
> implemented?
>
Change? What change?
C:\Python27>python
Python 2.7 (r27:82525, Jul 4 2010, 09:01:59) [MSC v.1500 32 bit
(Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> class C(object):
... "Hello world"
...
>>> C.__doc__ = "whatever"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: attribute '__doc__' of 'type' objects is not writable
>>>
Or even:
Python 2.3.5 (#1, Oct 13 2005, 09:17:23)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class C(object):
... "Hello world"
...
>>> C.__doc__ = "whatever"
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: attribute '__doc__' of 'type' objects is not writable
>>>
--
Duncan Booth http://kupuguy.blogspot.com
Steven D'Aprano
> Eric Snow <ericsnow...@gmail.com> writes:
>
>> AttributeError: attribute '__doc__' of 'type' objects is not writable
>>
>> That is on 3.3.
>
> Well, that sucks :-(
>
> Where can we see the discussion of that change before it was
> implemented?
It goes back to Python 2.2, when new style classes were first introduced.
[steve@sylar ~]$ python2.2
Python 2.2.3 (#1, Aug 12 2010, 01:08:27)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class K(object):
... "foo"
...
>>> K.__doc__ = 'bar'
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: attribute '__doc__' of 'type' objects is not writable
It's an unnecessary restriction, as far as I'm concerned, but an old one.
--
Steven
Ethan Furman
> p.s. Am I missing something or can you really not change the docstring
> of a class? I was thinking about the idea of inheriting class
> docstrings too.
8<--------------------------------------------------------
"""module level docstring"""
def func():
"""function level docstring"""
class Test(object):
"""class level docstring"""
def meth(self):
"""method level docstring"""
if __name__ == '__main__':
import sys
import traceback
hmmm = (
sys.modules['__main__'],
func,
Test(),
Test().meth,
Test,
Test.meth,
)
for obj in hmmm:
try:
obj.__doc__ = 'new docstring'
print('successfully changed %s\n' % obj)
except:
traceback.print_exc()
print()
8<--------------------------------------------------------
Tested from 2.5 - 3.2. The first three always work, the last one works
in 3.1+, the fourth and fifth always fail.
-----------------actual output for 2.5--------------------
successfully changed <module '__main__' from 'docstring.py'>
successfully changed <function func at 0x00A8F570>
successfully changed <__main__.Test object at 0x00A94230>
Traceback (most recent call last):
File "docstring.py", line 25, in <module>
obj.__doc__ = 'new docstring'
AttributeError: attribute '__doc__' of 'instancemethod' objects is not
writable
()
Traceback (most recent call last):
File "docstring.py", line 25, in <module>
obj.__doc__ = 'new docstring'
AttributeError: attribute '__doc__' of 'type' objects is not writable
()
Traceback (most recent call last):
File "docstring.py", line 25, in <module>
obj.__doc__ = 'new docstring'
AttributeError: attribute '__doc__' of 'instancemethod' objects is not
writable
()
-----------------actual output for 3.2--------------------
successfully changed <module '__main__' from 'docstring.py'>
successfully changed <function func at 0x00BE6F60>
successfully changed <__main__.Test object at 0x00BFE730>
Traceback (most recent call last):
File "docstring.py", line 25, in <module>
obj.__doc__ = 'new docstring'
AttributeError: attribute '__doc__' of 'method' objects is not writable
Traceback (most recent call last):
File "docstring.py", line 25, in <module>
obj.__doc__ = 'new docstring'
AttributeError: attribute '__doc__' of 'type' objects is not writable
successfully changed <function meth at 0x00BE6ED0>
-----------------actual output----------------------------
~Ethan~
Ben Finney
> On Thu, 09 Jun 2011 17:44:32 +1000, Ben Finney wrote:
>
> > Eric Snow <ericsnow...@gmail.com> writes:
> >
> >> AttributeError: attribute '__doc__' of 'type' objects is not writable
> >>
> >> That is on 3.3.
> >
> > Well, that sucks :-(
> >
> > Where can we see the discussion of that change before it was
> > implemented?
>
> It goes back to Python 2.2, when new style classes were first introduced.
[…]
> It's an unnecessary restriction, as far as I'm concerned, but an old one.
Well, it's incompatible with the Python compiler I keep in my head. Have
these developers no consideration for backward-thinking-compatibility?
--
\ “The fact that I have no remedy for all the sorrows of the |
`\ world is no reason for my accepting yours. It simply supports |
_o__) the strong probability that yours is a fake.” —Henry L. Mencken |
Ben Finney
Gregory Ewing
around like this in the first place. Either it should happen
automatically, or help() should be smart enough to look up
the inheritance hierarchy when given a method that doesn't
have a docstring of its own.
Unfortunately, since unbound methods were ditched,
help(Foo.blarg) no longer has an easy way to find the base
classes, so help from the compiler may be needed.
--
Greg
Eric Snow
<greg....@canterbury.ac.nz> wrote:
> IMO, it shouldn't be necessary to explicitly copy docstrings
> around like this in the first place. Either it should happen
> automatically, or help() should be smart enough to look up
> the inheritance hierarchy when given a method that doesn't
> have a docstring of its own.
>
Auto inheriting docstrings would be nice, in some cases. WRT help(),
keep in mind that docstrings are used for a bunch of other things,
like doctests and some DSLs.
-eric
> Unfortunately, since unbound methods were ditched,
> help(Foo.blarg) no longer has an easy way to find the base
> classes, so help from the compiler may be needed.
>
> --
> Greg
Ben Finney
> IMO, it shouldn't be necessary to explicitly copy docstrings around
> like this in the first place. Either it should happen automatically,
> or help() should be smart enough to look up the inheritance hierarchy
> when given a method that doesn't have a docstring of its own.
Since the docstrings are useful in more places than just ‘help’, I'm +1
on having docstrings be automatically inherited if not specified.
Would the OP like to propose this on ‘python-ideas’?
--
\ “Odious ideas are not entitled to hide from criticism behind |
`\ the human shield of their believers' feelings.” —Richard |
_o__) Stallman |
Ben Finney
Eric Snow
> Gregory Ewing <greg....@canterbury.ac.nz> writes:
>
>> IMO, it shouldn't be necessary to explicitly copy docstrings around
>> like this in the first place. Either it should happen automatically,
>> or help() should be smart enough to look up the inheritance hierarchy
>> when given a method that doesn't have a docstring of its own.
>
> Since the docstrings are useful in more places than just ‘help’, I'm +1
> on having docstrings be automatically inherited if not specified.
>
> Would the OP like to propose this on ‘python-ideas’?
>
Yeah, I'll do that. Thanks.
-eric
> --
> \ “Odious ideas are not entitled to hide from criticism behind |
> `\ the human shield of their believers' feelings.” —Richard |
> _o__) Stallman |
> Ben Finney
Carl Banks
> IMO, it shouldn't be necessary to explicitly copy docstrings
> around like this in the first place. Either it should happen
> automatically, or help() should be smart enough to look up
> the inheritance hierarchy when given a method that doesn't
> have a docstring of its own.
Presumably, the reason you are overriding a method in a subclass is to change its behavior; I'd expect an inherited docstring to be inaccurate more often than not. So I'd be -1 on automatically inheriting them.
However, I'd be +1 easily on a little help from the language to explicitly request to inherit the docstring.
Carl Banks
Ben Finney
> Presumably, the reason you are overriding a method in a subclass is to
> change its behavior; I'd expect an inherited docstring to be
> inaccurate more often than not.
In which case the onus is on the programmer implementing different
behaviour to also override the docstring.
--
\ “When we pray to God we must be seeking nothing — nothing.” |
`\ —Saint Francis of Assisi |
_o__) |
Ben Finney
Eric Snow
> On Thursday, June 9, 2011 3:27:36 PM UTC-7, Gregory Ewing wrote:
>> IMO, it shouldn't be necessary to explicitly copy docstrings
>> around like this in the first place. Either it should happen
>> automatically, or help() should be smart enough to look up
>> the inheritance hierarchy when given a method that doesn't
>> have a docstring of its own.
>
> Presumably, the reason you are overriding a method in a subclass is to change its behavior; I'd expect an inherited docstring to be inaccurate more often than not. So I'd be -1 on automatically inheriting them.
>
When I write ABCs to capture an interface, I usually put the
documentation in the docstrings there. Then when I implement I want
to inherit the docstrings. Implicit docstring inheritance for
abstract base classes would meet my needs. I'm just not clear on the
impact this would have for the other use cases of docstrings.
> However, I'd be +1 easily on a little help from the language to explicitly request to inherit the docstring.
>
Yeah, that's more or less how I feel too. But what would fill that
role? This comes back to my original question. A method at
definition time does not know its class, nor would the decorator, so
they won't know where from to inherit the docstring.
Like I said originally, you can approach this a number of ways, but
the one that appeals to me most (plain function decorators) doesn't
work without some explicit help, which I would rather avoid. Implicit
help would be nice, but how to do it?
The most direct form, presenting the class to the execution frame of
the body somehow, seems risky and strange. It's sort of like the
function object being inserted into the locals when it is called.
However, the class object would have to be created before the body
gets exec'ed, rather than as now, where <metaclass>.__new__ is called
after... Changing that would require changes to type.__new__ and how
it's used.
Perhaps a good approach would be to have a special decorator in the
stdlib that type.__new__ would recognize, like this:
def inherits_docstring(f):
if f.__doc__ is None:
f.__doc__ = NotImplemented
return f
# actually in typeobject.c, or something
def type.__new__(meta, name, bases, namespace):
# do the normal stuff here
# now handle docstring inheritance
for name, obj in namespace.items():
if hasattr(obj, "__doc__") and obj.__doc__ is NotImplemented:
# inherit the docstring...
But then I look at that and wonder if it's too special-cased to be
worth the trouble. I can just use a metaclass or class decorator that
does that, and override builtin.__build__class__ to force its use
everywhere; or use one base class for all my classes that uses the
metaclass. But it would be nice to have implicit support.
-eric
>
> Carl Banks
> --
> http://mail.python.org/mailman/listinfo/python-list
>
Carl Banks
> Carl Banks
>
> > Presumably, the reason you are overriding a method in a subclass is to
> > change its behavior; I'd expect an inherited docstring to be
> > inaccurate more often than not.
>
> In which case the onus is on the programmer implementing different
> behaviour to also override the docstring.
Totally disagree. The programmer should never be under onus to correct mistakes made by the langauge. "In the face of ambiguity, refuse the temptation to guess."
When the language tries to guess what the programmer wants, you get monstrosities like Perl. Don't want to go there.
Carl Banks
Carl Banks
> When I write ABCs to capture an interface, I usually put the
> documentation in the docstrings there. Then when I implement I want
> to inherit the docstrings. Implicit docstring inheritance for
> abstract base classes would meet my needs.
Do all the subclasses do exactly the same thing? What's the use of a docstring if it doesn't document what the function does?
class Shape(object):
def draw(self):
"Draw a shape"
raise NotImplementedError
class Triangle(Shape):
def draw(self):
print "Triangle"
class Square(Shape):
def draw(self):
print "Square"
x = random.choice([Triange(),Square()])
print x.draw.__doc__ # prints "Draws a shape"
Quick, what shape is x.draw() going to draw? Shouldn't your docstring say what the method is going to do?
So, I'm sorry, but I don't see this being sufficient for your use case for ABCs.
> I'm just not clear on the
> impact this would have for the other use cases of docstrings.
Whenever somebody overrides a method to do something different, the inherited docstring will be insufficient (as in your ABC example) or wrong. This, I would say, is the case most of the time when overriding a base class method. When this happens, the language is committing an error.
Put it this way: if Python doesn't automatically inherit docstrings, the worst that can happen is missing information. If Python does inherit docstrings, it can lead to incorrect information.
Carl Banks
Terry Reedy
> Presumably, the reason you are overriding a method in a subclass is to change its behavior; I'd expect an inherited docstring to be inaccurate more often than not. So I'd be -1 on automatically inheriting them.
>
> However, I'd be +1 easily on a little help from the language to explicitly request to inherit the docstring.
An empty docstring "" could be interpreted as 'ditto' ;-)
It would be useless otherwise.
--
Terry Jan Reedy
Eric Snow
> On 6/9/2011 9:12 PM, Carl Banks wrote:
>
>> change its behavior; I'd expect an inherited docstring to be inaccurate more
>> often than not. So I'd be -1 on automatically inheriting them.
>>
>> However, I'd be +1 easily on a little help from the language to explicitly
>> request to inherit the docstring.
>
> It would be useless otherwise.
>
I kind of like that. The only catch is for cases out there where
someone used an empty string. Then it would change the behavior,
maybe. But how uncommon that is, not sure. I would guess pretty
uncommon.
Whole implicitly inherit idea would require the empty docstring to say
don't do it. With your idea you easily, clearly, and explicitly
indicate that you want the inheritance activated. That would work for
me.
-eric
> --
> Terry Jan Reedy
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
Ben Finney
> On Thursday, June 9, 2011 7:37:19 PM UTC-7, Eric Snow wrote:
> > When I write ABCs to capture an interface, I usually put the
> > documentation in the docstrings there. Then when I implement I want
> > to inherit the docstrings. Implicit docstring inheritance for
> > abstract base classes would meet my needs.
>
> Do all the subclasses do exactly the same thing? What's the use of a
> docstring if it doesn't document what the function does?
The docstring should document the object (module, class, or function) in
a way useful for the user of that API.
Differing implementations don't necessarily make for differing external
behaviour. In those cases where the external behaviour can be adequately
described by exactly the same docstring as the parent class's method,
it's tedious and error-prone to repeat or duplicate the docstring.
> class Shape(object):
> def draw(self):
> "Draw a shape"
> raise NotImplementedError
class Shape(object):
""" Abstract class for shapes. """
def draw(self):
""" Draw this shape. """
raise NotImplementedError
> class Triangle(Shape):
> def draw(self):
> print "Triangle"
class Triangle(Shape):
""" A three-sided polygon. """
def draw(self):
trace_three_sided_polygon()
> class Square(Shape):
> def draw(self):
> print "Square"
class Square(Shape):
""" An equal-sided quadrilateral polygon. """
def draw(self):
trace_quadrilateral_with_equal_sides()
> x = random.choice([Triange(),Square()])
> print x.draw.__doc__ # prints "Draws a shape"
x = random.choice([Triangle(), Square()])
print x.draw.__doc__ # => "Draw this shape."
> Quick, what shape is x.draw() going to draw?
print x.__doc__ # => " An equal-sided quadrilateral polygon. "
> Shouldn't your docstring say what the method is going to do?
There's nothing wrong with the docstring for a method referring to the
context within which the method is defined.
> Whenever somebody overrides a method to do something different, the
> inherited docstring will be insufficient (as in your ABC example) or
> wrong.
I hope the above demonstrates that your assertion is untrue. Every
single method on a class doesn't need to specify the full context; a
docstring that requires the reader to know what class the method belongs
to is fine.
--
\ “In any great organization it is far, far safer to be wrong |
`\ with the majority than to be right alone.” —John Kenneth |
_o__) Galbraith, 1989-07-28 |
Ben Finney
Ben Finney
> class Square(Shape):
> """ An equal-sided quadrilateral polygon. """
That this docstring is imprecise (it describes any rhombus, not
necessarily a square) is something I hope no-one else notices or draws
attention to.
Oh, darn.
--
\ “The sun never sets on the British Empire. But it rises every |
`\ morning. The sky must get awfully crowded.” —Steven Wright |
_o__) |
Ben Finney
Chris Angelico
> Ben Finney <ben+p...@benfinney.id.au> writes:
>
>> class Square(Shape):
>> """ An equal-sided quadrilateral polygon. """
>
> That this docstring is imprecise (it describes any rhombus, not
> necessarily a square) is something I hope no-one else notices or draws
> attention to.
class Square(Number):
""" A class designed to confuse the issue arbitrarily. """
pass
Chris Angelico
Gregory Ewing
> Presumably, the reason you are overriding a method in a subclass
> is to change its behavior;
Not always true by any means, and maybe not even usually true.
Consider overriding for the purpose of implementing an abstract
method, or because something about the internal operation of a
method needs to be modified to suit the requirements of the subclass.
I have a lot of situations like this in PyGUI, where there is a
bunch of generic classes defining the public API, and subclasses
of them for each implementation (Cocoa, Gtk and Windows). There
are heaps and heaps of overridden methods in the implementation
classes, and very few of them need or should have a docstring
different from the generic one. Not automatically inheriting
the docstrings puts a big burden on the maintainer to keep all
of them in sync.
--
Greg
Gregory Ewing
> x = random.choice([Triange(),Square()])
> print x.draw.__doc__ # prints "Draws a shape"
>
> Quick, what shape is x.draw() going to draw?
Your debugging code is insufficient. It should include
print type(x)
and then it will be obvious what shape is going to get
drawn.
--
Greg
Steven D'Aprano
> Steven D'Aprano <steve+comp....@pearwood.info> writes:
>> It's an unnecessary restriction, as far as I'm concerned, but an old
>> one.
>
> Well, it's incompatible with the Python compiler I keep in my head. Have
> these developers no consideration for backward-thinking-compatibility?
+1 QOTW
--
Steven
Steven D'Aprano
+1
Better than an decorator!
--
Steven
Steven D'Aprano
> x = random.choice([Triange(),Square()]) print x.draw.__doc__ # prints
> "Draws a shape"
>
>
> Quick, what shape is x.draw() going to draw?
That's easy... it will draw a type(x).__name__.
I think this not a terribly convincing argument. I don't particularly see
how it is very different (worse, or better) from what you can already get
in Python. If you don't know what x is, you might not know what it will
do.
>>> assert issubclass(ValueError, Exception)
>>> ValueError.__doc__
'Inappropriate argument value (of correct type).'
>>> Exception.__doc__
'Common base class for all non-exit exceptions.'
>>> from random import choice
>>> x = choice([ValueError, Exception])
Quick, what will x.__doc__ print?
> Shouldn't your docstring
> say what the method is going to do?
But it does say what the method does. It prints a shape, just like the
docstring says. It might not be a terribly detailed description, but that
counts as a quality of implementation issue, not a functional bug.
> So, I'm sorry, but I don't see this being sufficient for your use case
> for ABCs.
>
>
>> I'm just not clear on the
>> impact this would have for the other use cases of docstrings.
>
> Whenever somebody overrides a method to do something different, the
> inherited docstring will be insufficient (as in your ABC example) or
> wrong. This, I would say, is the case most of the time when overriding
> a base class method. When this happens, the language is committing an
> error.
It's hardly a *language* error if you, the developer, writes an
incomplete or incorrect docstring.
If you want to argue that the language shouldn't enable a failure mode of
the developer (namely the use of an incomplete or incorrect docstring),
well, perhaps you are right. But you are assuming that an inherited
docstring is necessarily wrong, which is not the case. "Prints a shape",
as in your above example, is a perfectly acceptable, if minimal,
docstring. It might not be a *great* docstring, but it's not a wrong one.
> Put it this way: if Python doesn't automatically inherit docstrings, the
> worst that can happen is missing information. If Python does inherit
> docstrings, it can lead to incorrect information.
This is no different from inheriting any other attribute. If your class
inherits "attribute", you might get an invalid value unless you take
steps to ensure it is a valid value. This failure mode doesn't cause us
to prohibit inheritance of attributes.
--
Steven
Tim Chase
> Sometimes when using class inheritance, I want the overriding methods
> of the subclass to get the docstring of the matching method in the
> base class. You can do this with decorators (after the class
> definition), with class decorators, and with metaclasses [1].
While asking for __doc__ ponies and picking colors for
bike-sheds, in a similar vein, I've occasionally wanted to do
something like
class Foo:
@property
def __doc__(self):
return dynamically_generated_string
# perhaps using introspection
This would have been most helpful in things like generating help
(like in command-line parsers), where the doc-string can
introspect the class and learn about its methods at runtime.
Some things seem to inherit, some don't, and help() doesn't seem
to pick up on any of the dynamically-defined __doc__ properties.
Test code below.
-tkc
from datetime import datetime
from sys import stdout
class Base(object):
"Base docstring"
@property
def __doc__(self):
return datetime.now().strftime('%c')
class WithDoc(Base):
"WithDoc docstring"
pass
class WithoutDoc(Base): pass
base = Base()
has = WithDoc()
lacks = WithoutDoc()
for test in (
"help(base)", # why not in help?
"help(has)", # expected
"help(lacks)", # why not in help?
"help(Base)",
"help(WithDoc)", # expected
"help(WithoutDoc)",
"stdout.write(repr(base.__doc__))", # works
"stdout.write(repr(has.__doc__))", # expected
"stdout.write(repr(lacks.__doc__))", # where'd it go?
"stdout.write(repr(Base.__doc__))", # expected
"stdout.write(repr(WithDoc.__doc__))", # expected
"stdout.write(repr(WithoutDoc.__doc__))", # what?
):
print test
eval(test)
print
Steven D'Aprano
> Sometimes when using class inheritance, I want the overriding methods of
> the subclass to get the docstring of the matching method in the base
> class. You can do this with decorators (after the class definition),
> with class decorators, and with metaclasses [1].
Here's some Python 3 code that uses a factory function as a metaclass to
inherit docstrings. Give the class a docstring of an empty string, and it
will be inherited from the first superclass found with a non-empty
docstring.
def InheritableDocstring(name, bases, dict):
mro = None
docstring = dict.get('__doc__')
if docstring == '':
# Search the MRO for the first non-empty docstring. We let Python
# do all the hard work of calculating the MRO.
mro = type('K', bases, {}).__mro__[1:] # Exclude the class K.
# Also exclude object.
assert mro[-1] == object
mro = mro[:-1]
for cls in mro:
if cls.__doc__:
docstring = cls.__doc__
break
else:
docstring = None
dict['__doc__'] = docstring
assert dict.get('__doc__') != ''
# Create the class we want, and return it.
cls = type(name, bases, dict)
if mro:
assert cls.__mro__ == (cls,) + mro + (object,)
return cls
class A(metaclass=InheritableDocstring):
pass
class B(A, metaclass=InheritableDocstring):
''
class C(B, metaclass=InheritableDocstring):
'A docstring.'
class D(B, metaclass=InheritableDocstring):
pass
class E(D, C, metaclass=InheritableDocstring):
''
class F(E, metaclass=InheritableDocstring):
''
assert all(cls.__doc__ is None for cls in (A, B, D))
assert all(cls.__doc__ == 'A docstring.' for cls in (C, E, F))
--
Steven
Eric Snow
<pytho...@tim.thechases.com> wrote:
> On 06/09/2011 01:22 AM, Eric Snow wrote:
>>
>> Sometimes when using class inheritance, I want the overriding methods
>> of the subclass to get the docstring of the matching method in the
>> base class. You can do this with decorators (after the class
>> definition), with class decorators, and with metaclasses [1].
>
> similar vein, I've occasionally wanted to do something like
>
> class Foo:
> @property
> def __doc__(self):
> return dynamically_generated_string
> # perhaps using introspection
>
However, on the class the property is just a property object (a
descriptor). If you want to have a dynamic doc on the class then you
have to do it on a metaclass:
class Meta(type):
@property
def __doc__(cls):
if not hasattr(cls, "_mydoc"):
cls._mydoc = None
return cls._mydoc
class X(metaclass=Meta): pass
X.__doc__
# None
X._mydoc
# None
X._mydoc = "test"
X.__doc__
# 'test'
X().__doc__
# None
The only problem, as seen in the last line, is that the __doc__ on
instances is not inherited on instances of the class. Object
attribute lookup only looks to the type's __dict__ for inheritance,
and not the types's type. However, that should not be that hard to
work around.
-eric
Eric Snow
<steve+comp....@pearwood.info> wrote:
> Here's some Python 3 code that uses a factory function as a metaclass to
> inherit docstrings. Give the class a docstring of an empty string, and it
> will be inherited from the first superclass found with a non-empty
> docstring.
>
>
Yeah, the idea of an empty docstring to trigger docstring inheritance
really appeals to me. Nice example. Incidently, aren't metaclasses
always inherited, as opposed to class decorators (which are never)?
-eric
Steven D'Aprano
> On Thu, 09 Jun 2011 00:22:54 -0600, Eric Snow wrote:
>
>> Sometimes when using class inheritance, I want the overriding methods
>> of the subclass to get the docstring of the matching method in the base
>> class. You can do this with decorators (after the class definition),
>> with class decorators, and with metaclasses [1].
>
>
> Here's some Python 3 code that uses a factory function as a metaclass to
> inherit docstrings. Give the class a docstring of an empty string, and
> it will be inherited from the first superclass found with a non-empty
> docstring.
[...]
And here's a version using a more conventional metaclass. Extending this
to work on methods is left as an exercise.
class MetaDocstring(type):
@staticmethod
def get_mro(bases):
return type('K', bases, {}).__mro__[1:-1]
@staticmethod
def get_docstring(mro):
for k in mro:
if k.__doc__:
return k.__doc__
def __new__(cls, name, bases, dict):
mro = None
docstring = dict.get('__doc__')
if docstring == '':
mro = cls.get_mro(bases)
dict['__doc__'] = cls.get_docstring(mro)
assert dict.get('__doc__') != ''
# Create the class we want, and return it.
K = super().__new__(cls, name, bases, dict)
if mro:
assert K.__mro__ == (K,) + mro + (object,)
return K
class U(metaclass=MetaDocstring):
pass
class V(U):
''
class W(V):
'A docstring.'
class X(V):
pass
class Y(X, W):
''
class Z(Y):
''
assert all(type(cls) is MetaDocstring for cls in (U, V, W, X, Y, Z))
assert all(cls.__doc__ is None for cls in (U, V, X))
assert all(cls.__doc__ == 'A docstring.' for cls in (W, Y, Z))
--
Steven
Ian Kelly
> The only problem, as seen in the last line, is that the __doc__ on
> instances is not inherited on instances of the class. Object
> attribute lookup only looks to the type's __dict__ for inheritance,
> and not the types's type. However, that should not be that hard to
> work around.
Everybody always focuses so much on properties and forgets that you
can also just write your own descriptors.
>>> class DocDescriptor(object):
... def __get__(self, instance, owner):
... return getattr(owner, "_mydoc", None)
...
>>> class Meta(type):
... def __init__(cls, name, bases, d):
... super(Meta, cls).__init__(name, bases, d)
... cls.__doc__ = DocDescriptor()
...
>>> class X(object):
... __metaclass__ = Meta
...
>>> X.__doc__
>>> X().__doc__
>>> X._mydoc = 'test'
>>> X.__doc__
'test'
>>> X().__doc__
'test'
>>> class Derived(X): pass
...
>>> Derived.__doc__
'test'
>>> Derived().__doc__
'test'
There you go, a metaclass that adds a __doc__ descriptor that can be
inherited (thanks to the metaclass), can be accessed from either the
class or the instance (thanks to the descriptor), and can easily be
modified to generate the doc string dynamically at call-time if
desired.
Cheers,
Ian
Steven D'Aprano
> On Fri, Jun 10, 2011 at 10:47 AM, Steven D'Aprano
> <steve+comp....@pearwood.info> wrote:
>> Here's some Python 3 code that uses a factory function as a metaclass
>> to inherit docstrings. Give the class a docstring of an empty string,
>> and it will be inherited from the first superclass found with a
>> non-empty docstring.
>>
>>
>>
> Yeah, the idea of an empty docstring to trigger docstring inheritance
> really appeals to me. Nice example. Incidently, aren't metaclasses
> always inherited, as opposed to class decorators (which are never)?
Metaclasses are inherited, but the example I give uses a factory function
as a metaclass: it manipulates the docstring inside the dict, then
returns an ordinary class. That makes it just a fancy class decorator
using metaclass syntax.
The type of each class A, B, ... F is just type, which means that when
you subclass each class you don't get any magic metaclass behaviour
unless you explicitly set the metaclass directly.
That is:
assert type(A) is type
succeeds, so class B(A) doesn't do anything special unless you explicitly
set the metaclass.
I followed up with a second example using a conventional metaclass, that
is, where the type of each class is *not* type. In that case, the magic
behaviour is inherited and there's no need to explicitly set the
metaclass except for the first time.
(Whew. Talking about metaclasses is hard work.)
--
Steven
Eric Snow
enough from the responses I've gotten here [1].
-eric
[1] http://mail.python.org/pipermail/python-ideas/2011-June/010473.html
Eric Snow
> Everybody always focuses so much on properties and forgets that you
> can also just write your own descriptors.
>
I'm so glad that you pointed this out. I totally forgot that
properties simply returned themselves if not called on the instance.
You are right about a custom descriptor.
-eric
Carl Banks
[snip example where programmer is expected to consult class docstring to infer what a method does]
> There's nothing wrong with the docstring for a method referring to the
> context within which the method is defined.
>
> > Whenever somebody overrides a method to do something different, the
> > inherited docstring will be insufficient (as in your ABC example) or
> > wrong.
>
> I hope the above demonstrates that your assertion is untrue. Every
> single method on a class doesn't need to specify the full context; a
> docstring that requires the reader to know what class the method belongs
> to is fine.
It does not. A docstring that requires the user to to figure out that is poor docstring.
There is nothing wrong, as you say, incomplete documentation that doesn't say what the function actually does. There's nothing wrong with omitting the docstring entirely for that matter. However, the question here is not whether a programmer is within right to use poor docstrings, but whether the langauge would go out of its way to support them. It should not.
There is one thing that is very wrong to do with a docstring: provide incorrect or misleading information. So, despite having brought the point up myself, I am going to say the point is moot. Even if it is absolutely desirable for a language to go out it's way to support incomplete docstrings, part of that bargain is that the language will go out of its way to support flat-out wrong docstrings, and that trumps any ostensible benefit.
Carl Banks
Carl Banks
Ridiculous. The docstring is an attribute of the function, not the class, which makes it very different from any other attribute. Consider this:
class A(object):
foo = SomeClass()
class B(A):
foo = SomeOtherUnrelatedClass()
Would you have B.foo "inherit" all the attributes of A.foo that it doesn't define itself? That's the analogous case to inheriting docstrings.
Carl Banks
Eric Snow
> Sometimes when using class inheritance, I want the overriding methods
> of the subclass to get the docstring of the matching method in the
> base class. You can do this with decorators (after the class
> definition), with class decorators, and with metaclasses [1].
>
> on the methods (no metaclass or class decorator). I am not sure if
> this is doable. I realize now that this is exactly the reason I got
> to thinking last week about objects being notified when they are bound
> [2].
>
> So, is there a way to do this with just decorators, or am I "stuck"
> with the metaclass/class decorator route? (It's not all that bad :)
>
Thanks for all the feedback on this thread everyone! I found a
solution that works pretty well, using descriptors:
class DocFunc:
TRIGGER = None
def __init__(self, f):
self.f = f
def __get__(self, obj, cls):
doc = self.f.__doc__
if doc == self.TRIGGER:
doc = self.get_doc(cls, self.f.__name__, self.TRIGGER)
self.f.__doc__ = doc
setattr(cls, self.f.__name__, self.f)
return self.f
@staticmethod
def get_doc(cls, fname, default=TRIGGER, member=True):
bases = cls.__mro__[:]
if member:
bases = bases[1:]
for base in bases:
func = getattr(base, fname, None)
if not func:
continue
doc = getattr(func, '__doc__', default)
if doc == default:
continue
return doc
return default
@staticmethod
def inherits_docstring(f, context=None, fname=None, default=TRIGGER):
if context is not None:
cls, namespace = context
fname = fname or f.__name__
f.__doc__ = DocFunc.get_doc(cls, fname, default, False)
return f
return DocFunc(f)
class X:
def something(self):
"""some method"""
class Y(X):
@DocFunc.inherits_docstring
def something(self):
...
This approach does not update the docstring if it changes in the base
class, but I don't need that for what I am doing. If you want to
trigger on an empty string, instead of None, just change the TRIGGER.
-eric
> Thanks!
>
> -eric
>
>
> p.s. Am I missing something or can you really not change the docstring
> of a class? I was thinking about the idea of inheriting class
> docstrings too.
>
>
> [1] http://code.activestate.com/recipes/577743-using-decorators-to-inherit-function-docstrings/
> [2] http://mail.python.org/pipermail/python-ideas/2011-June/010446.html
>
Steven D'Aprano
> On Friday, June 10, 2011 2:51:20 AM UTC-7, Steven D'Aprano wrote:
>> On Thu, 09 Jun 2011 20:36:53 -0700, Carl Banks wrote:
>> > Put it this way: if Python doesn't automatically inherit docstrings,
>> > the worst that can happen is missing information. If Python does
>> > inherit docstrings, it can lead to incorrect information.
>>
>> This is no different from inheriting any other attribute. If your class
>> inherits "attribute", you might get an invalid value unless you take
>> steps to ensure it is a valid value. This failure mode doesn't cause us
>> to prohibit inheritance of attributes.
>
> Ridiculous. The docstring is an attribute of the function, not the
> class, which makes it very different from any other attribute.
I don't know about you, but I'm talking about inheritance of both class
and method docstrings.
> Consider this:
>
>
> class A(object):
> foo = SomeClass()
>
>
> class B(A):
> foo = SomeOtherUnrelatedClass()
>
>
> Would you have B.foo "inherit" all the attributes of A.foo that it
> doesn't define itself?
If A.foo and B.foo are *unrelated*, they probably don't belong in
*related* classes. But putting that aside, if they truly are unrelated,
then no, of course you wouldn't inherit attributes of A.foo in B.foo,
including the docstring. That would be a stupid thing to do.
But why do you assume they are unrelated? Nobody is suggesting that (say)
str.len should inherit its doc string from dict.update. That would be
ridiculous, but not as ridiculous as assuming that's what we want to
happen!
If the classes, or methods, are related, chances are good that the
docstrings need to be related too. Possibly even identical. If they need
to be identical, then this proposal gives a way of enforcing that
identity without needing to keep two docstrings in sync manually.
Carl, I'm not exactly sure what your opposition is about here. Others
have already given real-world use cases for where inheriting docstrings
would be useful and valuable. Do you think that they are wrong? If so,
you should explain why their use-case is invalid and what solution they
should use.
If you fear that such docstring inheritance will become the default,
leading to a flood of inappropriate documentation, then I think we all
agree that this would be a bad thing.
But we can already "inherit" docstrings, in a manner of speaking, via an
explicit name binding step, and that hasn't lead to inappropriate
documentation:
def blarg1(*args):
"""Blarg the input and return a wibble."""
# implementation #1
def blarg2(*args):
# implementation #2
blag2.__doc__ = blag1.__doc__
# or perhaps blag1.__doc__.replace("wibble", "frob")
When you need to keep the docstrings of blag1 and blag2 in sync, it may
be better to "inherit" them rather than keep two independent strings that
need to be manually edited in sync.
functools.wraps() already does this. This proposal merely extends that
same idea to classes and methods via inheritance instead of explicit name
binding. Provided that such "inheritance" requires a deliberate choice by
the caller (a decorator, a metaclass, some other syntax), where's the
harm?
--
Steven
Carl Banks
> Carl, I'm not exactly sure what your opposition is about here. Others
> have already given real-world use cases for where inheriting docstrings
> would be useful and valuable. Do you think that they are wrong? If so,
> you should explain why their use-case is invalid and what solution they
> should use.
I don't have any issue with inheriting docstrings explicitly. Elsewhere in this thread I said I was +1 on the language helping to simplify this. What I am opposed to automatically inheriting the docstrings.
I do think people are overstating the uses where inherited methods would share the same docstring, but that's besides the point. Overstated or not, one cannot deny that the base method's docstring is frequently unacceptable for the derived method, and my opposition to automatic inheritance is because in those cases will lead to incorrect docstrings, and no other reason.
> If you fear that such docstring inheritance will become the default,
> leading to a flood of inappropriate documentation, then I think we all
> agree that this would be a bad thing.
That is exactly what I fear, and you are wrong that "we all agree that this would be a bad thing". Several people in this thread are arguing that inheriting docstrings by default is the right thing, and that would lead to heaps of inappropriate documentation.
Carl Banks