Class variable inheritance
Henry 'Pi' James
superclass until it's instantiated for the first time, but not any
more afterwards:
Python 3.1 (r31:73574, Jun 26 2009, 20:21:35) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class A:
... n = 0
... def __init__(self):
... type(self).n += 1
>>> class B(A):
... pass
>>> A.n, B.n
(0, 0)
>>> (A().n, A.n, B.n), (A().n, A.n, B.n), (B().n, A.n, B.n)
((1, 1, 1), (2, 2, 2), (3, 2, 3))
>>> (A().n, A.n, B.n), (A().n, A.n, B.n), (B().n, A.n, B.n)
((3, 3, 3), (4, 4, 3), (4, 4, 4))
>>> (A().n, A.n, B.n), (A().n, A.n, B.n), (B().n, A.n, B.n)
((5, 5, 4), (6, 6, 4), (5, 6, 5))
This makes no sense to me at all. Could it possibly be a bug?
Steven D'Aprano
> I've just found out that a subclass shares the class variables
String variables are strings.
Int variables are ints.
Float variables are floats.
List variables are lists.
Class variables are classes.
Classes are first-class objects in Python. Perhaps you mean class
attributes?
> of its
> superclass until it's instantiated for the first time, but not any more
> afterwards:
...
> This makes no sense to me at all. Could it possibly be a bug?
You have misinterpreted what you have seen, and if there's a bug, it's in
your code.
When you retrieve an attribute, Python first looks for instance
attributes, then class attributes, then attributes attached to
superclasses.
When you assign to an attribute, Python conceptually uses the exact same
"search path", except that instance assignment always succeeds.
(Well, almost -- but if it fails, you get an exception.)
So in practice:
x = obj.whatever
may return the contents of an instance attribute "whatever", a class
attribute, or an attribute of a superclass. But:
obj.whatever = x
always attempts to store x as an instance attribute, because there's no
way for Python to read your mind and know that you mean a class attribute
unless you say so explicitly. This attempt will either succeed, or it
will raise an exception. Python doesn't try writing further along the
hierarchy of instance/class/superclass(es).
If you wish to write to a class attribute, you have to explicitly say so:
obj.__class__.whatever = x
Your other misunderstanding relates to augmented assignment:
x += 1
does not modify x in place, it is *exactly* equivalent to:
x = x + 1
Given the rules of attribute access, obj.whatever += 1 is exactly
equivalent to:
obj.whatever = obj.whatever + 1
The right hand attribute access finds a class attribute, and the left
hand one sets an instance attribute.
--
Steven
Chris Rebert
> I've just found out that a subclass shares the class variables of its
> superclass until it's instantiated for the first time, but not any
> more afterwards:
>
> Python 3.1 (r31:73574, Jun 26 2009, 20:21:35) [MSC v.1500 32 bit
> (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> class A:
> ... n = 0
> ... def __init__(self):
> ... type(self).n += 1
>>>> class B(A):
> ... pass
> This makes no sense to me at all. Could it possibly be a bug?
Makes sense to me. To step through what's happening:
>>> A.n, B.n
(0, 0)
Here, the lookup on B.n fails (that is, B itself has no variable n),
and thus falls back to A.n
Thus, at this point, the expressions `A.n` and `B.n` are equivalent.
>>> (A().n, A.n, B.n)
(1, 1, 1)
A.__init__() gets called, incrementing A.n; again, B.n falls back to A.n
>>> (A().n, A.n, B.n)
(2, 2, 2),
<same thing>
>>> (B().n, A.n, B.n)
(3, 2, 3)
A.__init__() gets called since B did not define one of its own and
this inherited A's.
Therein, type(self) evaluates to B (not A as before).
B.n += 1 is in this case equivalent to:
B.n = B.n +1
Evaluating the right side, B.n falls back A.n once again, and we add 1.
Now the assignment takes place, creating a new variable B.n, *separate* from A.n
>From hereon in, lookups of B.n don't fall back to A.n since B now has
its own variable 'n'.
Thus, the values of A.n and B.n differ and the expressions now refer
to 2 distinct variables.
The rest falls out from this and is left as an exercise for the reader.
Cheers,
Chris
--
http://blog.rebertia.com
HPJ
>
> >>> A.n, B.n
> (0, 0)
>
> Here, the lookup on B.n fails (that is, B itself has no variable n),
> and thus falls back to A.n
See, this is what tripped me up, right at the beginning. I thought B
would inherit (as in copy) the variable n from A.
Can you point out to me which part (or parts) of the Language
Reference says this is the way it's supposed to be?
Terry Reedy
>> Makes sense to me. To step through what's happening:
>>
>>>>> A.n, B.n
>> (0, 0)
>>
>> Here, the lookup on B.n fails (that is, B itself has no variable n),
>> and thus falls back to A.n
>
> See, this is what tripped me up, right at the beginning. I thought B
> would inherit (as in copy) the variable n from A.
Python does not copy objects unless asked to.
Inheritance is a link request, not a copy request.
> Can you point out to me which part (or parts) of the Language
> Reference says this is the way it's supposed to be?
I could, but I will let you read and find what it says about class
attributes.
tjr
HPJ
> attributes.
You think I would have asked specifically about the Language Reference
if I hadn't read it and failed to find what I was looking for?
The closest thing I was able to find was section 3.2. "The standard
type hierarchy", subsection "Custom classes", where it says:
"When the attribute name is not found [in its namespace dictionary],
the attribute search continues in the base classes. This search of the
base classes uses the C3 method resolution order [...]"
That tells me how the lookup works, which I already knew. But it
doesn't tell me what happens when a class is inherited.
Now, one could argue if the attributes were copied, there would be no
need for a lookup in the base classes, so derivatively it says that's
not the case. To this I would say yes, there still would because after
they've been copied through inheritance, attributes can still be
removed via "del". Anyway, this passage, which is the only one I could
find, doesn't provide a direct and conclusive answer to my question.
Steven D'Aprano
>> Makes sense to me. To step through what's happening:
>>
>> >>> A.n, B.n
>> (0, 0)
>>
>> Here, the lookup on B.n fails (that is, B itself has no variable n),
>> and thus falls back to A.n
>
> See, this is what tripped me up, right at the beginning. I thought B
> would inherit (as in copy) the variable n from A.
Inherit does not mean copy. What makes you think it does? Given the
following:
class A(object):
def foo(self):
return "foo"
class B(A):
pass
would you expect the B class to have a copy of the foo method?
--
Steven
HPJ
Sorta. I would expect B to have a copy of the "foo" attribute, which
then refers to the same method as A.foo. So the method itself will be
be copied, but its address stored separately in A.foo and B.foo.
Carl Banks
No, I'm afraid not. Here is what happens. Conceptually, Python
checks for the presence of B.foo, and if it's not there it checks for
foo's presence in the base classes. (In actuality Python premaps
attributes to the approprirate base class, so only two dict lookups
are necessary.)
Python is a very dynamic langauge which allows you to modify class
objects at runtime. So if you inherit from a class, then later modify
that class, what should happen?
class A(object):
foo = 1
class B(A):
pass
A.foo = 2
print B.foo # what should this print, 1 or 2?
You could argue that copying class attributes (so that B.foo would be
1) is a reasonable way to do inheritance, but IMO the referencing
attributes in base classes reflects the underlying concept of
inheritance better.
Carl Banks
HPJ
> not there it checks for foo's presence in the base classes.
Yes, I have no problem believing you guys that this is what Python
does. Still, my question remains about where in the Language Reference
this is specified. And if the answer is nowhere, than the LR needs to
be amended, for obviously the way inheritance is done is no small
matter and its understanding should not be left to the user's own
intuition.
Mark Hammond
http://docs.python.org/reference/datamodel.html#new-style-and-classic-classes
- search for 'method resolution order' for other hits in that document.
HTH,
Mark
HPJ
> - search for 'method resolution order' for other hits in that document.
First of all, that's the LR for Python 2, I'm with Python 3. Second of
all, there's one single passage containing the phrase "method
resolution order" in the Python 3 LR, and it's one that I've already
quoted -- and discarded -- previously in this thread.
HPJ
because I have something like this:
class A:
class X:
n = 'a'
x = X()
class B(A):
class X(A.X):
n = 'b'
# x = X()
The line commented out was originally not there, but I found out I had
to add it if I wanted B().x.n to be 'b' instead of 'a'.
I might be wrong, but aren't there some (non-obscure) OOP language in
which the equivalent code (without the out-commented line) would have
made B().x an object of type B.X instead of A.X?
Steven D'Aprano
>> I could, but I will let you read and find what it says about class
>> attributes.
>
> You think I would have asked specifically about the Language Reference
> if I hadn't read it and failed to find what I was looking for?
You must be new to the Internet *wink*
Of course people ask without having made the effort themselves. "Please
sir, will you do my work for me?" is practically the norm on Internet
forums. Only without the please.
> The closest thing I was able to find was section 3.2. "The standard type
> hierarchy", subsection "Custom classes", where it says:
>
> "When the attribute name is not found [in its namespace dictionary], the
> attribute search continues in the base classes. This search of the base
> classes uses the C3 method resolution order [...]"
>
> That tells me how the lookup works, which I already knew. But it doesn't
> tell me what happens when a class is inherited.
The inheriting class has its base class set, and the MRO (method
resolution order), and that's pretty much it as far as I can tell.
Seems to me that the documentation is a little incomplete in this regard.
It looks to me like one of those "language lawyer" details that needs
fleshing out.
Out of curiosity, are there languages where inheritance means copy?
--
Steven
Carl Banks
Well, I don't know if this detail alone is all that grave an omission--
it'll matter to very few users--but it does seem the LRM could use a
subsection on inheritance in general. I'm sure the doc maintainers
would welcome a contribution.
Carl Banks
Carl Banks
> And by the way, the reason I've come across this problem at all is
> because I have something like this:
>
> class A:
> class X:
> n = 'a'
> x = X()
>
> class B(A):
> class X(A.X):
> n = 'b'
> # x = X()
You've nested classes here, that's a whole new layer of complexity.
I would strongly recommend against nesting classes in Python, unless
the inner class is a smallish class (such as a custom exception type)
that doesn't interact with the outer class. It's just that nested
classes don't ever seem to behave like anyone expects them to. Also
you can't take advantage of Python's lexical scoping with nested
classes (unlike, for instance, Java) so there really isn't much
benefit to nesting them.
> The line commented out was originally not there, but I found out I had
> to add it if I wanted B().x.n to be 'b' instead of 'a'.
Hm, even if class B did get copies of class A's attributes, B().x.n
would still return 'a'. It seems as if you expect class B to re-
execute A's statements in B's context, or something. That's not how
it works at all.
(Now if you define self.x=X() inside of __init__, that's a different
story.)
> I might be wrong, but aren't there some (non-obscure) OOP language in
> which the equivalent code (without the out-commented line) would have
> made B().x an object of type B.X instead of A.X?
Maybe. For some languages this might be an obvious behavior, but
Python would have different circumstances so it isn't so obvious (or
possible) there.
Carl Banks
Chris Rebert
D'Aprano<ste...@remove.this.cybersource.com.au> wrote:
<snip>
> Out of curiosity, are there languages where inheritance means copy?
I think some prototype-based ones handle it that way...
HPJ
> Python would have different circumstances so it isn't so obvious (or
> possible) there.
Which means this topic definitely needs to be handled in detail by the
LR, and preferably also in the Tutorial. Otherwise there is no way
anyone coming to Python could know which of the many ways of
inheritance Python follows.
Carl Banks
> > Maybe. For some languages this might be an obvious behavior, but
> > Python would have different circumstances so it isn't so obvious (or
> > possible) there.
>
> Which means this topic definitely needs to be handled in detail by the
> LR, and preferably also in the Tutorial.
As I said, I'm sure the doc maintainers would likey welcome your
contributions.
> Otherwise there is no way
> anyone coming to Python could know which of the many ways of
> inheritance Python follows.
"Mo way"? You are definitely overstating things here.
Carl Banks
Lie Ryan
Perhaps in a language where classes attribute are declarative. Not in
python where class attributes are executable statements.
Note that when the python interpreter meets this statement:
class B(P):
def foo(self):
print('ab')
X = 'f'
the compiler sees a class statement -> create a new blank class
-> assign P as the new class' parent
-> *execute* the class' body
# new context
the compiler sees a def statement -> *compile* the def's body
-> assign the compiled body to B.foo
the compiler sees X = 'f' statement -> assign 'f' to B.X
# exit the new context
-> assign the new class to B
Your problem is related to how the interpreter *execute* a class
definition rather than the name resolution.
Terry Reedy
>
> Note that when the python interpreter meets this statement:
>
> class B(P):
> def foo(self):
> print('ab')
> X = 'f'
>
> the compiler sees a class statement -> create a new blank class
> -> assign P as the new class' parent
No, it saves the name 'B' and bases tuple P, and create a new *dict*,
call it d here though it is anonymous as far as the class body is concerned.
> -> *execute* the class' body
with the new dict d as the local namespace. In other words, the
equivalent of
exec('''body''', globals(), {}) # new Py3 exec() form
> # new context
> the compiler sees a def statement -> *compile* the def's body
to a code object which is then attached to a function object
> -> assign the compiled body to B.foo
No, d['foo'] = <function object>
> the compiler sees X = 'f' statement -> assign 'f' to B.X
No, d['X'] = 'f'
> # exit the new context
Up to this point, there is no new class object.
> -> assign the new class to B
It calls meta(name, bases, d), where meta is the metaclass 'type' by
default but can be any callable which does anything, though the
intention is that it be a subclass of type or at least something that
creates a class object, and that d become the backstage attribute dict
for the result.
> Your problem is related to how the interpreter *execute* a class
> definition rather than the name resolution.
Terry Jan Reedy
Lie Ryan
> Lie Ryan wrote:
>
>>
>> Note that when the python interpreter meets this statement:
>>
>> class B(P):
>> def foo(self):
>> print('ab')
>> X = 'f'
>>
>> the compiler sees a class statement -> create a new blank class
>> -> assign P as the new class' parent
>
> No, it saves the name 'B' and bases tuple P, and create a new *dict*,
> call it d here though it is anonymous as far as the class body is
> concerned.
Neat, I'd never thought that it creates the ".__dict__" before the class
itself.
Duncan Booth
It has to be that way: some of the internal methods cannot be modified
after the initial creation of the class, so you need to use a namespace
that exists before the class itself exists.
The situation changes slightly in Python 3 where the metaclass can hook
into the creation of the dict and instead create any kind of object which
exposes a dict-like interface. That opens the door to classes where the
order in which the order attributes are defined is significant or even
where you can give multiple definitions for the same attribute (e.g. to
support overloaded methods).
--
Duncan Booth http://kupuguy.blogspot.com
Terry Reedy
The documentation for this, with an example, is in RefMan 3.3.3,
Customizing Class Creation. See the metaclass .__prepare__ method.