data attributes override method attributes?

[Jayden]

Dear All,

In the Python Tutorial, Section 9.4, it is said that

"Data attributes override method attributes with the same name."

But in my testing as follows:

#Begin
class A:
i = 10
def i():
print 'i'

A.i
<unbound method A.i>
#End

I think A.i should be the number 10 but it is the method. There must be something I misunderstand. Would you please tell me why?

Thanks,

Jayden

[alex23]

What the tutorial is referring to is this, I think:

class A(object):
def i(self):
print 'i'

>>> a = A()
>>> a.i = 10
>>> a.i
10

That is, you can create a data attribute on an object even if a method
attribute of the same name exists.

[Peter Otten]

Jayden wrote:

> In the Python Tutorial, Section 9.4, it is said that
>
> "Data attributes override method attributes with the same name."

The tutorial is wrong here. That should be

"Instance attributes override class attributes with the same name."

As methods are usually defined in the class and data attributes are usually
set in the instance it will look like data override method attributes.
What the author had in mind:

>>> class A:
... def i(self): print "method"
...
>>>
>>> a = A()
>>> a.i()
method
>>> a.i = 42 # this could also happen in a method with self.i = 42
>>> a.i()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
>>> a.i
42

> But in my testing as follows:
>
> #Begin
> class A:
> i = 10
> def i():
> print 'i'
>
> A.i
> <unbound method A.i>
> #End

but

class A:
def i(self): print "i"
i = 42

print A().i # 42

If two objects are assigned to the same name the last assignment always
wins.

> I think A.i should be the number 10 but it is the method. There must be
something I misunderstand. Would you please tell me why?

No, you're right. Please file a bug report at http://bugs.python.org

[alex23]

On Sep 26, 12:08 am, Peter Otten <__pete...@web.de> wrote:
> Jayden wrote:
> > In the Python Tutorial, Section 9.4, it is said that
>
> > "Data attributes override method attributes with the same name."
>
> The tutorial is wrong here. That should be
>
> "Instance attributes override class attributes with the same name."
>
> As methods are usually defined in the class and data attributes are usually
> set in the instance it will look like data override method attributes.

But you can assign attributes on the class, which has the same impact,
so the tutorial is correct.

> No, you're right. Please file a bug report athttp://bugs.python.org

Didn't you just demonstrate the behaviour you're now saying is a bug?

[Steven D'Aprano]

When you create the class, two things happen: first you define a class-
level attribute i, then you define a method i. Since you can only have a
single object with the same name in the same place, the method replaces
the attribute.

In this case, classes and methods are irrelevant. It is exactly the same
behaviour as this:

i = 10
i = 20
# i now equals 20, not 10


except that instead of 20, you use a function object:

i = 10
def i():

return "something"
# i is now a function object, not 10


What the manual refers to is the use of attributes on an instance:

py> class Test(object):
... def f(self):
... return "something"
...
py> t = Test()
py> t.f = 20
py> t.f
20

In this case, there is an attribute called "f" (a method) which lives in
the class and is shared by all instances, and another attribute called
"f" which lives in the instance t, is not shared, and has the value 20.
This instance attribute masks the method with the same name. We can see
that it is only hidden, not gone, by creating a new instance:

py> u = Test()
py> u.f
<bound method Test.f of <__main__.Test object at 0x871390c>>



--
Steven

[Peter Otten]

To me

"Data attributes override method attributes with the same name"

implies that data attributes take precedence over method attributes, not
that they replace them only when there is an assignment of data after the
method definition.

With your interpretation (if I understand you correctly)

"Method attributes override data attributes with the same name"

is equally correct, and therefore I think it is misleading to focus on the
type of the attributes at all.

I would even consider replacing the whole paragraph

"""
Data attributes override method attributes with the same name; to avoid
accidental name conflicts, which may cause hard-to-find bugs in large
programs, it is wise to use some kind of convention that minimizes the
chance of conflicts. Possible conventions include capitalizing method names,
prefixing data attribute names with a small unique string (perhaps just an
underscore), or using verbs for methods and nouns for data attributes.
"""
http://docs.python.org/dev/py3k/tutorial/classes.html

with something like

"Data attributes and method attributes share the same namespace. To avoid
name conflicts consider using verbs for methods and nouns for data
attributes"

[Chris Angelico]

On Wed, Sep 26, 2012 at 12:54 AM, Peter Otten <__pet...@web.de> wrote:
> To me
>
> "Data attributes override method attributes with the same name"
>
> implies that data attributes take precedence over method attributes, not
> that they replace them only when there is an assignment of data after the
> method definition.
>

> I would even consider replacing the whole paragraph

> with something like
>
> "Data attributes and method attributes share the same namespace. To avoid
> name conflicts consider using verbs for methods and nouns for data
> attributes"

Instance attributes override (shadow) class attributes. Since methods
tend to be on the class and data tends to be on the instance, the
original sentence does make some sense. The section is talking about
conventions, so it's not inherently wrong, but perhaps just needs a
comment about methods not usually being attached to the instance.

ChrisA

[Ulrich Eckhardt]

Am 25.09.2012 16:11, schrieb alex23:
> On Sep 26, 12:08 am, Peter Otten <__pete...@web.de> wrote:
>> Jayden wrote:
>>> In the Python Tutorial, Section 9.4, it is said that
>>
>>> "Data attributes override method attributes with the same name."
>>
>> The tutorial is wrong here. That should be
>>
>> "Instance attributes override class attributes with the same name."
>>
>> As methods are usually defined in the class and data attributes are usually
>> set in the instance it will look like data override method attributes.
>
> But you can assign attributes on the class, which has the same impact,
> so the tutorial is correct.

You can assign attributes of the class or the instance, and you can
assign with functions or data (actually, both functions and data are
objects, Python doesn't make a distinction there). The important thing
is that lookup first looks in the instance (where data attributes are
usually set) before looking in the class (where method attributes are
usually set). Observing typical use and deriving a rule from this is
misleading though.

>> No, you're right. Please file a bug report athttp://bugs.python.org
>
> Didn't you just demonstrate the behaviour you're now saying is a bug?

I think he meant a bug in the tutorial, not in the implementation of Python.


Uli

[Terry Reedy]

On 9/25/2012 11:03 AM, Chris Angelico wrote:
> On Wed, Sep 26, 2012 at 12:54 AM, Peter Otten <__pet...@web.de> wrote:
>> To me
>>
>> "Data attributes override method attributes with the same name"
>>
>> implies that data attributes take precedence over method attributes, not
>> that they replace them only when there is an assignment of data after the
>> method definition.
>>
>> I would even consider replacing the whole paragraph
>> with something like
>>
>> "Data attributes and method attributes share the same namespace. To avoid
>> name conflicts consider using verbs for methods and nouns for data
>> attributes"
>
> Instance attributes override (shadow) class attributes.

except for (some? all?) special methods

> Since methods
> tend to be on the class and data tends to be on the instance, the
> original sentence does make some sense.

but it *is* wrong

The section is talking about
> conventions, so it's not inherently wrong,

The suggestion to Capitalize method names and prefix data names with '_'
are wrong with respect to the style guide.

but perhaps just needs a
> comment about methods not usually being attached to the instance.
>
> ChrisA
>

--
Terry Jan Reedy

[Thomas Rachel]

Am 25.09.2012 16:08 schrieb Peter Otten:
> Jayden wrote:
>
>> In the Python Tutorial, Section 9.4, it is said that
>>
>> "Data attributes override method attributes with the same name."
>
> The tutorial is wrong here. That should be
>
> "Instance attributes override class attributes with the same name."

I jump in here:

THere is one point to consider: if you work with descriptors, it makes a
difference if they are "data descriptors" (define __set__ and/or
__delete__) or "non-data descriptors" (define neither).

As http://docs.python.org/reference/datamodel.html#invoking-descriptors
tells us, methods are non-data descriptors, so they can be overridden by
instances.

OTOH, properties are data descriptors which cannot be overridden by the
instance.

So, to stick to the original example:

class TestDesc(object):
def a(self): pass
@property
def b(self): print "trying to get value - return None"; return None
@b.setter
def b(self, v): print "value", v, "ignored."
@b.deleter
def b(self): print "delete called and ignored"


and now

>>> t=TestDesc()
>>> t.a
<bound method TestDesc.a of <__main__.TestDesc object at 0xb7387ccc>>
>>> t.b
trying to get value - return None
>>> t.a=12
>>> t.b=12
value 12 ignored.
>>> t.a
12
>>> t.b
trying to get value - return None
>>> del t.a
>>> del t.b
delete called and ignored
>>> t.a
<bound method TestDesc.a of <__main__.TestDesc object at 0xb7387ccc>>
>>> t.b
trying to get value - return None


Thomas

[Ian Kelly]

On Tue, Sep 25, 2012 at 1:58 PM, Terry Reedy <tjr...@udel.edu> wrote:
> On 9/25/2012 11:03 AM, Chris Angelico wrote:
>> Instance attributes override (shadow) class attributes.
>
>
> except for (some? all?) special methods

Those names are shadowed too. If you call foo.__len__() and the name
is bound on the instance, it will call that function preferentially.
It's just that when the special Python machinery calls the method, it
skips the instance and goes straight to the class.

[Terry Reedy]

On 9/25/2012 10:54 AM, Peter Otten wrote:

> alex23 wrote:
>
>> On Sep 26, 12:08 am, Peter Otten <__pete...@web.de> wrote:
>>> Jayden wrote:
>>>> In the Python Tutorial, Section 9.4, it is said that
>>>
>>>> "Data attributes override method attributes with the same name."
>>>
>>> The tutorial is wrong here. That should be
>>>
>>> "Instance attributes override class attributes with the same name."

Except for special methods.

> I would even consider replacing the whole paragraph

I agree

>
> """
> Data attributes override method attributes with the same name; to avoid
> accidental name conflicts, which may cause hard-to-find bugs in large
> programs, it is wise to use some kind of convention that minimizes the
> chance of conflicts. Possible conventions include capitalizing method names,
> prefixing data attribute names with a small unique string (perhaps just an

> underscore), or using verbs for methods and nouns for data attributes.
> """
> http://docs.python.org/dev/py3k/tutorial/classes.html

>
> with something like
>
> "Data attributes and method attributes share the same namespace.

and instance attributes usually override class attributes

> To avoid
> name conflicts consider using verbs for methods and nouns for data
> attributes"

This applies within and between. I opened

http://bugs.python.org/issue16048

--
Terry Jan Reedy

[Terry Reedy]

I added "Ian Kelly reminds me that instance.__xxx__ is only skipped by
the internal machinery and not by direct accesses in user code. In the
other hand, docs, official or otherwise, are filled with things like
'len(a) calls a.__len__', so I think something should be said that
giving instances special method attributes does not have the effect one
might expect."

to the issue.


--
Terry Jan Reedy

[Prasad, Ramit]

Just to make sure I am following, if you call
foo.__len__() it goes to the instance code while
if you do len(foo) it will go to class.__len__()?

If so, why?

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.

[Ian Kelly]

On Fri, Sep 28, 2012 at 12:02 PM, Prasad, Ramit
<ramit....@jpmorgan.com> wrote:
> Just to make sure I am following, if you call
> foo.__len__() it goes to the instance code while
> if you do len(foo) it will go to class.__len__()?

Yes:

>>> class Foo(object):
... def __len__(self):
... return 42
...
>>> foo = Foo()
>>> foo.__len__ = lambda: 43
>>> foo.__len__()
43
>>> len(foo)
42

> If so, why?

In the first case, "foo.__len__" just does the normal attribute lookup
for the class. Instance attributes shadow class attributes, so the
instance attribute is returned and then called.

In the second case, "len(foo)" is implemented by a method in a
prescribed location: foo.__class__.__len__. It only looks in the
class for efficiency and because that is what the class object is for:
to define how its instances behave.

[Terry Reedy]

On 9/28/2012 2:02 PM, Prasad, Ramit wrote:

> Just to make sure I am following, if you call
> foo.__len__() it goes to the instance code while
> if you do len(foo) it will go to class.__len__()?

len(foo) calls someclass.__len__(foo) where someclass is foo.__class__or
some superclass.

> If so, why?

Efficiency and perhaps simpler implementation, especially for binary ops.

--
Terry Jan Reedy

[Steven D'Aprano]

On Fri, 28 Sep 2012 18:02:04 +0000, Prasad, Ramit wrote:

> Just to make sure I am following, if you call foo.__len__() it goes to
> the instance code while if you do len(foo) it will go to
> class.__len__()?

If you call foo.__len__, the attribute lookup of __len__ will use the
exact same search path as foo.spam would use:

1) does __getattribute__ exist and intercept the call?
2) if not, does a instance attribute exist?
3) if not, does a class attribute exist?
4) if not, does a superclass attribute exist?
5) if not, does __getattr__ exist and intercept the call?

Using len(foo) bypasses steps 1) and 2) as a speed optimization. For the
common case where an instance's class directly defines a __len__ method,
that saves about 10-15% of the overhead of calling a special method,
compared to old-style classes.



--
Steven