Inconsistent behavior of descriptors
Denis S. Otkidach
I've noticed that the order of attribute lookup is inconsistent
when descriptor is used. property instance takes precedence of
instance attributes:
>>> class A(object):
... def _get_attr(self):
... return self._attr
... attr = property(_get_attr)
...
>>> a=A()
>>> a.__dict__
{}
>>> a.__dict__['attr']=1
>>> a.__dict__
{'attr': 1}
>>> a.attr
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 3, in _get_attr
AttributeError: 'A' object has no attribute '_attr'
But it doesn't when I use custom class of descriptor:
>>> class descr(object):
... def __get__(self, inst, cls):
... return inst._attr
...
>>> class B(object):
... attr = descr()
...
>>> b=B()
>>> b.__dict__
{}
>>> b.__dict__['attr']=1
>>> b.__dict__
{'attr': 1}
>>> b.attr
1
Subclasses of property behave like property itself:
>>> class descr2(property):
... def __get__(self, inst, cls):
... return inst._attr
...
>>> class C(object):
... attr = descr2()
...
>>> c=C()
>>> c.__dict__
{}
>>> c.__dict__['attr']=1
>>> c.__dict__
{'attr': 1}
>>> c.attr
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 3, in __get__
AttributeError: 'C' object has no attribute '_attr'
Is it an undocumented feature or I have to submit a bug report?
--
Denis S. Otkidach
http://www.python.ru/ [ru]
John Roth
"Denis S. Otkidach" <o...@strana.ru> wrote in message
news:mailman.1064938610...@python.org...
I'm not entirely sure what you're describing, however, I
notice that your descriptor object has a __get__ method,
but no __set__ or __del__ methods. The lookup chain
is quite different for descriptors with only __get__ methods
than it is for descriptors for all three.
In particular, it's intended that instance attributes should
be ignored if there is a data descriptor (that is, a descriptor
with both__get__ and __set__ methods.)
I'm not sure if this applies to your problem.
John Roth
Michael Chermside
> I've noticed that the order of attribute lookup is inconsistent
> when descriptor is used. property instance takes precedence of
> instance attributes:
> But it doesn't when I use custom class of descriptor:
> Subclasses of property behave like property itself:
> Is it an undocumented feature or I have to submit a bug report?
Looks like a bug to me.
-- Michael Chermside
Denis S. Otkidach
JR> > I've noticed that the order of attribute lookup is
JR> inconsistent
JR> > when descriptor is used. property instance takes
JR> precedence of
JR> > instance attributes:
[...]
JR> > But it doesn't when I use custom class of descriptor:
[...]
JR> I'm not entirely sure what you're describing, however, I
JR> notice that your descriptor object has a __get__ method,
JR> but no __set__ or __del__ methods. The lookup chain
JR> is quite different for descriptors with only __get__ methods
JR> than it is for descriptors for all three.
JR>
JR> In particular, it's intended that instance attributes should
JR> be ignored if there is a data descriptor (that is, a
JR> descriptor
JR> with both__get__ and __set__ methods.)
Is it documented somewhere? What is the reason for such
inconsistent behavior?
John Roth
The behavior of descriptors is documented in the article on
the python web site. You might have to hunt a bit to find it.
The reason for the inconsistency is simply that descriptors
with only a __get__ method are used for methods, functions
and similar entities, while descriptors with both __get__ and
__put__ are used for properties and similar entities.
It's useful to be able to put a unique version of a function
or method in an instance, it would completely subvert the
meaning of a property if you could do so.
John Roth
Alex Martelli
...
> The behavior of descriptors is documented in the article on
> the python web site. You might have to hunt a bit to find it.
Actually the best doc for it (IMHO) is OFF the python site, at
http://users.rcn.com/python/download/Descriptor.htm .
Alex
Michael Hudson
> On Tue, 30 Sep 2003, John Roth wrote:
>
> JR> In particular, it's intended that instance attributes should
> JR> be ignored if there is a data descriptor (that is, a
> JR> descriptor
> JR> with both__get__ and __set__ methods.)
>
> Is it documented somewhere?
http://www.python.org/doc/current/ref/descriptor-invocation.html
Cheers,
mwh
--
Python enjoys making tradeoffs that drive *someone* crazy <wink>.
-- Tim Peters, comp.lang.python
Bengt Richter
I think (not having read the above carefully) that it's all documented in
http://users.rcn.com/python/download/Descriptor.htm
(thanks to Raymond Hettinger).
excerpt:
"""
The details of invocation depend on whether obj is an object or a class.
Either way, descriptors only work for new style objects and classes. A
class is new style if it is a subclass of object.
For objects, the machinery is in object.__getattribute__ which
transforms b.x into type(b).__dict__['x'].__get__(b, type(b)). The
implementation works through a precedence chain that gives data
descriptors priority over instance variables, instance variables
priority over non-data descriptors, and assigns lowest priority to
__getattr__ if provided. The full C implementation can be found in
PyObject_GenericGetAttr() in Objects/object.c.
For classes, the machinery is in type.__getattribute__ which transforms
B.x into B.__dict__['x'].__get__(None, B). In pure Python, it looks
like:
def __getattribute__(self, key):
"Emulate type_getattro() in Objects/typeobject.c"
v = object.__getattribute__(self, key)
if hasattr(v, '__get__'):
return v.__get__(None, self)
return v
The important points to remember are:
descriptors are invoked by the __getattribute__ method
overriding __getattribute__ prevents automatic descriptor calls
__getattribute__ is only available with new style classes and objects
object.__getattribute__ and type.__getattribute__ make different calls to __get__.
data descriptors always override instance dictionaries.
non-data descriptors may be overridden by instance dictionaries.
"""
Regards,
Bengt Richter
Denis S. Otkidach
JR> The behavior of descriptors is documented in the article on
JR> the python web site. You might have to hunt a bit to find
JR> it.
http://users.rcn.com/python/download/Descriptor.htm
"The implementation works through a precedence chain that gives
data descriptors priority over instance variables, instance
variables priority over non-data descriptors, and assigns lowest
priority to __getattr__ if provided."
I can't see any reason why "data descriptors" have priority over
instance variables anyway.
JR> The reason for the inconsistency is simply that descriptors
JR> with only a __get__ method are used for methods, functions
JR> and similar entities, while descriptors with both __get__
JR> and
JR> __put__ are used for properties and similar entities.
There are many examples when data (in common sense) descriptors
are used with the only __get__ method defined, i.e. non-data
descriptors in your terms. Here are two I use most frequently:
class CachedAttribute(object):
def __init__(self, method, name=None):
self.method = method
self.name = name or method.__name__
def __get__(self, inst, cls):
# Some tricks here to walk around bug in Python 2.2 are
# skiped
result = self.method(inst)
setattr(inst, self.name, result)
return result
class ReadAliasAttribute(object):
def __init__(self, name):
self.name = name
def __get__(self, inst, cls):
return getattr(inst, self.name)
JR> It's useful to be able to put a unique version of a function
JR> or method in an instance, it would completely subvert the
JR> meaning of a property if you could do so.
Why? You can't do it directly anyway if __set__ method is
defined.
John Roth
"Alex Martelli" <al...@aleax.it> wrote in message
news:61zeb.148593$hE5.5...@news1.tin.it...
That's the one I meant. It's referenced somewhere in the
Python site as well.
It's a very good piece of doc, it needs to get into the official
doc some time real soon.
John Roth
>
>
> Alex
>
John Roth
"Denis S. Otkidach" <o...@strana.ru> wrote in message
news:mailman.106500978...@python.org...
***********************************************
***********************************************
But that's ***WHY*** you can't do it if the __set__ method
is defined.
***********************************************
John Roth
Alex Martelli
>
> "Alex Martelli" <al...@aleax.it> wrote in message
> news:61zeb.148593$hE5.5...@news1.tin.it...
>> John Roth wrote:
>> ...
>> > The behavior of descriptors is documented in the article on
>> > the python web site. You might have to hunt a bit to find it.
>>
>> Actually the best doc for it (IMHO) is OFF the python site, at
>> http://users.rcn.com/python/download/Descriptor.htm .
>
> That's the one I meant. It's referenced somewhere in the
> Python site as well.
Oh, I thought you meant http://www.python.org/2.2.2/descrintro.html
(which, as its pagename reveals, IS mostly about descriptors,
though the title is about "unifying types and classes"...:-).
> It's a very good piece of doc, it needs to get into the official
> doc some time real soon.
Seconded. It remains to be seen if Fred Drake, the official doc's
supremo, agrees;-).
Alex
Michael Chermside
> I've noticed that the order of attribute lookup is inconsistent
> when descriptor is used. property instance takes precedence of
> instance attributes:
> But it doesn't when I use custom class of descriptor:
> Subclasses of property behave like property itself:
> Is it an undocumented feature or I have to submit a bug report?
Michael Chermside writes:
> Looks like a bug to me.
John Roth writes:
> I'm not entirely sure what you're describing, however, I
> notice that your descriptor object has a __get__ method,
> but no __set__ or __del__ methods. The lookup chain
> is quite different for descriptors with only __get__ methods
> than it is for descriptors for all three.
>
> In particular, it's intended that instance attributes should
> be ignored if there is a data descriptor (that is, a descriptor
> with both__get__ and __set__ methods.)
Doh! Of course! John is right of course.
-- Michael Chermside