Lost in descriptor land

[Ankush Thakur]

Hello,

There's something I don't understand about descriptors. On a StackOverflow discussion (http://stackoverflow.com/questions/12846116/python-descriptor-vs-property) one of the answers provides the following descriptors example:

class Celsius( object ):
def __init__( self, value=0.0 ):
self.value= float(value)
def __get__( self, instance, owner ):
return self.value
def __set__( self, instance, value ):
self.value= float(value)

class Temperature( object ):
celsius= Celsius()
farenheit= Farenheit()

... and then gets chided in the comments:

"I believe your descriptor implementation of celsius is not correct. You should have set the celsius on instance rather than self; If you create two Temperature objects they will share the same celsius value."

Overall, I have two problems:
1) I don't get the idea behind the 'instance' and 'owner' parameters at all. Is there some simple tutorial that can explain these?
2) I don't understand the motivation behind the comment. Of course declaring a class variable would cause celcius to be the same for all objects. Shouldn't we be instead using self.celcius in, say, __init__() and then everything will work fine?

I have seen examples (http://programeveryday.com/post/an-introduction-to-python-descriptors/) where "instance" is used as keys of a dictionary, but given my argument above, isn't this approach an overkill?

Regards,
Ankush Thakur

[Ian Kelly]

On Thu, Jun 30, 2016 at 7:06 PM, Ankush Thakur
<ankush....@gmail.com> wrote:
> Hello,
>
> There's something I don't understand about descriptors. On a StackOverflow discussion (http://stackoverflow.com/questions/12846116/python-descriptor-vs-property) one of the answers provides the following descriptors example:
>
> class Celsius( object ):
> def __init__( self, value=0.0 ):
> self.value= float(value)
> def __get__( self, instance, owner ):
> return self.value
> def __set__( self, instance, value ):
> self.value= float(value)
>
> class Temperature( object ):
> celsius= Celsius()
> farenheit= Farenheit()
>
> ... and then gets chided in the comments:
>
> "I believe your descriptor implementation of celsius is not correct. You should have set the celsius on instance rather than self; If you create two Temperature objects they will share the same celsius value."

That criticism is accurate. "self" above refers to the Celsius
instance, and there is only one Celsius instance for the Temperature
class, not one per Temperature instance.

> Overall, I have two problems:
> 1) I don't get the idea behind the 'instance' and 'owner' parameters at all. Is there some simple tutorial that can explain these?

First of all, do you understand what descriptors are? This is a fairly
advanced Python concept. For a general tutorial, I would point you to
https://docs.python.org/3/howto/descriptor.html

Whereas the self argument refers to the Celsius instance, the instance
argument refers to the instance of the object currently being accessed
through the descriptor protocol. That is, the object that the Celsius
descriptor is implementing a property of. Using the above example, if
I were to write:

temp = Temperature()
print(temp.celsius)

The access of temp.celsius will cause Celsius.__get__ to be called,
and the value of instance for that call will be temp

The owner argument is the class that this Celsius instance is a
descriptor of. Normally that is just type(instance); however it is
possible to invoke the __get__ method on the class object itself
rather than on an instance, in which case owner is (still) the class
object but instance is None.

> 2) I don't understand the motivation behind the comment. Of course declaring a class variable would cause celcius to be the same for all objects. Shouldn't we be instead using self.celcius in, say, __init__() and then everything will work fine?

It's not clear to me what alternative you're proposing. The reason
celsius is declared on the class is because it's a descriptor, and
descriptors implement properties of classes. Setting an instance
attribute to a descriptor won't accomplish anything.

[Lawrence D’Oliveiro]

On Friday, July 1, 2016 at 1:06:12 PM UTC+12, Ankush Thakur wrote:

> There's something I don't understand about descriptors.

There’s a lot to not understand about them. :)

Every time I feel unsure, I go back to the horse’s mouth: here <http://python-history.blogspot.com/2010/06/inside-story-on-new-style-classes.html> is GvR himself with all the details on “new-style” classes, including the clearest explanation I have come across about descriptors.

[Lawrence D’Oliveiro]

On Friday, July 1, 2016 at 1:06:12 PM UTC+12, Ankush Thakur wrote:

> 1) I don't get the idea behind the 'instance' and 'owner' parameters at
> all. Is there some simple tutorial that can explain these?

Read the GvR blog post I referenced separately. Then after that, the relevant section of the language reference <https://docs.python.org/3/reference/datamodel.html#descriptors> should make a little bit more sense.

Remember, a descriptor has its own class. You can have multiple instances of this class attached to the same owner class. The actual values you want each descriptor instance to get and set are attached to *instances* of the owner class.

> Shouldn't we be instead using self.celcius in, say, __init__() and then
> everything will work fine?

I guess it might. But this way, using descriptors, all the setup is done once at class definition, rather then every time in class instantiation.

[Ankush Thakur]

On Friday, July 1, 2016 at 7:07:09 AM UTC+5:30, Ian wrote:

> First of all, do you understand what descriptors are? This is a fairly
> advanced Python concept. For a general tutorial, I would point you to
> https://docs.python.org/3/howto/descriptor.html

I understand what descriptors try to accomplish, but that link is pretty scary! :P I found your explanation to be much better. The other tutorials would simply ignore "owner", offer no explanation, and provide complete working programs -- this made me go crazy!

> The owner argument is the class that this Celsius instance is a
> descriptor of. Normally that is just type(instance); however it is
> possible to invoke the __get__ method on the class object itself
> rather than on an instance, in which case owner is (still) the class
> object but instance is None.

This is the golden explanation I will always be thankful for! :)

> It's not clear to me what alternative you're proposing. The reason
> celsius is declared on the class is because it's a descriptor, and
> descriptors implement properties of classes. Setting an instance
> attribute to a descriptor won't accomplish anything.

The alternative I'm proposing is indeed what you said: Make `celcius` a per-object attribute and set it up in __init__(). Why will it not accomplish the same thing as setting it on a class-level? Please bear with me a little more and explain. :P

[Ankush Thakur]

On Friday, July 1, 2016 at 9:03:44 AM UTC+5:30, Lawrence D’Oliveiro wrote:
> Every time I feel unsure, I go back to the horse’s mouth: here <http://python-history.blogspot.com/2010/06/inside-story-on-new-style-classes.html> is GvR himself with all the details on “new-style” classes, including the clearest explanation I have come across about descriptors.

Well, that's certainly a very interesting link (and blog!) but my current priority is to just understand this damn thing! :D

[Ankush Thakur]

On Friday, July 1, 2016 at 9:13:19 AM UTC+5:30, Lawrence D’Oliveiro wrote:
>
> > Shouldn't we be instead using self.celcius in, say, __init__() and then
> > everything will work fine?
>
> I guess it might. But this way, using descriptors, all the setup is done once at class definition, rather then every time in class instantiation.

So you're saying the (sole) reason to instantiate descriptor classes at class-level is performance?

[Ian Kelly]

Performance might be part of it (it also saves some checks when
looking up attributes), but in my opinion a big reason for it is
separation of responsibilities. Classes define object behavior;
instances contain object state. For example, you can't define a method
on an instance (though you can certainly store a function there and
call it).

[Lawrence D’Oliveiro]

Haste makes waste, as they say. At least read the relevant part of the article.

[Ankush Thakur]

On Sunday, July 3, 2016 at 4:37:49 AM UTC+5:30, Ian wrote:
> Classes define object behavior;
> instances contain object state. For example, you can't define a method
> on an instance (though you can certainly store a function there and
> call it).

Nice! Thanks for clearing that up. :)

[Ankush Thakur]

On Sunday, July 3, 2016 at 6:32:56 AM UTC+5:30, Lawrence D’Oliveiro wrote:
> Haste makes waste, as they say. At least read the relevant part of the article.

I really feel like I've been pushed into studying its genetics while I only wanted to pluck the fruit. Why do descriptors have to be so overwhelming?! . . . Okay, I guess I'll read at least the descriptor part of the article, but then I'm leaving the whole subject for another day.

[Steven D'Aprano]

On Sun, 3 Jul 2016 09:06 am, Ian Kelly wrote:

> you can't define a method
> on an instance (though you can certainly store a function there and
> call it).

Yes -- the difference is that Python doesn't apply the descriptor protocol
to attributes attached to the instance. So the function remains a function
and is not converted to a method, which means it doesn't automatically get
the special "self" argument:


class Test(object):
pass

instance = Test()

def method(self):
print("method called from self", self)

instance.method = method



If you try to call the "method", you get an error:

py> instance.method()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method() missing 1 required positional argument: 'self'


But if you prepare the method ahead of time, it works:

from types import MethodType
instance.method = MethodType(method, instance)



Now calling it works fine:

py> instance.method()
method called from self <__main__.Test object at 0xb79fcf14>





--
Steven
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[eryk sun]

On Sun, Jul 3, 2016 at 3:32 PM, Steven D'Aprano <st...@pearwood.info> wrote:
>
> But if you prepare the method ahead of time, it works:
>
> from types import MethodType
> instance.method = MethodType(method, instance)

That's a fine way to bind a method, but in the context of this
"descriptor land" topic, I think it's more insightful to call the
function's __get__ method:

>>> type(method)
<class 'function'>
>>> instance.method = method.__get__(instance)
>>> type(instance.method)
<class 'method'>
>>> instance.method.__self__ is instance
True
>>> instance.method.__func__ is method
True
>>> instance.method()
method called from self <__main__.Test object at 0x7faf51d1a198>

[Steven D'Aprano]

On Mon, 4 Jul 2016 09:59 am, eryk sun wrote:

> On Sun, Jul 3, 2016 at 3:32 PM, Steven D'Aprano <st...@pearwood.info>
> wrote:
>>
>> But if you prepare the method ahead of time, it works:
>>
>> from types import MethodType
>> instance.method = MethodType(method, instance)
>
> That's a fine way to bind a method, but in the context of this
> "descriptor land" topic, I think it's more insightful to call the
> function's __get__ method:

Sure, so long as you remember that in real code you shouldn't be calling
dunder methods directly. I don't know whether Python does any pre- or
post-processing of a descriptor's __get__ method, but it could (if not now,
but in the future).