Why this doesn't work?
mk
Sorry to bother everyone again, but I have this problem bugging me:
#!/usr/bin/python -i
class Foo(object):
def nostat(self,val):
print val
nostat.__orig_get__ = nostat.__get__
@staticmethod
def nostatget(*args, **kwargs):
print 'args:', args, 'kwargs:', kwargs
nostat.__orig_get__(*args, **kwargs)
nostat.__get__ = nostatget
setattr(nostat,'__get__',nostatget)
f = Foo()
f.nostat('a')
print f.nostat.__get__ is f.nostat.__orig_get__
This produces:
a
False
I expected to see 'nostatget' output: nostat.__get__ = nostatget
obviously failed to replace this function's __get__ method.
The question is why? Isn't __get__ a normal attribute of a function nostat?
This is made so much weirder that nostat.__get__ is no longer original
__get__, so it looks like it should have been replaced, but if so, why
nostatget doesn't get called?
Regards,
mk
Steven D'Aprano
> nostat.__orig_get__ = nostat.__get__
I should point out that leading-and-trailing-double-underscore names are
reserved for use by the language.
It's unlikely that Python will introduce a special method named
__orig_get__, and in truth the prohibition against using such names is
honoured more in the breach than in the observance (e.g. people often use
metadata names like __version__, __author__, etc.). But just be aware
that you are in technical breach of Python best practices, and should
consider renaming it as _orig_get or __orig_get.
--
Steven
John Posner
>
> Sorry to bother everyone again, but I have this problem bugging me:
>
> #!/usr/bin/python -i
>
> class Foo(object):
>
> def nostat(self,val):
> print val
>
> nostat.__orig_get__ = nostat.__get__
>
> @staticmethod
> def nostatget(*args, **kwargs):
> print 'args:', args, 'kwargs:', kwargs
> nostat.__orig_get__(*args, **kwargs)
>
> nostat.__get__ = nostatget
> setattr(nostat,'__get__',nostatget)
>
>
> f = Foo()
>
> f.nostat('a')
>
> print f.nostat.__get__ is f.nostat.__orig_get__
>
>
> This produces:
>
> a
> False
>
> I expected to see 'nostatget' output: nostat.__get__ = nostatget
> obviously failed to replace this function's __get__ method.
I don't quite understand the above sentence, so I'm assuming that you
wanted the final "is" test to be "True" instead of "False".
It looks like you've replaced ...
(A) the original __get__() method
... with ...
(B) a new method that *calls* the original __get__() method
So why should you expect (A) and (B) to be the same object?
>
> The question is why? Isn't __get__ a normal attribute of a function nostat?
>
> This is made so much weirder that nostat.__get__ is no longer original
> __get__, so it looks like it should have been replaced, but if so, why
> nostatget doesn't get called?
>
> Regards,
> mk
>
>
>
-John
Bruno Desthuilliers
>
(snip)
Sorry, no time to get into details now - but I can at least provide a
couple hints.
The first point is that, to override a method on an _instance_, you have
to provide a method object, not a plain function - remember that the
descriptor protocol is only invoked on _class_ attributes, not on
instance attributes.
class Foo(object):
def bar(self):
print "the original bar"
def mybar(self):
print "mybar"
>>> f = Foo()
>>> f.bar = mybar
>>> f.bar()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/python-1287O_i.py", line 32, in <module>
f.bar()
TypeError: mybar() takes exactly 1 argument (0 given)
>>> type(f.bar)
<type 'function'>
>>> f.bar is mybar
True
>>> f.bar(f)
mybar
As you see, "bar" is here resolved as an ordinary instance attribute -
so here it evals to the "mybar" function object. If you want it to be a
method object, you do have to invoke the descriptor protocol manually:
>>> f.bar = mybar.__get__(f, type(f))
>>> f.bar
<bound method Foo.mybar of <__main__.Foo object at 0xb7e16b0c>>
>>> f.bar()
mybar
Or alternatively, you can use the types module:
>>> import types
>>> f.bar = types.MethodType(mybar, f, type(f))
>>> f.bar
<bound method Foo.mybar of <__main__.Foo object at 0xb7e16b0c>>
>>> f.bar()
mybar
Second point is that the descriptor protocol is invoked on each and
every lookup. So when it comes to function class attributes, you get a
new method object each time:
>>> f = Foo()
>>> m1 = f.bar
>>> m1
<bound method Foo.bar of <__main__.Foo object at 0xb7cb522c>>
>>> m2 = f.bar
>>> m2
<bound method Foo.bar of <__main__.Foo object at 0xb7cb522c>>
>>> id(m1)
3084861188L
>>> id(m2)
3083656564L
>>> m1 is m2
False
>>>
I think this should help you understand why your code doesn't work as
you assumed it would !-)
PS : as a side note, a common optimization trick is to "short-circuit"
the whole lookup / descriptor mechanism when you have to call the same
method of the same object in a loop:
l = []
append = l.append
for i in xrange(100):
append(i)
print l
HTH
mk
> On Thu, 18 Feb 2010 18:28:44 +0100, mk wrote:
>
>> nostat.__orig_get__ = nostat.__get__
>
> I should point out that leading-and-trailing-double-underscore names are
> reserved for use by the language.
Right... I completely missed that. I will try to change the habit.
I am under impression that a function with no underscore in name is
meant to be called "publicly" on instance, like Foo().nostat, a function
with one underscore (and no trailing underscores) is meant to be like
"it's mostly intended for internal use, but you can still call it",
somewhat like "protected" in C++, and a function with two leading
underscores (and no trailing underscores) is meant as completely
internal to the class, not meant to be called by outsiders, somewhat
like "private" in C++ (I abstract from obvious point in C++ that
semantics of those keywords is enforced by a compiler in C++).
Is that correct?
Regards,
mk
mk
>> a
>> False
>>
>> I expected to see 'nostatget' output: nostat.__get__ = nostatget
>> obviously failed to replace this function's __get__ method.
>
> I don't quite understand the above sentence, so I'm assuming that you
> wanted the final "is" test to be "True" instead of "False".
No. I should have described my goal in plain English in first place.
I wanted to replace nostat function's __get__ descriptor with a function
that would allow me to peek inside it (i.e. into __get__ descriptor
arguments) and that would then call original nostat.__get__.
So no, I expected it to be false, that is, I expected to replace
original descriptor with a "custom" descriptor. I just wondered why this
custom "nostatget" descriptor didn't get called even though the new
nostat.__get__ didn't seem to be its old self (so quite likely it was
the new, nostatget, descriptor).
But Bruno pointed out that I need instancemethod for that, not plain
function.
Regards,
mk
Bruno Desthuilliers
Mostly, yes.
- s/function/whatever/ : these rules apply to all names
- the single leading underscore denote implementation, IOW : you should
not use it, unless you have a pretty good reason to do so AND you take
full responsability for what may happens
- the double leading underscore is in fact mostly to protect your
attribute from accidental overloading. It triggers a (simple)
name-mangling operation that turns "__yourname" into
"_Yourclassname__yourname". FWIW, this is something I must have a use
for at most once a year, and even then most of the times because I'm a
bit of a control freak sometimes.
Bruno Desthuilliers
> John Posner wrote:
>>> a
>>> False
>>>
>>> I expected to see 'nostatget' output: nostat.__get__ = nostatget
>>> obviously failed to replace this function's __get__ method.
>>
>> I don't quite understand the above sentence, so I'm assuming that you
>> wanted the final "is" test to be "True" instead of "False".
>
> No. I should have described my goal in plain English in first place.
>
> I wanted to replace nostat function's __get__ descriptor
A "descriptor" (shortcut for "object implementing the descriptor
protocol) is an object that have a __get__ method - not the __get__
method itself. A function is a descriptor. The __get__ method of the
function type is a method, and methods are not descriptors.
> with a function
> that would allow me to peek inside it (i.e. into __get__ descriptor
> arguments)
=> __get__(self, instance, cls=None)
When called as a result of a look up on an instance f of class Foo,
nostat.__get__ will get (nostat, f, Foo) as args.
> So no, I expected it to be false, that is, I expected to replace
> original descriptor with a "custom" descriptor.
s/descriptor/__get__/
Anyway: these types (function, method etc) are implemented in (higly
optimized) C, and with a lot of restriction for both performance and
sanity reasons. It's a case of practicality beats purity.
Another - perhaps more rewarding - exercise might be to implement a
custom callable type that implements the descriptor protocol the same
way the function do.