Decorating methods - where do my arguments go?
Mikael Olofsson
I have long tried to avoid decorators, but now I find myself in a
situation where I think they can help. I seem to be able to decorate
functions, but I fail miserably when trying to decorate methods. The
information I have been able to find on-line focuses on decorating
functions, and do not mention any special problem regarding methods.
Consider the following example. I start by defining a simple decorator:
>>> class test_decorator(object):
... def __init__(self,func):
... self._func = func
... def __call__(self, *args):
... print 'Decorator:', args
... self._func(*args)
Then I decorate a function:
>>> @test_decorator
... def func(*args):
... print 'Function: ', args
Let's try that:
>>> func(1,2,3)
Decorator: (1, 2, 3)
Function: (1, 2, 3)
OK! That was what I expected. Let's decorate a method:
>>> class cls(object):
... @test_decorator
... def meth(self,*args):
... print 'Method: ', args
Then try that:
>>> cls().meth(1,2,3)
Decorator: (1, 2, 3)
Method: (2, 3)
Oops! That's weird. I had expected - or at least wanted - the same
result as for the function above.
Where did the first argument go? If it was sent to the original method
meth as the real first argument (self), then why did I not get an exception?
More importantly: How do I write a decorator that does not drop arguments?
I guess it all has to do with the fact that the returned callable isn't
a method of cls. Is it possible to write a decorator that returns a
callable that is a method of cls, when used on methods in cls?
/MiO
Peter Otten
You have to turn your decorator into a descriptor by providing a __get__()
method. A primitive example:
class test_decorator(object):
def __init__(self,func):
self._func = func
def __call__(self, *args):
print 'Decorator:', args
self._func(self.inst, *args)
def __get__(self, inst, cls):
self.inst = inst
return self
Read more about the details at
http://users.rcn.com/python/download/Descriptor.htm
Peter
George Sakkis
Yes, just return an actual function from the decorator instead of a
callable object:
def test_decorator2(func):
def wrapper(*args):
print 'Decorator2:', args
func(*args)
return wrapper
class cls(object):
@test_decorator
def meth(self,*args):
print 'Method: ', args
@test_decorator2
def meth2(self,*args):
print 'Method2: ', args
>>> cls.meth
<__main__.test_decorator object at 0x87663cc>
>>> cls.meth2
<unbound method cls.wrapper>
>>> cls().meth2(1,2,3)
Decorator2: (<__main__.cls object at 0x8766ecc>, 1, 2, 3)
Method2: (1, 2, 3)
The reason this works is that functions are already descriptors, so
you don't have to explicitly define __get__() as you would for a new
callable class (see Peter's example).
HTH,
George
MRAB
@test_decorator
def meth(self,*args):
print 'Method: ', args
>>> cls().meth(1,2,3)
Decorator: (1, 2, 3)
self: 1
Method: (2, 3)
Michele Simionato
> >>> class test_decorator(object):
> ... def __init__(self,func):
> ... self._func = func
> ... def __call__(self, *args):
> ... print 'Decorator:', args
> ... self._func(*args)
Or you could use the decorator module (http://pypi.python.org/pypi/
decorator):
from decorator import decorator
@decorator
def test_decorator(func, *args, **kw):
print 'Decorator:', args
return func(*args)
Mikael Olofsson
> You have to turn your decorator into a descriptor by providing a __get__()
> method. A primitive example:
>
> class test_decorator(object):
> def __init__(self,func):
> self._func = func
> def __call__(self, *args):
> print 'Decorator:', args
> self._func(self.inst, *args)
> def __get__(self, inst, cls):
> self.inst = inst
> return self
Thanks! Works perfectly for methods, as far as I can see. That's
perfectly OK, since that is what I was asking for. What I failed to
include in my original post is that I intend to use the decorator for
functions as well. The above fails for functions on the second row of
__call__ with
AttributeError: 'test_decorator' object has no attribute 'inst'
It seems to me like __get__ is not called when decorating a function. I
guess there's no reasonable value for inst in that case. It might be the
function itself, but it doesn't make sense to call the function with
itself as its first argument, when it isn't supposed to be called that way.
George Sakkis decorator function solution seems to work equally well for
functions and methods. However, I prefer the cleaner encapsulation given
by a class. Based on those observations, I think I will use the
following approach:
>>> class test_decorator(object):
... _inst = None
... def __init__(self,func):
... self._func = func
... def __call__(self, *args):
... print 'Decorator:', args
... if self._inst is None:
... print 'We seem to be decorating a function.'
... self._func(*args)
... else:
... print 'We seem to be decorating a method.'
... self._func(self._inst,*args)
... def __get__(self, inst, cls):
... self._inst = inst
... return self
...
>>>
>>> @test_decorator
... def func(*args):
... print 'Function: ', args
...
>>>
>>> func(1,2,3)
Decorator: (1, 2, 3)
We seem to be decorating a function.
Function: (1, 2, 3)
>>>
>>> class cls(object):
... @test_decorator3
... def meth(self,*args):
... print 'Method: ', args
...
>>>
>>> cls().meth(1,2,3)
Decorator: (1, 2, 3)
We seem to be decorating a method.
Method: (1, 2, 3)
If there are any drawbacks with this approach that I fail to see, please
enlighten me.
/MiO
Mikael Olofsson
> Yes, just return an actual function from the decorator instead of a
> callable object:
>
> def test_decorator2(func):
> def wrapper(*args):
> print 'Decorator2:', args
> func(*args)
> return wrapper
>
>
> class cls(object):
> @test_decorator
> def meth(self,*args):
> print 'Method: ', args
>
> @test_decorator2
> def meth2(self,*args):
> print 'Method2: ', args
Thanks! This has the merit over Peter's solution that it seems to work
for both functions and methods. However, as you can see from my answer
to Peter, I think I will go for a class based approach anyway. Still,
your answer helped me grasping the concepts.
/MiO
Duncan Booth
> George Sakkis decorator function solution seems to work equally well
for
> functions and methods. However, I prefer the cleaner encapsulation
given
> by a class. Based on those observations, I think I will use the
> following approach:
>
> >>> class test_decorator(object):
> ... _inst = None
> ... def __init__(self,func):
> ... self._func = func
> ... def __call__(self, *args):
> ... print 'Decorator:', args
> ... if self._inst is None:
> ... print 'We seem to be decorating a function.'
> ... self._func(*args)
> ... else:
> ... print 'We seem to be decorating a method.'
> ... self._func(self._inst,*args)
> ... def __get__(self, inst, cls):
> ... self._inst = inst
> ... return self
> ...
The __get__ method should be returning a new object, NOT modifying the
state of the decorator. As written it will break badly and unexpectedly
in a variety of situations:
>>> inst1 = cls()
>>> inst2 = cls()
>>> inst1, inst2
(<__main__.cls object at 0x011BC470>, <__main__.cls object at
0x011BCC90>)
>>> m = inst1.meth
>>> m()
Decorator: ()
We seem to be decorating a method.
Method: <__main__.cls object at 0x011BC470> ()
>>> inst2.meth()
Decorator: ()
We seem to be decorating a method.
Method: <__main__.cls object at 0x011BCC90> ()
>>> m()
Decorator: ()
We seem to be decorating a method.
Method: <__main__.cls object at 0x011BCC90> ()
or even this:
>>> inst1.meth(inst2.meth())
Decorator: ()
We seem to be decorating a method.
Method: <__main__.cls object at 0x011BCC90> ()
Decorator: (None,)
We seem to be decorating a method.
Method: <__main__.cls object at 0x011BCC90> (None,)
--
Duncan Booth http://kupuguy.blogspot.com
Mikael Olofsson
> The __get__ method should be returning a new object, NOT modifying the
> state of the decorator. As written it will break badly and unexpectedly
> in a variety of situations:
>
> [snip good examples of things going bad]
Ouch! So, does that mean that George's solution based on a function is
the way to go, or does that mean that my __call__ should analyze the
passed callable?
/MiO
Peter Otten
I usually use decorator functions, but I think the following should work,
too:
class deco(object):
def __init__(self, func):
self._func = func
def __call__(self, *args):
print "Decorator:", args
self._func(*args)
def __get__(self, *args):
return deco(self._func.__get__(*args))
Peter
Mikael Olofsson
> I usually use decorator functions, but I think the following should work,
> too:
>
> class deco(object):
> def __init__(self, func):
> self._func = func
> def __call__(self, *args):
> print "Decorator:", args
> self._func(*args)
> def __get__(self, *args):
> return deco(self._func.__get__(*args))
This looks like a winner to me. It is elegant, works for functions as
well as for methods, and does not mess with the internals of the
decorator which avoids the problems that Duncan pointed out.
Thanks Peter, and thanks everyone else that chimed in and contributed.
/MiO
Michele Simionato
> Peter Otten wrote:
> > I usually use decorator functions, but I think the following should work,
> > too:
>
> > class deco(object):
> > def __init__(self, func):
> > self._func = func
> > def __call__(self, *args):
> > print "Decorator:", args
> > self._func(*args)
> > def __get__(self, *args):
> > return deco(self._func.__get__(*args))
>
> This looks like a winner to me. It is elegant, works for functions as
> well as for methods, and does not mess with the internals of the
> decorator which avoids the problems that Duncan pointed out.
Still it turns something which is a function into an object and you
lose the
docstring and the signature. pydoc will not be too happy with this
approach.
Duncan Booth
Also it means every time the decorator is invoked 'self' references a
new instance of deco. I don't know why Mikael wants to use a class
rather than a function but if he wants to save state between calls this
isn't going to help.
Something like this may work, and preserves the function name, type and
docstring:
from functools import wraps
class Decorator(object):
def __init__(self, func):
self._func = func
def __new__(cls, func):
decorator = object.__new__(cls)
decorator.__init__(func)
@wraps(func)
def wrapper(*args, **kwds):
return decorator(*args, **kwds)
return wrapper
class deco(Decorator):
def __init__(self, func):
self.calls = 0
Decorator.__init__(self, func)
def __call__(self, *args):
self.calls += 1
print "Decorator:", args, "called", self.calls, "times"
return self._func(*args)
class C(object):
@deco
def meth(self, arg):
"""meth's docstring"""
print "meth %r %r" % (self, arg)
return arg+1
inst1 = C()
inst2 = C()
print repr(inst1.meth), inst1.meth.__doc__
print inst1.meth(5)
print inst1.meth(inst2.meth(3))
-------------------
Output:
<bound method C.meth of <__main__.C object at 0x00B45390>> meth's
docstring
Decorator: (<__main__.C object at 0x00B45390>, 5) called 1 times
meth <__main__.C object at 0x00B45390> 5
6
Decorator: (<__main__.C object at 0x00B453D0>, 3) called 2 times
meth <__main__.C object at 0x00B453D0> 3
Decorator: (<__main__.C object at 0x00B45390>, 4) called 3 times
meth <__main__.C object at 0x00B45390> 4
5
Mikael Olofsson
> Still it turns something which is a function into an object and
> you lose the docstring and the signature. pydoc will not be too
> happy with this approach.
Duncan Booth wrote:
> I don't know why Mikael wants to use a class rather than a function
> but if he wants to save state between calls this isn't going to help.
I've never used decorators before, and that's the main reason for my
confusion. I have found myself in a situation where I think that
decorators is the way to go. So I tried to find information about how to
write them the usual way: Googling and browsing documentation. I found
examples using both functions and classes, but not much explanation of
what's going on, especially not in the class case. I didn't find any
arguments for favouring any of those alternatives. Actually, most
examples I found did not even include the definition of the actual
decorator or any information about where they might come from, only the
@-row.
Based on the above observations, I assumed that both alternatives were
equally possible. The first answers I got here also led me to believe
that that was the case. My understanding is that in most cases, you can
use a callable object instead of a function with little or no practical
difference, except the syntax of its definition. In my eyes, a class
definition looks nicer and is more readable than a nested function.
Therefore, I tend to define a class with a __call__ method and helper
methods instead of nested functions, if I find the need.
Practicality beats purity, and it seem like using a class for decoration
is a lot more hassle than using a function. I fold.
/MiO