How to peek inside a decorated function

[Steven D'Aprano]

Suppose I have a function f() which I know has been decorated, but I don't
have access to the original undecorated function any longer:

def reverse(func):
def f(*args):
args = list(args)
args.reverse()
return func(*args)
return f

def say(*args):
print args

rsay = reverse(say)
del say

Is there any way to peek inside the decorated function rsay() to get access
to the undecorated function say()?

If I look at the code object I can see a reference to the original:

>>> rsay.func_code.co_names
('list', 'args', 'reverse', 'func')

and if I disassemble the code object I can see it being dereferenced:

>>> dis.dis(rsay.func_code)
[snip for brevity]
5 22 LOAD_DEREF 0 (func)
25 LOAD_FAST 0 (args)
28 CALL_FUNCTION_VAR 0
31 RETURN_VALUE

but if I look at the closure object, nothing seems useful:

>>> dir(rsay.func_closure[0])
['__class__', '__cmp__', '__delattr__', '__doc__',
'__getattribute__', '__hash__', '__init__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__str__']

and I can't find any other attributes which refers back to the undecorated
original function.

--
Steven

[Terry Reedy]

In 3.0, rsay.__closure__[0].cell_contents would be the original function
object.

[Peter Otten]

Steven D'Aprano wrote:

Following Terry's lead it turns out that cell contents become easily
accessible in python 2.5:

$ cat cellcontents.py
"""
>>> def reverse(func):
... def f(*args):
... args = list(args)
... args.reverse()
... return func(*args)
... return f
...
>>> def say(*args):
... print args
...

>>> rsay = reverse(say)
>>> del say

>>> c = rsay.func_closure[0]
>>> "cell_contents" in dir(c)
True
>>> c.cell_contents(1,2,3)
(1, 2, 3)
>>>
"""

import doctest
doctest.testmod()
$ python2.6 cellcontents.py
$ python2.5 cellcontents.py
$ python2.4 cellcontents.py
**********************************************************************
File "cellcontents.py", line 15, in __main__
Failed example:
"cell_contents" in dir(c)
Expected:
True
Got:
False
**********************************************************************
File "cellcontents.py", line 17, in __main__
Failed example:
c.cell_contents(1,2,3)
Exception raised:
Traceback (most recent call last):
File "doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest __main__[6]>", line 1, in ?
c.cell_contents(1,2,3)
AttributeError: 'cell' object has no attribute 'cell_contents'
**********************************************************************
1 items had failures:
2 of 7 in __main__
***Test Failed*** 2 failures.

Peter

[Hrvoje Niksic]

Steven D'Aprano <st...@pearwood.info> writes:

> Suppose I have a function f() which I know has been decorated, but I don't
> have access to the original undecorated function any longer:
>
> def reverse(func):
> def f(*args):
> args = list(args)
> args.reverse()
> return func(*args)
> return f
>
> def say(*args):
> print args
>
> rsay = reverse(say)
> del say
>
> Is there any way to peek inside the decorated function rsay() to get access
> to the undecorated function say()?

This works in Python 2.5.2:

>>> rsay.func_closure[0].cell_contents
<function say at 0xb7e67224>

Of course, this applies only if you know there's only one free
variable, and you know that the decorator is in fact implemented with
a closure, and so on.

>>>> dir(rsay.func_closure[0])
> ['__class__', '__cmp__', '__delattr__', '__doc__',
> '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__',
> '__reduce_ex__', '__repr__', '__setattr__', '__str__']

I got 'cell_contents' as well when I did the 'dir', at least under
Python 2.5.2.

[Aaron Brady]

Assume the decorator preserves the function at all; that is, it
doesn't discard it. Then you can scan func_closure. In poor cases,
you might have to search through a nested decoration as well. In
others, such as when the decorator is an object, not a function, you
might have to search an instance dictionary, also possibly nested.
The leading assumption also requires that the function is stored in a
Python object; that is, not in a C structure.

In Python 3.0.1:

>>> def f( fun ):
... def g( *ar, **kw ):
... return fun( *ar, **kw )
... return g
...
>>> def h( x ):
... print( x )
...
>>> i= f( h )
>>> i.__closure__[0].cell_contents
<function h at 0x00B59810>

[Steven D'Aprano]

Hrvoje Niksic wrote:

>> Is there any way to peek inside the decorated function rsay() to get
>> access to the undecorated function say()?
>
> This works in Python 2.5.2:
>
>>>> rsay.func_closure[0].cell_contents
> <function say at 0xb7e67224>

Thanks to everyone who responded.

The reason I ask is because I've just spent the weekend battling with
doctests of decorated functions, and discovering that without
functools.wraps() all my doctests weren't being called. I'm wondering
whether it would be a good idea for doctest to auto-detect tests in
closures.

This is not needed if you call wraps() appropriately, but you might not
always do that.

--
Steven

[Peter Otten]

Steven D'Aprano wrote:

> Suppose I have a function f() which I know has been decorated, but I don't
> have access to the original undecorated function any longer:
>
> def reverse(func):
> def f(*args):
> args = list(args)
> args.reverse()
> return func(*args)
> return f
>
> def say(*args):
> print args
>
> rsay = reverse(say)
> del say
>
>
> Is there any way to peek inside the decorated function rsay() to get
> access to the undecorated function say()?

Here's a hack for Python 2.4:

def make_extractor(x=None):
def extractor(): return x
return extractor

extractor = make_extractor()
function = type(extractor)

def get_cell_contents(cell):
return function(extractor.func_code, {}, "yadda", None, (cell,))()

get_cell_contents(rsay.func_closure[0])(1,2,3)

Peter

[Duncan Booth]

Steven D'Aprano <st...@pearwood.info> wrote:
>
> The reason I ask is because I've just spent the weekend battling with
> doctests of decorated functions, and discovering that without
> functools.wraps() all my doctests weren't being called. I'm wondering
> whether it would be a good idea for doctest to auto-detect tests in
> closures.
>
> This is not needed if you call wraps() appropriately, but you might not
> always do that.
>

I don't think it is a good idea to call the wrapped doctests automatically,
but I do think it might be a good idea to detect and warn about anything
which looks like it is a wrapper where the wrapped function includes a
doctest and the wrapper doesn't.

--
Duncan Booth http://kupuguy.blogspot.com