Why doesn't __call__ lead to infinite recursion?
Patrick Lioi
foo is a function
foo is a callable object
foo has method __call__ defined
foo.__call__ is a function
foo.__call__ is a callable object
foo.__call__ has method __call__ defined
foo.__call__.__call__ is a function...
This seems to go on forever. How does calling foo() not lead to
an infinite loop while trying to execute?
bromden
> foo is a callable object
> foo has method __call__ defined
true
true
false
even if you "setattr(foo, '__call__', another_function)"
foo.__call__ won't be called
>>> def foo(): print 'foo'
>>> def loo(): print 'loo'
>>> foo.__call__ = loo
>>> foo()
foo
--
bromden[at]gazeta.pl
Aahz
bromden <bro...@gazeta.pl.no.spam> wrote:
>> foo is a function
>> foo is a callable object
>> foo has method __call__ defined
>
>true
>true
>false
Wrong. See my followup.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
This is Python. We don't care much about theory, except where it intersects
with useful practice. --Aahz
Aahz
Patrick Lioi <pat...@novaroot.com> wrote:
>
>def foo(): pass
>
>foo is a function
>foo is a callable object
>foo has method __call__ defined
>
>foo.__call__ is a function
>foo.__call__ is a callable object
>foo.__call__ has method __call__ defined
You're mixing up the distinction between objects and types. If you do
print foo.__dict__
you'll see that __call__ isn't there. However, if you try
print type(foo).__dict__
you'll see __call__ there. When you do foo(), Python actually does
type(foo).__call__(foo). Because type(foo).__call__ is manipulating
foo, you don't get the circular reference.
John J. Lee
[...]
> you'll see __call__ there. When you do foo(), Python actually does
> type(foo).__call__(foo). Because type(foo).__call__ is manipulating
> foo, you don't get the circular reference.
Still seems weird that type(foo).__call__.__call__ (etc.) is defined.
John
Andrew Dalke
> you'll see __call__ there. When you do foo(), Python actually does
> type(foo).__call__(foo). Because type(foo).__call__ is manipulating
> foo, you don't get the circular reference.
Not quite, but I don't understand how everything works so what I
say may also need corrections.
The call syntax "foo()" does two things. The first is to
get the 'thing' used for the call and the second is to actually
call it. The latter is not done recursively - if the returned
thing can't be called, the attempt at making the call fails.
If 'foo' is an instance, then the implementation code is
something like
thing_to_call = getattr(foo, "__call__")
if thing_to_call is not None:
DO_CALL(thing_to_call, args, kwargs)
The 'DO_CALL' is not a Python function, it's part of
how the implementation works.
The getattr implementation for an instance first tries to
find "__call__" in foo's instance __dict__. If that fails, it
looks in the parent class, and returns a bound method,
that is, a new 'thing' with references to the class method
and to the instance itself. The DO_CALL does the
actual call to this thing with the new arguments. The
bound method thing prepends the self parameter and
does the actual call to the underlying code.
Pretty complicated, and I don't think I was very clear
on that. Here's an example though to show that
foo() != foo.__class__.__call__(foo)
>>> class SPAM:
... def __init__(self):
... def callme(x):
... print "Hello", x
... self.__call__ = callme
... def __call__(self, x):
... print "Hej", x
...
>>> spam = SPAM()
>>> spam("world")
Hello world
>>>
>>> spam.__class__.__call__(spam, "world")
Hej world
>>>
>>> getattr(spam, "__call__")
<function callme at 0x014DA9B0>
>>> getattr(spam, "__init__")
<bound method SPAM.__init__ of <__main__.SPAM instance at 0x013CF148>>
>>> getattr(SPAM, "__call__")
<unbound method SPAM.__call__>
>>>
I'm also missing something because I don't know how
functions work. I thought it was always 'use
getattr(obj, "__call__") to get the thing to call then do
the call machinery on that thing", but it doesn't seem
to do that for functions.
>>> def f(x):
... print "f(%s)" % x
...
>>> def g(x):
... print "g(%s)" % x
...
>>> f(5)
f(5)
>>> f(7)
f(7)
>>> f.__call__ = g.__call__
>>> f(4)
f(4)
>>> f.__call__
<method-wrapper object at 0x014AD6F0>
>>> f.__call__
<method-wrapper object at 0x014AD6F0>
>>> g.__call__
<method-wrapper object at 0x014A02D0>
>>> g.__call__
<method-wrapper object at 0x014AD5B0>
>>>
I expected the 'f(4)' call to return 'g(4)' since I replaced
the function's __call__ with g's __call__.
What's throwing me off is that g.__call__ returns
a new wrapper object each time, while once I
assigned f.__call__, it persistently stayed that way.
So there's some getattr shenanigans with functions
I don't understand. To make it worse, there's also
>>> import types
>>> types.FunctionType.__call__
<slot wrapper '__call__' of 'function' objects>
>>>
I'll leave the clarification to someone else.
Andrew
da...@dalkescientific.com
Aahz
Andrew Dalke <ada...@mindspring.com> wrote:
>
> [...]
No time to investigate further, but all your examples used classic
classes instead of new-style classes; I'm pretty sure that new-style
classes will more closely emulate the way functions work. There's also
the wrinkle I didn't mention that functions use a dict proxy IIRC.
Andrew Dalke
> No time to investigate further, but all your examples used classic
> classes instead of new-style classes; I'm pretty sure that new-style
> classes will more closely emulate the way functions work. There's also
> the wrinkle I didn't mention that functions use a dict proxy IIRC.
Interesting. Very interesting.
>>> class XYZ(object):
... def __init__(self):
... def abc(x):
... print "Hello", x
... self.__call__ = abc
... def __call__(self, x):
... print "Yo", x
...
>>> xyz = XYZ()
>>> xyz("fred")
Yo fred
>>>
>>> getattr(xyz, "__call__")
<function abc at 0x0168CB70>
>>>
I wonder if this will affect any of my code.
It does explain the observed differences better, since FunctionType
in 2.3 is derived from object while my class was not.
Andrew
da...@dalkescientific.com
Michael Hudson
> Aahz:
> > No time to investigate further, but all your examples used classic
> > classes instead of new-style classes; I'm pretty sure that new-style
> > classes will more closely emulate the way functions work. There's also
> > the wrinkle I didn't mention that functions use a dict proxy IIRC.
>
> Interesting. Very interesting.
Yes :-)
You have to have something like this when you do things like 'print
type(foo)'. This should call the *types* *bound* __str__ method, not
try to call the *instances* *unbound* __str__ method...
Cheers,
mwh
--
Its unmanageable complexity has spawned more fear-preventing tools
than any other language, but the solution _should_ have been to
create and use a language that does not overload the whole goddamn
human brain with irrelevant details. -- Erik Naggum, comp.lang.lisp
Beni Cherniavsky
public. I apologize, it's late...]
The point is that down there, sits the C level which doesn't go
through the Python definition. When you call ``foo()``, ``type(foo)``
is asked at the *C* level how to call it (read: the type struct is
accessed and the call-behavior slot, if non-NULL, is called by plain
C function call - which can't be intercepted in C so there is no futher
recursion).
It so happens that classes, when asked this, go back into the Python
level and look for the `__call__` attribute. They do this for all
operations, giving you the dynamism and flexibility we all love.
OTOH, functions, when asked this, simply execute their code objects with the
given arguments. The attribute `__call__` on function is a "proxy" attribute.
It is not used by function objects for the call, it only exposes the C-level
call machinery at Python level. The same convention is used for other
built-in types and operations, so that from Python you see Python-level and
C-level methods in the same way. You only have to be aware of the distinction
when working with bizzare extension types that don't respect this convention,
or asking questions like this one about how it all works... (Another one: why
__getattribute__ is not infinitely recusive? Same explanation.)
--
Beni Cherniavsky <cb...@tx.technion.ac.il>
Look, Mom, no viruses! [Hint: I use GNU/Linux]