New-style classes and special methods
Jp Calderone
class Foo:
def __getattr__(self, name):
return lambda arg: arg
It can be used thusly:
f = Foo()
print f[10]
Producing the output "10". Now consider this class:
class NewFoo(object):
def __getattr__(self, name):
return lambda arg: arg
When used in the same way, this exception is raised:
TypeError: unindexable object
From a quick glance at the source, it seems that neither tp_as_mapping nor
tp_as_sequence are being initialized, presumably because __getitem__ isn't
seen as being defined.
Is this analysis of the behavior correct? If so, what are the chances of
this being "fixed"?
Jp
--
A sad spectacle. If they be inhabited, what a scope for misery
and folly. If they be not inhabited, what a waste of space.
-- Thomas Carlyle, looking at the stars
--
up 15 days, 10:29, 7 users, load average: 0.00, 0.00, 0.00
Andrew Bennetts
>
> Is this analysis of the behavior correct? If so, what are the chances of
> this being "fixed"?
I believe you are correct -- new-style classes only check their class, not
instances, for the __magic_methods__.
-Andrew.
Carl Banks
> [-- text/plain, encoding quoted-printable, 37 lines --]
>
> Consider this class:
>
> class Foo:
> def __getattr__(self, name):
> return lambda arg: arg
>
> It can be used thusly:
>
> f = Foo()
> print f[10]
Sure?
> Producing the output "10". Now consider this class:
>
> class NewFoo(object):
> def __getattr__(self, name):
> return lambda arg: arg
>
> When used in the same way, this exception is raised:
>
> TypeError: unindexable object
It might have worked if you had used __getitem__ instead of
__getattr__. At least it did for me.
> From a quick glance at the source, it seems that neither tp_as_mapping nor
> tp_as_sequence are being initialized, presumably because __getitem__ isn't
> seen as being defined.
Correct.
> Is this analysis of the behavior correct? If so, what are the chances of
> this being "fixed"?
>
> Jp
>
--
CARL BANKS
Alex Martelli
Exactly! And this in turn IS a fix of the previous murky behavior,
which still abides in classic classes only for backwards compat --
there is no chance that the murky behavior of old will resurface,
thankfully.
Why the old behavior was ill defined is made clearest by the fact
that classes are just instances of types (their metaclasses). E.g.:
When you call a class, say C(), what __call__ do you want to
use -- C.__call__, the per-instance one, or type(C).__call__,
the metaclass one? Obviously the latter, or you'd have a hard
time creating instances of any class that defines __call__. The
new and clearer rule does away with the murkiness.
Of course, you and I, Andrew, just unearthed a neat trick to
still allow "per-instance customization" over on the properties
thread -- give each instance its own custom subclass. Maybe
Jp Calderone will want to have a look at that thread, towards
the end where the solution emerged by Andrew's and my joint
efforts. Basically it's just a question of giving the class
whose instances have that particular need a specific __new__
method -- howeer, you can also do it in others ways, for ex. if
only a FEW of the instances need such customization you can
choose to equip only those instances with their own custom
subclasses and leave all the others to share the main class.
Here's an example of the latter idea:
def setSpecialAttr(inst, name, value):
try:
inst.__customized_subclass__
except AttributeError:
class subclass(inst.__class__):
__customized_subclass__ = True
inst.__class__ = subclass
setattr(inst.__class__, name, value)
Alex
Jack Diederich
> On Mon, Feb 24, 2003 at 07:13:44AM +0000, Carl Banks wrote:
> > Jp Calderone wrote:
> > > [-- text/plain, encoding quoted-printable, 37 lines --]
> > >
> > > Consider this class:
> > >
> > > class Foo:
> > > def __getattr__(self, name):
> > > return lambda arg: arg
> > >
> > > It can be used thusly:
> > >
> > > f = Foo()
> > > print f[10]
> >
> > Sure?
>
>
Confused me at first, too
class Foo:
def __getattr__(self, name):
print name
return lambda x:x
f = Foo()
f[10]
prints
'__getitem__'
So writing it without the obfu
class Foo:
def __getitem__(self, name):
return name
An aside, don't try 'print self' above, it tries to get __repr__
for the class ...
IMO the newer behavior makes more sense; python wasn't meant
to provide you impossible questions to ask at interviews
-jackdied
Andrew Bennetts
> > [-- text/plain, encoding quoted-printable, 37 lines --]
> >
> > Consider this class:
> >
> > class Foo:
> > def __getattr__(self, name):
> > return lambda arg: arg
> >
> > It can be used thusly:
> >
> > f = Foo()
> > print f[10]
>
> Sure?
>>> class Foo:
... def __getattr__(self, name):
... return lambda arg: arg
...
>>> f = Foo()
>>> print f[10]
10
Yep, I'd say he's sure.
-Andrew.
Alex Martelli
Indeed. For the benefit of readers who might be puzzled by
this: Python uses __getattr__ to look up (almost) ALL
attributes not found by other means -- including special
methods. Here, when you write f[10], Python looks for special
method __getitem__ -- and since f instances a classic class,
Python looks for said method per-instance, and does so (not
having found it by other means) by calling f.__getattr__ ...
and getting the "lambda arg: arg" callable as the result.
Of course, this class is nearly unusable...:
>>> f
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: <lambda>() takes exactly 1 argument (0 given)
>>> f+2
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: coercion should return None or 2-tuple
>>> f == 23
23
>>>
as its instances claim to have all kinds of special methods
which in fact they do not implement with proper signatures
(all "special methods" end up with the one-mandatory-argument
signature of that naughty "lambda arg: arg" trick).
Add something like "print 'returning fake',name" as the
first statement in the body of that __getattr__ to see what
is going on in a bit more detail.
If you want to create a similarly-unusable class of the
new-style persuasion, you can do it with a custom metaclass,
of course. Left as an exercise for the true masochists...;-)
Alex
Terry Reedy
"Alex Martelli" <al...@aleax.it> wrote in message
news:qhl6a.303192$AA2.11...@news2.tin.it...
> Of course, this class is nearly unusable...:
>
> >>> f
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> TypeError: <lambda>() takes exactly 1 argument (0 given)
Yes, subverting Python's 'Every object has a printable representation'
feature can be a real PITA:
>>> globals()
{'C': <class __main__.C at 0x007960D0>, 'f': Traceback (most recent
call last):
File "<stdin>", line 1, in ?
TypeError: <lambda>() takes exactly 1 argument (0 given)
I discovered this time bomb when I came back to my interactive
session, an hour after running the experiment with the cracked class,
to do something completely different.
I am tempted to suggest that the feature be made more robust by
wrapping internal calls to __repr__/__str__ with try/except. The
except clause would call the dependable default versions -- and maybe
tack on a bug note.
< C instance at 0xabc with bad __str__>
TJR
Carl Banks
D'oh. The trickery escaped me.
--
CARL BANKS
Jp Calderone
Thanks, this is a neat trick, I'll have to keep it in mind. My use case
lies along different lines (I tell a lie, I have no use case, but if I did,
it would lie along these...):
class AccessError(AttributeError): pass
class PromiscuousPolicy:
def allowAccess(self, name):
return True
class Proxy(object):
def __init__(self, original, policy=PromiscuousPolicy()):
self.__original = original
self.__policy = policy
def __getattribute__(self, name):
p = super(Proxy, self).__getattribute__('__policy')
if not p.allowAccess(name):
raise AccessError
return getattr(
super(Proxy, self).__getattribute__('__original'),
name
)
I'm not sure I see a way to apply the approach setSpecialAttr uses here...
Though, maybe Proxy could iterate over the attributes of 'original', adding
fake-out attributes to a new class object, then set its own class to that
new one... Hrm... *wanders off to play around in the interpreter*.
Jp
--
up 16 days, 4:29, 5 users, load average: 0.30, 0.12, 0.04
Alex Martelli
...
>> def setSpecialAttr(inst, name, value):
>> try:
>> inst.__customized_subclass__
>> except AttributeError:
>> class subclass(inst.__class__):
>> __customized_subclass__ = True
>> inst.__class__ = subclass
>> setattr(inst.__class__, name, value)
>>
>
> Thanks, this is a neat trick, I'll have to keep it in mind. My use case
> lies along different lines (I tell a lie, I have no use case, but if I
> did, it would lie along these...):
>
> class AccessError(AttributeError): pass
>
> class PromiscuousPolicy:
> def allowAccess(self, name):
> return True
>
> class Proxy(object):
> def __init__(self, original, policy=PromiscuousPolicy()):
> self.__original = original
> self.__policy = policy
>
> def __getattribute__(self, name):
> p = super(Proxy, self).__getattribute__('__policy')
I think (haven't tested) that this would fail due to name
mangling issue. My advice: use ONE leading underscore, NOT
two, unless you're perfectly sure you know what you're doing.
> if not p.allowAccess(name):
> raise AccessError
> return getattr(
> super(Proxy, self).__getattribute__('__original'),
> name
> )
>
> I'm not sure I see a way to apply the approach setSpecialAttr uses
> here...
> Though, maybe Proxy could iterate over the attributes of 'original',
That's pretty hard -- 'original' could get its attributes dynamically
too, in its own __getattr__ or __getattribute__ -- so, for perfect
generality, you can't assume the ability to iterate over ALL of them.
Still, you surely CAN iterate over all SPECIAL names, since they're
a finite, predefined set. This should suffice, since non-special
names can be handled as above. You'll get into hairy problems if
you want to deal with 'original' potentially being a classic class
instance as well as a new-style class instance, etc, etc, but the
"custom class" approach (perhaps different custom classes in different
cases) can probably help anyway.
E.g., a QAD approach might be (untested):
class Proxy(object):
def __new__(cls, original, policy=PromiscuousPolicy()):
class custom_subclass(original.__class__):
def __new__(cls): return object.__new__(cls)
def __init__(cls): pass
def __getattribute__(self, name):
if not policy.allowAccess(name):
raise AccessError, name
return original.__class__.__getattribute__(original, name)
return custom_subclass()
Note that this trick exploits lexical scopes to access original and
policy, thus avoiding clashes of __getattribute__ semantics. The
overrides of __new__ and __init__ in the custom_subclass should ensure
against unwelcome "multiple initialization" attempts. No doubt there
is much play needed to make this right, but I hope the general idea
can still be of use.
Alex