Can __iter__ be used as a classmethod?
Bjorn Pettersen
instance) using a regular for loop: "for item in MyClass:...". I can't
seem to get it to work however. I've got the follwing class:
class ClassIter(object):
vals = ['a', 'b', 'c']
def __iter__(cls):
print 'iter'
return cls.items()
__iter__ = classmethod(__iter__)
def items(cls):
print 'items'
for item in cls.vals:
yield item
items = classmethod(items)
if I say:
for x in ClassIter.items(): ...
it works and 'items' is printed out once as expected. If I change it to:
for x in ClassIter: ...
it fails with "TypeError: iteration over non-sequence" on that line, and
'iter' is never printed. Finally, if I change it to:
for x in ClassIter.__iter__(): ...
it works (but is not very useful :-)
I've read LibRef 2.2.5 and also RefMan 3.3, and I can't see any
limitations mentioned on which objects can implement __iter__(), or
other __xx__ methods (I haven't found any that work so far). I did
notice "...if a class defines a method named __getitem__(), and x is an
instance of this class, then x[i]...", but I read that as saying "it
definitely works for instance", and not "this only works for instances"
<wink>:
Just for fun I tried a couple more, with the same result.
get = classmethod(__getitem__) -> (vanilla getitem) SystemError 999:
bad argument to internal function
:-)
def adder(cls, other):
return `cls` + `other`
__add__ = classmethod(adder) -> TypeError: unsupported operand
types
for +: 'type' and 'type'
add = classmethod(adder) -> works.
Is this a bug or am I missing something (as far as I can tell, I can't
get around this with metaclasses either...)
-- bjorn
Lulu of the Lotus-Eaters
|I would like to be able to traverse the content of a class (not an
|instance) using a regular for loop: "for item in MyClass:...".
If you want to add methods to a *class*, you create the class using a
custom metaclass:
% cat classiter.py
from __future__ import generators
class ClassIter(object):
vals = ['a', 'b', 'c']
class __metaclass__(type):
def __iter__(cls):
print 'iter'
return cls.items()
def items(cls):
print 'items'
for item in cls.vals:
yield item
items = classmethod(items)
for x in ClassIter:
print x
% python classiter.py
iter
items
a
b
c
Yours, Lulu...
--
mertz@ | The specter of free information is haunting the `Net! All the
gnosis | powers of IP- and crypto-tyranny have entered into an unholy
.cx | alliance...ideas have nothing to lose but their chains. Unite
| against "intellectual property" and anti-privacy regimes!
-------------------------------------------------------------------------
Bjorn Pettersen
> "Bjorn Pettersen" <BPett...@NAREX.com> wrote previously:
> |I would like to be able to traverse the content of a class (not an
> |instance) using a regular for loop: "for item in MyClass:...".
>
> If you want to add methods to a *class*, you create the class using a
> custom metaclass:
Thank you very much! Could you also explain why <wink>? (e.g. why can I
make items() a classmethod but not __iter__, and how does
__metaclass__.__iter__ get called from iter(ClassIter)? -- it doesn't
show up when using dir()...)
Just doing my first meta class project since Smalltalk in the early
90's...
-- bjorn
Alex Martelli
>> From: Lulu of the Lotus-Eaters [mailto:me...@gnosis.cx]
>>
>> "Bjorn Pettersen" <BPett...@NAREX.com> wrote previously:
>> |I would like to be able to traverse the content of a class (not an
>> |instance) using a regular for loop: "for item in MyClass:...".
>>
>> If you want to add methods to a *class*, you create the class using a
>> custom metaclass:
>
> Thank you very much! Could you also explain why <wink>? (e.g. why can I
> make items() a classmethod but not __iter__, and how does
> __metaclass__.__iter__ get called from iter(ClassIter)? -- it doesn't
> show up when using dir()...)
In Python 2.2 and later (excepting, for backwards compatibility,
'classic classes') an operation always looks up the special methods
of interest on the CLASS (aka type) of the object being operated on.
The class of a class object is also conventionally known as its
"metaclass". But the relationship of a class to its metaclass is
absolutely identical to that of any other instance to its class
(again excepting the muddied semantics of "classic classes" and
their instances, which need to stay around for backwards compat).
So, iter(x) ALWAYS attempts calling type(x).__iter__(x), for
ANY x (excluding classic-class instances).
OTOH, explicitly accessing an attribute (including a callable
attribute -- a method) on any object x generally gets the attribute
on the object x itself (unless descriptors -- such as properties --
or fancy __getattribute__ overrides in type(x) dictate otherwise).
If you just forget the existence of "classic classes", everything
else is perfectly regular and predictable -- working with a class
and a custom metaclass is no different than working with any
other instance and that instance's type (aka class).
Alex
Giovanni Bajo
"Lulu of the Lotus-Eaters" <me...@gnosis.cx> ha scritto nel messaggio
news:mailman.1046733846...@python.org...
> "Bjorn Pettersen" <BPett...@NAREX.com> wrote previously:
> |I would like to be able to traverse the content of a class (not an
> |instance) using a regular for loop: "for item in MyClass:...".
> If you want to add methods to a *class*, you create the class using a
> custom metaclass:
So, what is classmethod exactly doing? Looking at the example snippet, it
seems like it added a new method "items()" to the class.
Giovanni
Samuele Pedroni
"Alex Martelli" <al...@aleax.it> ha scritto nel messaggio
news:dv%8a.2507$pG1....@news1.tin.it...
>
> In Python 2.2 and later (excepting, for backwards compatibility,
> 'classic classes') an operation always looks up the special methods
> of interest on the CLASS (aka type) of the object being operated on.
>
this is true (only exception is partially the static special method
__new__), OTOH looking at what the OP was trying to do it is seems that:
- the support for both classmethods (ignored wrt to special methods) and
metatypes with their methods can be confusing
- people tend still to think in terms of the old model (that of classics
classes) which was purely a lookup-based one, so I presume they imagine
iter() as something like:
def iter(x):
if hasattr(x,'__iter__'):
iterator = x.iter()
# check wheter iterator is such ...
return iterator
else: # seq case
...
but this absolutely not the case as you have explained (apart for __iter__
for classics classes).
Same for __add__:
>>> class C: pass
...
>>> c=C()
>>> c.__add__ = lambda x: "result=c+x"
>>> c+1
'result=c+x'
>>> class N(object): pass
...
>>> n=N()
>>> n.__add__ = lambda x: "result=n+x"
>>> n+1
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: unsupported operand types for +: 'N' and 'int'
>>> del n.__add__
>>> def Nadd(self,x): return "result=N+X"
...
>>> N.__add__=Nadd
>>> n+1
'result=N+X'
Alex Martelli
It does -- it adds a (class) method to the class object. "Lulu"'s
assertion should pedantically be rephrased as: "if you want to add
SPECIAL methods that WILL AFFECT OPERATIONS ON a *class* [as opposed
to operations on *instances* of the class], you create the class
using a custom metaclass".
E.g., if X is a class, X.__iter__ affects iterations on INSTANCES
of X, *NOT* iterations on X itself; the latter are instead affected
by type(X).__iter__, if any. type(X) is X's metaclass -- unless
it's a custom one, type(X) will be the built-in named 'type', which
has no special method __iter__.
This doesn't apply to "ciassic classes", which, for backwards
compatibility reasons, need to keep old, muddled semantics, and
can't really have custom metaclasses anyway (their metaclass is
types.ClassType, and it doesn't fit the new, regular model).
Attributes that you look up on a specific object are not the
same thing as special-methods that are looked up (on the TYPE
of the object -- except for classic classes) by operations
that you perform on the object. Normal method calls use normal
attribute lookup.
Alex
Giovanni Bajo
"Alex Martelli" <al...@aleax.it> ha scritto nel messaggio
news:n519a.3620$zo2.1...@news2.tin.it...
> E.g., if X is a class, X.__iter__ affects iterations on INSTANCES
> of X, *NOT* iterations on X itself;
> the latter are instead affected
> by type(X).__iter__, if any. type(X) is X's metaclass -- unless
> it's a custom one, type(X) will be the built-in named 'type', which
> has no special method __iter__.
Let's talk about new-style classes only (otherwise I could blow a fuse
before I understand something ;)
Is it possible to add (class) methods to class objects by defining them in a
custom metaclass, or is the metaclass used only to lookup special methods
(__iter__, etc.) for operations executed on class objects? Since I define
methods of the instances within the class (object's type) definition, I
would think that I might define methods of the class objects within the
metaclass (class' type) definition.
Given:
def A(object):
def __iter__(cls):
pass
__iter__ = classmethod(__iter__)
how do you classify (word joke not intended) the (rebound) __iter__? Is it a
class method of class A? Is it still used for iterations on instances of A?
How does the classmethod() affected the type and the semantic of __iter__?
>Attributes that you look up on a specific object are not the
>same thing as special-methods that are looked up (on the TYPE
>of the object -- except for classic classes) by operations
>that you perform on the object. Normal method calls use normal
>attribute lookup.
Probably I'm confused because to me:
class B(object):
def f(self):
pass
def __iter__(self):
pass
f and __iter__ seems to be of the same kind (type). But you say that B().f()
is doing a lookup on the specific instance (and the lookup resolves to B.f),
while iter(B()) is looked up on the TYPE of the object, but it resolves to
B.__iter__ as well. So, if the lookup are performed on different entities,
why do they resolve in the same way? Is it just because they are not rebound
after the instance is created? Would that mean that rebinding B().__iter__
has no effect since B.__iter__ is always looked up when iterating over
instances of B(), while B().f() can be freely rebound?
Giovanni Bajo
Alex Martelli
> "Alex Martelli" <al...@aleax.it> ha scritto nel messaggio
> news:n519a.3620$zo2.1...@news2.tin.it...
>
>> E.g., if X is a class, X.__iter__ affects iterations on INSTANCES
>> of X, *NOT* iterations on X itself;
>> the latter are instead affected
>> by type(X).__iter__, if any. type(X) is X's metaclass -- unless
>> it's a custom one, type(X) will be the built-in named 'type', which
>> has no special method __iter__.
>
> Let's talk about new-style classes only (otherwise I could blow a fuse
> before I understand something ;)
Sure, let's -- no "classic classes" in the following (and, feel free
to switch the discussion to it.comp.lang == with [Python] at the start
of the subject == if discussing this in Italian will make it easier for
you... otherwise, keeping it on c.l.py does seem preferable).
> Is it possible to add (class) methods to class objects by defining them in
> a custom metaclass, or is the metaclass used only to lookup special
> methods (__iter__, etc.) for operations executed on class objects? Since I
Just as any instance "inherits" attributes from its class, so does any class
"inherit" attributes from its metaclass -- there is no difference between
the two cases, nor between callable and non-callable attributes. Therefore,
for example:
[alex@lancelot bin]$ python
Python 2.3a2 (#1, Feb 21 2003, 10:22:48)
[GCC 3.2 (Mandrake Linux 9.0 3.2-1mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class meta(type):
... def amethod(*args): print 'a method', args
...
>>> class aclass:
... __metaclass__ = meta
...
>>> aclass.amethod()
a method (<class '__main__.aclass'>,)
However, the "instance-from-type inheritance" is not transitive:
>>> aninstance = aclass()
>>> aninstance.amethod()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'aclass' object has no attribute 'amethod'
>>>
See at the very end of the post a paragraph explaining (I hope) why
it's a very good thing that such inheritance is not transitive AND
that operations look up special-named attributes on the types of their
arguments, not on the arguments themselves -- the case of __call__.
I hope that such motivations/rationales make things easier to
understand and therefore to retain and to use in practice.
> define methods of the instances within the class (object's type)
> definition, I would think that I might define methods of the class objects
> within the metaclass (class' type) definition.
Sure, you can do that if you wish. You MUST do that for special
methods that you want to be automatically invoked for operations
on the class object, you CAN do that for methods that you want to
be invokable on the class-object but not on its instance-objects.
> Given:
>
> def A(object):
I assume you rather mean:
class A(object):
and will proceed based on this assumption.
> def __iter__(cls):
> pass
> __iter__ = classmethod(__iter__)
>
> how do you classify (word joke not intended) the (rebound) __iter__? Is it
> a class method of class A? Is it still used for iterations on instances of
> A? How does the classmethod() affected the type and the semantic of
> __iter__?
iter(A()) will do its best, but it will get a None and therefore fail with
a TypeError: iteration over non-sequence. But if the A.__iter__ call did
return an iterator, as it should, then there would be no problem using it
for iterations over instances of A:
>>> class A(object):
... def __iter__(cls):
... return iter((1,2,3))
... __iter__ = classmethod(__iter__)
...
>>> for xx in A(): print xx
...
1
2
3
>>>
The problem is not with A.__iter__ being a classmethod (and thus getting
as its automatic first argument the class object rather than an instance
of the class), it's with A.__iter__ returning None, which is not a valid
iterator object (None has no method named 'next' callable without arguments
etc etc).
>>Attributes that you look up on a specific object are not the
>>same thing as special-methods that are looked up (on the TYPE
>>of the object -- except for classic classes) by operations
>>that you perform on the object. Normal method calls use normal
>>attribute lookup.
>
> Probably I'm confused because to me:
>
> class B(object):
> def f(self):
> pass
> def __iter__(self):
> pass
>
> f and __iter__ seems to be of the same kind (type). But you say that
They are! And both return None (which makes B instances not iterable
on, but that's another issue). It's easy to check that they are
indeed the same type, and both equally well callable (and bereft of
any effect) as long as you use identical syntax to call both:
>>> class B(object):
... def f(self): pass
... def __iter__(self): pass
...
>>> type(B.f)
<type 'instancemethod'>
>>> type(B.__iter__)
<type 'instancemethod'>
>>> B().f()
>>> B().__iter__()
>>>
> B().f() is doing a lookup on the specific instance (and the lookup
> resolves to B.f), while iter(B()) is looked up on the TYPE of the object,
Sure! iter(B()) is NOT the same thing as B().__iter__() -- that's
all there is to it.
Many built-ins work similarly. abs(B()) looks up type(B).__abs__ --
it's NOT the same thing as B().__abs__, which looks up __abs__ in
the INSTANCE object. Check this, maybe it's easier to see with
abs than with iter perhaps?
>>> class Z(object):
... def __init__(self):
... self.__abs__ = 23
... def __abs__(self):
... return 45
...
>>> print Z.__abs__
<unbound method Z.__abs__>
>>> print Z().__abs__
23
>>> print abs(Z())
45
>>>
> but it resolves to B.__iter__ as well. So, if the lookup are performed on
> different entities, why do they resolve in the same way? Is it just
Because every instance "inherits" attributes from its class, unless
the instance has re-bound the name specifically as per-instance attribute
names; thus when you look up any attribute in X() you will often get the
same thing as by looking it up in X, though there are many exceptions:
-- if the instance X() has specifically bound that attribute name in
itself,
-- for methods bound in X (or inherited by X from base classes),
where you get a different type of object depending on where you
look it up (because of descriptor mechanics...),
-- for other names bound to special descriptors in X (e.g., property),
-- for attributes that X itself inherits from type(X) -- inheritance
is not transitive,
and the like.
> because they are not rebound after the instance is created? Would that
That's one condition (rebinding an attribute name on an instance
specifically does affect lookups for that name on that instance).
> mean that rebinding B().__iter__ has no effect since B.__iter__ is always
> looked up when iterating over instances of B(), while B().f() can be
> freely rebound?
It's not true that rebinding name '__iter__' on an instance has NO
effect -- it does affect calls DIRECTLY to thatinstance.__iter__, AND
iterations on INSTANCES of that instance (if, of course, that instance
is usable and gets used as a type. AKA class). But it has no effect
on ITERATIONS on that instance, or calls to iter(...) with that
instance as their argument.
The freedom you have to rebind name '__iter__' on an instance is
exactly the same as you have to rebind name 'f' on that instance.
The effects are the same *when you fetch those two attributes from
the instance in the same, identical way* -- say the instance is
bound to name 'x', then:
-- IF you have bound names 'f' and/or '__iter__' directly on x,
then by fetching x.f and/or x.__iter__ you get exactly the
object you had bound to that name on x;
-- otherwise, if in type(x) you had bound 'f' and/or 'iter' to
objects of type 'function', then by fetching x.f and/or
x.__iter__ you get a bound-method whose im_self attribute
is x and whose im_func attribute is the function object you
had bound to that name in type(x)
This doesn't affect what happen when you call iter(x) or have
it implicitly called for you by looping e.g. with "for a in x:".
In THIS case, name 'f' does not enter the picture, however you
may have bound, re-bound, or failed to bind it. Name '__iter__'
DOES enter the picture, and it's looked up in type(x).
It's just the same for builtin such as abs, hash, len, ... and
the correlated special-methods __abs__, __hash__, __len__, ....
The built-in MAY sometimes do a bit more work and/or checks on
your behalf -- some built-ins (and operations expressed in
other ways than by built-ins) fall back to some alternative
solution if appropriate (e.g. hash(x) falls back to id(x)
if type(x) has none of '__hash__', '__eq__', '__cmp__' as
attribute names) -- but apart from such potential extras, the
normal operation of these built-ins is to look up the special
method on the type (aka class) of their argument [always
excepting arguments that are instances of classic-classes,
as we said we would right at the start of this post!].
Besides built-ins, there are quite a few Python operations
that (as a part or whole of their job) look up attributes with
special names on the types of their arguments. Consider for
example calling an object:
x()
this is affected by type(x).__call__. So, for example, when
you call a class x, this executes the __call__ special method
(if any) of type(x), x's metaclass, quite independently of
whether class x itself defines a __call__ special method (THAT
one would be used when INSTANCES of x are called). You would
not expect that calling a class (to instantiate it) is affected
by whether the INSTANCES of that class are callable, right? This
may help explain why inheritance is not transitive AND why the
operations and built-ins access special attributes on the TYPES
of their arguments, not on their arguments themselves.
Alex
Thomas Heller
> > a custom metaclass, or is the metaclass used only to lookup special
> > methods (__iter__, etc.) for operations executed on class objects? Since I
>
> Just as any instance "inherits" attributes from its class, so does any class
> "inherit" attributes from its metaclass -- there is no difference between
> the two cases, nor between callable and non-callable attributes.
It should be noted that this (since it also works in Python extensions
in C) can be used to create classmethods in extensions in a way
compatible with 2.2 and 2.3. Python 2.2 does not have the METH_CLASS
flag!
Thomas
Michele Simionato
> "Alex Martelli" <al...@aleax.it> ha scritto nel messaggio
> news:dv%8a.2507$pG1....@news1.tin.it...
> >
> > In Python 2.2 and later (excepting, for backwards compatibility,
> > 'classic classes') an operation always looks up the special methods
> > of interest on the CLASS (aka type) of the object being operated on.
> >
>
> this is true (only exception is partially the static special method
> __new__), OTOH looking at what the OP was trying to do it is seems that:
>
> - the support for both classmethods (ignored wrt to special methods) and
> metatypes with their methods can be confusing
>
Is there any good reason for supporting classmethods in Python ?
Looking at Guido's essay it seems they have been introduced by mistake
(trying to implement __new__ as a classmethod).
It seems to me I could always use a regular method instead of
a classmethod, by accessing the class trough self.__class__ or
type(self), therefore classmethods only provide a very small amount
of syntactical sugar.
Actually they seems to me an unessential complication to the language,
expecially because they are easily confused with the methods in the
metaclass.
On the other hand, I like very much static methods.
However, I noticed that
>>> isinstance(object.__new__,staticmethod)
False
Therefore if I try to retrieve all the staticmethods from a class I will
miss __new__. That's unfortunate, even if I understand this is done in order
to avoid to write
__new__=staticmethod(__new__)
each time I redefine __new__. In a sense, __new__ is an "implicit" static
method.
explicit-is-better-than-implicit-and-no-special-case-is-special-enough-ly
yours,
Michele
Giovanni Bajo
> Sure, let's -- no "classic classes" in the following (and, feel free
> to switch the discussion to it.comp.lang == with [Python] at the start
> of the subject == if discussing this in Italian will make it easier for
> you... otherwise, keeping it on c.l.py does seem preferable).
Is my english that bad? :)
It's ok to keep it in here, thanks anyway for the offer.
Your reply was pretty clear and exhaustive. I think the main problem is that
I was thinking of iter(B()) as a different way of calling B().__iter__
(usually 'inherited' from B.__iter__), and not type(B).__iter__. Everything
makes more sense now.
I still have a small doubt by classmethod(). Let's consider this:
class A:
def f(cls):
pass
f = classmethod(f)
def g(self):
pass
a = A()
Now, in my current understanding, the differences between A.f and A.g are:
1) A.f() is a valid call (and the function will receive class A as
parameter), while A.g() will raise a runtime exception (unbound method).
A.g(a) is correct because it is equivalent to a.g().
2) a.f() is a valid call (and the function will receive class A, aka
type(a), as parameter); a.g() is a valid call as well, and the function will
receive a as parameter
Also, both are attributes of A and not of a. They can be used on a because
they are automatically looked up on type(a), but they belong to A.
Another source of confusion in my mind is that type(A.g) is called "instance
method", because it must be bound to an instance of the class, to become
'callable'. But it does not 'belong' to the instance, it is actually defined
in A.__dict__, not in A().__dict__.
> > mean that rebinding B().__iter__ has no effect since B.__iter__ is
always
> > looked up when iterating over instances of B(), while B().f() can be
> > freely rebound?
>
> It's not true that rebinding name '__iter__' on an instance has NO
> effect --
Yes, it should have been 'has no effect with respect to iterating over B
instances'.
> it does affect calls DIRECTLY to thatinstance.__iter__, AND
> iterations on INSTANCES of that instance (if, of course, that instance
> is usable and gets used as a type. AKA class). But it has no effect
> on ITERATIONS on that instance, or calls to iter(...) with that
> instance as their argument.
So, if I rebind B().__iter__, Python will use the rebound function when
calling "for x in B().__iter__()" (assuming that 'B()' always refers to the
same instance of course) but it will not change "for x in B", which is
semantically the same of "for x in iter(B)", or "for x in
type(B).__iter__()" (maybe modulo obscure situations that I can ignore for
now).
> [snip]
> This
> may help explain why inheritance is not transitive AND why the
> operations and built-ins access special attributes on the TYPES
> of their arguments, not on their arguments themselves.
Yes, I understand very well why 'inheritance' cannot be transitive. By the
way, it seems that the word 'inheritance' has been abused in this context:
this is not the normal inheritance as in class A(B), but something different
(A() will lookup attributes from A if they have not been rebound, and A will
lookup attributes from A.__metaclass__ -- more generically, A will lookup
attributes in type(A)). Is there a more correct word for this?
'meta-inheritance'? :)
Thanks again for your time
Giovanni Bajo
Giovanni Bajo
"Samuele Pedroni" <pedr...@bluewin.ch> ha scritto nel messaggio
news:3e649...@news.bluewin.ch...
> - the support for both classmethods (ignored wrt to special methods) and
> metatypes with their methods can be confusing
I agree. It has been source of big confusion for me as well.
Giovanni Bajo
Giovanni Bajo
"Michele Simionato" <mi...@pitt.edu> ha scritto nel messaggio
news:2259b0e2.03030...@posting.google.com...
> It seems to me I could always use a regular method instead of
> a classmethod, by accessing the class trough self.__class__ or
> type(self), therefore classmethods only provide a very small amount
> of syntactical sugar.
Actually, they let you also call the method without any instance of the
class (they do not need to be bound to any instance).
> Actually they seems to me an unessential complication to the language,
> expecially because they are easily confused with the methods in the
> metaclass.
Yes:
class A(object):
x = 10
class __metaclass__(type):
def f(cls):
print "Hello from", cls, ", value is", cls.x
def g(cls):
print "Hello from", cls, ", value is", cls.x
g = classmethod(g)
>>> A.g()
Hello from <class '__main__.A'> , value is 10
>>> A.f()
Hello from <class '__main__.A'> , value is 10
So for basic usage, you accomplished the same: you have a method in class A
that you can call without any instance of A, and can work on the attributes
of the class.
The only difference I can see is:
>>> a = A()
>>> a.g()
Hello from <class '__main__.A'> , value is 10
>>> a.f()
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
AttributeError: 'A' object has no attribute 'f'
This is because of the lack of transitivity in the 'inheritance' from types,
as explained by Alex in another post in this thread.
So, what are the other differences (if any)? Why do classmethods exist?
Giovanni Bajo
Samuele Pedroni
>
> >>> isinstance(object.__new__,staticmethod)
> False
>
> Therefore if I try to retrieve all the staticmethods from a class I will
> miss __new__. That's unfortunate, even if I understand this is done in
order
> to avoid to write
>
> __new__=staticmethod(__new__)
>
> each time I redefine __new__. In a sense, __new__ is an "implicit" static
> method.
no, object.__new__ is a special case:
>>> class C(object):
... def __new__(cls):
... print "C__new__"
... return object.__new__(cls)
...
>>> c=C()
C__new__
>>> C.__new__
<function __new__ at 0x00795F40>
>>> C.__dict__['__new__']
<staticmethod object at 0x00792010>
>>> isinstance(_,staticmethod)
1
>>> isinstance(object.__dict__['__new__'],staticmethod)
0
for normal new style classes, __new__ lives as staticmethod in the class
__dict__ but notice it is unwrapped to a function on lookup.
That's true for general staticmethods:
>>> class C(object):
... def f():
... return 'f'
... f=staticmethod(f)
...
>>> C.f
<function f at 0x00794220>
>>> C.__dict__['f']
<staticmethod object at 0x007941F0>
regards.
Chermside, Michael
See also the recently released article:
http://www-106.ibm.com/developerworks/library/l-pymeta.html
It's well-written with clear, simple examples and may be of
great benefit to someone trying to understand these things.
-- Michael Chermside
Alex Martelli
>
> "Michele Simionato" <mi...@pitt.edu> ha scritto nel messaggio
> news:2259b0e2.03030...@posting.google.com...
>
>> It seems to me I could always use a regular method instead of
>> a classmethod, by accessing the class trough self.__class__ or
>> type(self), therefore classmethods only provide a very small amount
>> of syntactical sugar.
>
> Actually, they let you also call the method without any instance of the
> class (they do not need to be bound to any instance).
Yes, that's difference #1.
>> Actually they seems to me an unessential complication to the language,
>> expecially because they are easily confused with the methods in the
>> metaclass.
...
> The only difference I can see is:
>
>>>> a = A()
>>>> a.g()
> Hello from <class '__main__.A'> , value is 10
>>>> a.f()
> Traceback (most recent call last):
> File "<interactive input>", line 1, in ?
> AttributeError: 'A' object has no attribute 'f'
>
> This is because of the lack of transitivity in the 'inheritance' from
> types, as explained by Alex in another post in this thread.
And that's difference #2.
> So, what are the other differences (if any)? Why do classmethods exist?
Aren't these reasons enough? How else would you code a method that
you can indifferently call either on any instance or on the class
object itself? And _having_ to write a custom metaclass as the
only way to get classmethods would be somewhat of an overkill.
Whether static methods (which to many are familiar from C++ or Java)
and class methods (which have some pluses and some minuses, and to
some are familiar from Smalltalk) need to exist at all is moot -- C++'ers
and Javaites have been clamoring for them for a long time, not wanting
to use free-standing (module top-level) functions instead (and in
effect, a free-standing function could not be written to receive the
classobject automatically -- and custom metaclasses where a nightmare
before Python 2.2).
Overall I'm slightly happier to have them in Python than not, but I
do realize they contribute to "featuritis" -- Python does keep getting
richer with every release, as it's always easier to give in to the
pressure of those who DO want the feature (and can muster good direct
arguments and use-cases for it), when the only negative is "the more
feechurs we keep piling up, the less simple our beloved language". Sigh.
Alex
Alex Martelli
>
> "Alex Martelli" <al...@aleax.it> ha scritto nel messaggio
> news:b42ev...@enews4.newsguy.com...
>
>> Sure, let's -- no "classic classes" in the following (and, feel free
>> to switch the discussion to it.comp.lang == with [Python] at the start
>> of the subject == if discussing this in Italian will make it easier for
>> you... otherwise, keeping it on c.l.py does seem preferable).
>
> Is my english that bad? :)
Not at all! Sorry for the unintended implication...
> It's ok to keep it in here, thanks anyway for the offer.
>
> Your reply was pretty clear and exhaustive. I think the main problem is
> that I was thinking of iter(B()) as a different way of calling
> B().__iter__ (usually 'inherited' from B.__iter__), and not
> type(B).__iter__. Everything makes more sense now.
OK. It _used_ to be that way with classic classes, btw, but that
just didn't generalize well.
> I still have a small doubt by classmethod(). Let's consider this:
>
> class A:
Better inherit it from object for clarity (or set __metaclass__ to type),
although offhand I think all you say below holds for classic classes too,
we did agree not to consider that source of potential complication anyway.
> def f(cls):
> pass
> f = classmethod(f)
>
> def g(self):
> pass
>
> a = A()
>
> Now, in my current understanding, the differences between A.f and A.g are:
>
> 1) A.f() is a valid call (and the function will receive class A as
> parameter), while A.g() will raise a runtime exception (unbound method).
Yes.
> A.g(a) is correct because it is equivalent to a.g().
Yes again.
> 2) a.f() is a valid call (and the function will receive class A, aka
> type(a), as parameter); a.g() is a valid call as well, and the function
> will receive a as parameter
And yes to this too.
> Also, both are attributes of A and not of a. They can be used on a because
> they are automatically looked up on type(a), but they belong to A.
Correct.
> Another source of confusion in my mind is that type(A.g) is called
> "instance method", because it must be bound to an instance of the class,
> to become 'callable'. But it does not 'belong' to the instance, it is
> actually defined in A.__dict__, not in A().__dict__.
Yes, but the point is: type(A.g) is type(A().g) -- bound and unbound
methods are the SAME type, only the value of their im_self attribute
distinguishes them (None for the unbund, instance for the bound one).
>> > mean that rebinding B().__iter__ has no effect since B.__iter__ is
> always
>> > looked up when iterating over instances of B(), while B().f() can be
>> > freely rebound?
>>
>> It's not true that rebinding name '__iter__' on an instance has NO
>> effect --
>
> Yes, it should have been 'has no effect with respect to iterating over B
> instances'.
Yep, this way it's correct.
>> it does affect calls DIRECTLY to thatinstance.__iter__, AND
>> iterations on INSTANCES of that instance (if, of course, that instance
>> is usable and gets used as a type. AKA class). But it has no effect
>> on ITERATIONS on that instance, or calls to iter(...) with that
>> instance as their argument.
>
> So, if I rebind B().__iter__, Python will use the rebound function when
> calling "for x in B().__iter__()" (assuming that 'B()' always refers to
> the same instance of course) but it will not change "for x in B", which is
> semantically the same of "for x in iter(B)", or "for x in
> type(B).__iter__()" (maybe modulo obscure situations that I can ignore for
> now).
Right, I think, but you should give names to things for clarity, e.g.:
after
x = B()
rebinding x.__iter__ does effect any further EXPLICIT use of x.__iter__,
but NOT iter(x) and "for a in x:" (much less, iter(B) and "for a in B:",
i.e. iteration on the class object itself).
>> [snip]
>> This
>> may help explain why inheritance is not transitive AND why the
>> operations and built-ins access special attributes on the TYPES
>> of their arguments, not on their arguments themselves.
>
> Yes, I understand very well why 'inheritance' cannot be transitive. By the
> way, it seems that the word 'inheritance' has been abused in this context:
Maybe, but it's what Lutz uses in "Learning Python" and I do think
it's not bad usage. A lookup "x.goo" delegates to type(X).goo when
goo does not name an attribute in x itself, "just as" a lookup of
type(X).goo delegates to type(X).__bases__[0].goo &c -- well, it IS
deucedly close.
> this is not the normal inheritance as in class A(B), but something
> different (A() will lookup attributes from A if they have not been
> rebound, and A will lookup attributes from A.__metaclass__ -- more
> generically, A will lookup attributes in type(A)). Is there a more correct
> word for this? 'meta-inheritance'? :)
"inheritance from type(x)" is admittedly not quite the same as
"inheritance from my __bases__" (for example, the latter DOES apply
perfectly well to automatic lookup of special methods, and it IS
transitive while the former isn't). But both mechanisms boil down,
90% or so, to delegating lookup of attribute names not found in
the object itself, so that using the same term for both does not
seem an "abuse" to me -- it seems quite sensible, though having a
way to distinguish the subtle differences would surely also help.
> Thanks again for your time
You're welcome!
Alex
Bjorn Pettersen
>
[...]
> Is there any good reason for supporting classmethods in Python ?
Yes, classmethods can do everything staticmethods can, plus
polymorphism. E.g.:
>>> class meta(type):
... def call_f(cls): cls.f()
...
>>> class A(object):
... __metaclass__ = meta
... def f(cls): print 'A.f'
... f = classmethod(f)
...
>>> class B(object):
... __metaclass__ = meta
... def f(cls): print 'B.f'
... f = classmethod(f)
...
>>> A.call_f()
A.f
>>> B.call_f()
B.f
>>>
Digressing: It never occurred to me that assignment to __metaclass__ was
defining the class of your class until it was pointed out earlier today.
That made everything much clearer :-)
-- bjorn
Michele Simionato
> From: Michele Simionato [mailto:mi...@pitt.edu]
> [...]
> Is there any good reason for supporting classmethods in Python ?
> Yes, classmethods can do everything staticmethods can, plus
>polymorphism. E.g.:
>>> class meta(type):
... def call_f(cls): cls.f()
...
>>> class A(object):
... __metaclass__ = meta
... def f(cls): print 'A.f'
... f = classmethod(f)
...
>>> class B(object):
... __metaclass__ = meta
... def f(cls): print 'B.f'
... f = classmethod(f)
...
>>> A.call_f()
A.f
>>> B.call_f()
B.f
>>>
This code does not make a good example for classmethods, since
it can be rewritten by using staticmethods (the knowledge of cls
is not used in f).
Anyway, I can emulate classmethods with regular methods in the
metaclass:
>>> class meta(type):
... def call_f(cls): cls.f() # a perverse way of calling f
... def f(cls): print "%s.f" % cls.__name__
>>> class A(object):
... __metaclass__ = meta
...
>>> class B(A): pass #notice: B inherits the metaclass of A
...
>>> A.call_f()
A.f
>>> B.call_f()
B.f
> Digressing: It never occurred to me that assignment to __metaclass__ was
> defining the class of your class until it was pointed out earlier today.
> That made everything much clearer :-)
>
> -- bjorn
You may find interesting this article:
http://www-106.ibm.com/developerworks/linux/library/l-pymeta.html
--
Michele Simionato - Dept. of Physics and Astronomy
210 Allen Hall Pittsburgh PA 15260 U.S.A.
Phone: 001-412-624-9041 Fax: 001-412-624-9163
Home-page: http://www.phyast.pitt.edu/~micheles/
Michele Simionato
> <snip>
> I still have a small doubt by classmethod(). Let's consider this:
>
> class A:
> def f(cls):
> pass
> f = classmethod(f)
>
> def g(self):
> pass
>
> a = A()
>
> Now, in my current understanding, the differences between A.f and A.g are:
>
> 1) A.f() is a valid call (and the function will receive class A as
> parameter), while A.g() will raise a runtime exception (unbound method).
> A.g(a) is correct because it is equivalent to a.g().
> 2) a.f() is a valid call (and the function will receive class A, aka
> type(a), as parameter); a.g() is a valid call as well, and the function will
> receive a as parameter
>
> Also, both are attributes of A and not of a. They can be used on a because
> they are automatically looked up on type(a), but they belong to A.
> Another source of confusion in my mind is that type(A.g) is called "instance
> method", because it must be bound to an instance of the class, to become
> 'callable'. But it does not 'belong' to the instance, it is actually defined
> in A.__dict__, not in A().__dict__.
>
> By the>
> way, it seems that the word 'inheritance' has been abused in this context:
> this is not the normal inheritance as in class A(B), but something different
> (A() will lookup attributes from A if they have not been rebound, and A will
> lookup attributes from A.__metaclass__ -- more generically, A will lookup
> attributes in type(A)). Is there a more correct word for this?
> 'meta-inheritance'? :)
>
> Thanks again for your time
>
> Giovanni Bajo
I think you have got all right. It is true that the current terminogy is
misleading at times :-(
Michele
Michele Simionato
Unfortunately, it is a little short and I think it does not make
full justice to the subject. But we had a limit of 2500 words :-(
Hope it will be useful to somebody, anyway ;)
Michele
Alex Martelli
...
> Anyway, I can emulate classmethods with regular methods in the
> metaclass:
>
>>>> class meta(type):
> ... def call_f(cls): cls.f() # a perverse way of calling f
> ... def f(cls): print "%s.f" % cls.__name__
>
>>>> class A(object):
> ... __metaclass__ = meta
> ...
>>>> class B(A): pass #notice: B inherits the metaclass of A
> ...
>
>>>> A.call_f()
> A.f
>>>> B.call_f()
> B.f
Not quite -- try A().call_f()... would work just fine if call_f
were a static method, but won't work for a metaclass method.
Not a decisive advantage, but there can be some elegance in
being able to call A.f() and A().f() similarly.
Alex
Samuele Pedroni
"Alex Martelli" <al...@aleax.it> ha scritto nel messaggio
news:4B89a.6715$zo2.2...@news2.tin.it...
> > So, what are the other differences (if any)? Why do classmethods exist?
>
> Aren't these reasons enough? How else would you code a method that
> you can indifferently call either on any instance or on the class
> object itself? And _having_ to write a custom metaclass as the
> only way to get classmethods would be somewhat of an overkill.
the existence discussion is moot, they are anyway a possibility of the
descriptors design:
import new
class myclassmethod(object):
__debug = 0
def __init__(self,f):
self.__f = f
def __get__(self,obj,typ=None):
if self.__debug:
print "MYCM__get__",obj,typ
if typ is None:
typ = type(obj)
return new.instancemethod(self.__f,typ,type(typ))
class C(object):
def g(cls):
print cls
g = myclassmethod(g)
class D(C):
def g(cls):
print 'Dg',cls
C.g()
g = myclassmethod(g)
This can be used as an argument to have them explicitely (you can implement
them anyway, so let's get them right once and for all) or to not have them
(who need them can cook them up).
It is worth to notice that once you override one of them (like g in D
overriding g in C) it is impossible to call in a clean way the parent
version (C g) passing the derived class (D), which is what would happen in
case of no overriding (if D would not define g, then D.g() would call g in C
passing D), (so above C.g() obviously is calling g in C passing C). "Let's
call this their price".
Michele Simionato
True, but my point was that this very little convenience does not
justify (IMHO) the introduction of classmethods in the language.
There are two typical cases:
1) I want to be able to call A().f(). Then I can use a regular method
defined in A and inside the method I extract the class with type(self);
2) A is not meant to be instantiated, and I want to use the notation A.f().
Then I can define a regular method in the metaclass.
Then, there is a limit case, where f is defined in the metaclass of A,
but I want to call it from an instance of A. In this case it is enough
to write type(A()).f().
Therefore in any case, simply by typing few characters more, I can emulate
the classmethod effect (if not the precise syntax and some subtility).
If Python had no metaclasses, I would think classmethods are useful; but
since Python has metaclasses, I think they are redundant and a possible
source of confusion.
Just my $0.02,
Michele
Michele Simionato
> import new
>
> class myclassmethod(object):
> __debug = 0
>
> def __init__(self,f):
> self.__f = f
>
> def __get__(self,obj,typ=None):
> if self.__debug:
> print "MYCM__get__",obj,typ
>
> if typ is None:
> typ = type(obj)
>
> return new.instancemethod(self.__f,typ,type(typ))
>
> class C(object):
> def g(cls):
> print cls
>
> g = myclassmethod(g)
>
> class D(C):
> def g(cls):
> print 'Dg',cls
> C.g()
>
> g = myclassmethod(g)
>
> This can be used as an argument to have them explicitely (you can implement
> them anyway, so let's get them right once and for all) or to not have them
> (who need them can cook them up).
In my view this is an argument against them.
> It is worth to notice that once you override one of them (like g in D
> overriding g in C) it is impossible to call in a clean way the parent
> version (C g) passing the derived class (D), which is what would happen in
> case of no overriding (if D would not define g, then D.g() would call g in C
> passing D), (so above C.g() obviously is calling g in C passing C). "Let's
> call this their price".
It seems to me that
class D(C):
def g(cls):
print 'Dg',cls
C.g.im_func(cls)
works under 2.3 (and also super(D,cls).g.im_func(cls) would work).
What surprised me, is that for staticmethods I can extract the wrapped
function with .__get__(None):
>>> def f(x): return x
...
>>> s=staticmethod(f)
>>> s.__get__(None)
<function f at 0x402c2534> # okay, here it is the original f
whereas for classmethods I have a quite different result:
>>> c=classmethod(f)
>>> c.__get__(None)
<bound method type.f of <type 'NoneType'>>
In a sense, staticmethods are really functions, whereas classmethods are
really methods. However, I wonder if there is a direct way to extract f
from the classmethod ?
Michele
Samuele Pedroni
"Michele Simionato" <mi...@pitt.edu> ha scritto nel messaggio
news:2259b0e2.03030...@posting.google.com...
> > This can be used as an argument to have them explicitely (you can
implement
> > them anyway, so let's get them right once and for all) or to not have
them
> > (who need them can cook them up).
>
> In my view this is an argument against them.
I sympathize.
> > It is worth to notice that once you override one of them (like g in D
> > overriding g in C) it is impossible to call in a clean way the parent
> > version (C g) passing the derived class (D), which is what would happen
in
> > case of no overriding (if D would not define g, then D.g() would call g
in C
> > passing D), (so above C.g() obviously is calling g in C passing C).
"Let's
> > call this their price".
>
> It seems to me that
>
> class D(C):
> def g(cls):
> print 'Dg',cls
> C.g.im_func(cls)
Yes, but I would not count that as clean way.
Alex Martelli
...
>> This can be used as an argument to have them explicitely (you can
>> implement them anyway, so let's get them right once and for all) or to
>> not have them (who need them can cook them up).
>
> In my view this is an argument against them.
Pretty weak argument too -- like saying we don't need the built-in
abs because everybody needing it could code it up for themselves!-)
>>>> c=classmethod(f)
>>>> c.__get__(None)
> <bound method type.f of <type 'NoneType'>>
>
> In a sense, staticmethods are really functions, whereas classmethods are
> really methods. However, I wonder if there is a direct way to extract f
> from the classmethod ?
Sure, c.im_func is f. That's how methods work.
Alex
Alex Martelli
...
> True, but my point was that this very little convenience does not
> justify (IMHO) the introduction of classmethods in the language.
> There are two typical cases:
>
> 1) I want to be able to call A().f(). Then I can use a regular method
> defined in A and inside the method I extract the class with type(self);
>
> 2) A is not meant to be instantiated, and I want to use the notation
> A.f(). Then I can define a regular method in the metaclass.
Actually, a very typical use case for class methods (in Smalltalk at
least) is for *factories* -- which of course are only useful for
classes that ARE meant to be instantiated, because they typically
return class instances.
> Then, there is a limit case, where f is defined in the metaclass of A,
> but I want to call it from an instance of A. In this case it is enough
> to write type(A()).f().
But with classmethods, client code doesn't NEED to know what is defined
where, and that is an IMPORTANT advantage.
> Therefore in any case, simply by typing few characters more, I can emulate
> the classmethod effect (if not the precise syntax and some subtility).
And not the smooth polymorphism -- an important issue.
You've already said you do see the usefulness of static methods. OK,
great, so assume static methods ARE in the language -- somebody writes
a nice package where you can call factories as a = A.fact(1,2,3) AND
also later b = a.fact(4,5,6) and live happily... *AS LONG AS* the
factory doesn't need to know exactly what class it's being called on,
for example because A is not designed to be subclasses, and say that's
the case initially. You write client code blissfully unaware, not
NEEDING to be aware, that A.fact is a static method, and of whether
some instances of A may override that method in certain cases -- all
is good in the world.
But now, second release of the same package, it turns out that in
some cases it IS important to know exactly on what class the method
is being called on -- because now you're allowed to subclass A, by
design, and after "class B(A): pass" you get the certainty that
b = B.fact(1,2,3) will generate an instance of B and so later will
b.fact(1,2,3). Pretty common case, by the way, at least if you
have any Smalltalk experience/background.
With classmethods, no problem at all. Without them, not a good place
to be in -- each author of a package in such a situation would need
to perform rather complicated metaclass magic to inject 'fact' in
the metaclass AND in every instance too... or else reimplement
classmethods, which is what sensible ones would do. Over and over
and over again.
And that's the argument FOR classmethods -- avoiding multiple and
redundant reimplementations of the same useful design pattern. What's
so problematic with it for you? Just the fact that you're not
familiar with it? I can see no other reason you'd like *static*
methods but dislike *class* ones, except familiarity!
> If Python had no metaclasses, I would think classmethods are useful; but
> since Python has metaclasses, I think they are redundant and a possible
> source of confusion.
So shall we abrogate unary minus and force everybody to code 0-x
where they'd now code -x? Unary minus is also "redundant and a possible
source of confusion" -=- I've lost count of how many times I've had
to explain to puzzled beginners why -2**2 is -4 (a wart, IMHO)...
Alex
Samuele Pedroni
"Alex Martelli" <al...@aleax.it> ha scritto nel messaggio
news:89G9a.12708$pG1.3...@news1.tin.it...
> Actually, a very typical use case for class methods (in Smalltalk at
> least) is for *factories* -- which of course are only useful for
> classes that ARE meant to be instantiated, because they typically
> return class instances.
>
yes.
On Smalltalk and class methods:
in Smalltak class methods are methods on the meta class, a meta class is
automatically created behind the scene for each class, and class methods are
only accessible through the class as in:
MyClass metamethod...
myinst class metamethod...
but:
myinst metamethod
would cause a message not understood exception.
Greg Chapman
wrote:
>It is worth to notice that once you override one of them (like g in D
>overriding g in C) it is impossible to call in a clean way the parent
>version (C g) passing the derived class (D), which is what would happen in
>case of no overriding (if D would not define g, then D.g() would call g in C
>passing D), (so above C.g() obviously is calling g in C passing C). "Let's
>call this their price".
>
This works for me with 2.22 and 2.3a2:
class C(object):
def g(cls):
print 'C', cls
g = classmethod(g)
class D(C):
def g(cls):
print 'D', cls
super(D, cls).g()
g = classmethod(g)
Both D.g() and D().g() print:
D <class '__main__.D'>
C <class '__main__.D'>
Am I missing some subtlety here?
---
Greg Chapman
Michele Simionato
> Michele Simionato wrote:
> ... I wonder if there is a direct way to extract f
> > from the classmethod ?
>
> Sure, c.im_func is f. That's how methods work.
>
>
I mean, how can I extract the function if I don't have the class?
If I convert a function
>>> def f(x): return x
to a staticmethod
>>> s=staticmethod(f)
I can have back the function via
>>> s.__get__(None)
If I have a classmethod
>>> c=classmethod(f)
I cannot do that. It seems I must define a dummy class
>>> class C(object): pass
bound the classmethod to it
>>> C.c=c
and then retrieve the original function via im_func
>>> C.c.im_func
It there must be a better way!
Michele
Michele Simionato
> <snip>
> You've already said you do see the usefulness of static methods. OK,
> great, so assume static methods ARE in the language -- somebody writes
> a nice package where you can call factories as a = A.fact(1,2,3) AND
> also later b = a.fact(4,5,6) and live happily... *AS LONG AS* the
> factory doesn't need to know exactly what class it's being called on,
> for example because A is not designed to be subclasses, and say that's
> the case initially. You write client code blissfully unaware, not
> NEEDING to be aware, that A.fact is a static method, and of whether
> some instances of A may override that method in certain cases -- all
> is good in the world.
>
> But now, second release of the same package, it turns out that in
> some cases it IS important to know exactly on what class the method
> is being called on -- because now you're allowed to subclass A, by
> design, and after "class B(A): pass" you get the certainty that
> b = B.fact(1,2,3) will generate an instance of B and so later will
> b.fact(1,2,3). Pretty common case, by the way, at least if you
> have any Smalltalk experience/background.
>
> With classmethods, no problem at all. Without them, not a good place
> to be in -- each author of a package in such a situation would need
> to perform rather complicated metaclass magic to inject 'fact' in
> the metaclass AND in every instance too... or else reimplement
> classmethods, which is what sensible ones would do. Over and over
> and over again.
I agree that this is a case where having classmethods can be useful.
Nevertheless, I see this case very rare and I would not see any
problem in implementing classmethods from scratch for this problem
(as Samuele has shown it is very easy).
Alternatively, one could have classmethods defined in a module of the
standard library (and I would like that).
I don't like the fact that they are in the language.
Of course, now there is nothing to do: they are in Python and they will stay
there. I also use them. I do not discuss that they have some convenience,
I am saying that their convenience is little. I posted these messages
because I wanted to make sure that I was not missing an important point
about their usefulness. It seems my impression was right.
> And that's the argument FOR classmethods -- avoiding multiple and
> redundant reimplementations of the same useful design pattern. What's
> so problematic with it for you? Just the fact that you're not
> familiar with it? I can see no other reason you'd like *static*
> methods but dislike *class* ones, except familiarity!
>
I was not familiar with staticmethods neither (I do not come from Java or
C++), nevertheless I am disturbed by the confusion between "metamethods"
(regular methods in the metaclass) and classmethods. If Samuele is
right (I have no experience in Smalltalk) class methods in Smalltalk
are actually metamethods. And this make much more sense to me than
Python classmethods. This is a puristic point of view, of course,
and I know that "practicality beats purity" ;)
I also was confused by the *syntactic* similarity between staticmethods
and classmethods: at the beginning I was induced to think that they
were similar concepts, but actually they are completely different
beasts. I mantain that the introduction of classmethods make harder
the life for newbies without giving a definitive advantage, whereas
on the contrary I would welcome a ternary operator.
Nevertheless I am sure many would think differently ;)
I don't want to enter in such religious issues.
Anyway Alex, thanks for the nice discussion,
Michele
Alex Martelli
...
> Alternatively, one could have classmethods defined in a module of the
> standard library (and I would like that).
> I don't like the fact that they are in the language.
Well, they're in-between -- they ARE defined in a module of the
standard library... it's just that said module is __builtin__, is all.
> Of course, now there is nothing to do: they are in Python and they will
> stay there. I also use them. I do not discuss that they have some
> convenience, I am saying that their convenience is little. I posted these
Personally I think they're more useful than several other built-ins,
such as (say) apply, buffer, coerce, intern, oct, reduce -- all names
I'd rather have in _other_ library modules and not in __builtin__.
> C++), nevertheless I am disturbed by the confusion between "metamethods"
> (regular methods in the metaclass) and classmethods. If Samuele is
I just don't see that confusion.
> I also was confused by the *syntactic* similarity between staticmethods
> and classmethods: at the beginning I was induced to think that they
> were similar concepts, but actually they are completely different
They're perfectly polymorphic -- they're called in exactly the same
ways, so, in terms of interface a certain class offers to its users,
they ARE extremely similar concepts... the class need not even
document whether a given name is a staticmethod or classmethod --
unless it's specifically designed to support being inherited from
with that method being overridden and called-upwards, a very
specific set of circumstances.
If I want client code to be able to call X.foo() and X().foo(),
it's strictly a matter for ME, as the implementer of class X, if
I want to make foo a staticmethod or a classmethod -- I'll use
the latter if I want foo's code to know what exact class (possibly
a subclass of X, rather than X itself) it's being called on.
That doesn't constituite "completely different" for me -- they're
VERY close in most things that matter (which DOESN'T include
calling type(...) on them and the like -- what Pythonically matters
is essentially signature-based polymorphism).
Alex
Samuele Pedroni
"Greg Chapman" <g...@well.com> ha scritto nel messaggio
news:90oe6vccthidi7h44...@4ax.com...
no, good catch.
Although it is an underdocumented feature, e.g. the pure python version of
super here
http://www.python.org/2.2.2/descrintro.html#cooperation
would not work. Here's how the feature was added:
and for a normal method:
>>> class C(object):
... def f(self): pass
...
>>> class D(C): pass
...
>>> D.f
<unbound method D.f>
>>> super(D,D).f
<bound method D.f of <class '__main__.D'>>
super(D,D) appear to misbehave.
Alex Martelli
> Alex Martelli <al...@aleax.it> wrote in message
> news:<FYF9a.16465$zo2.5...@news2.tin.it>...
>> Michele Simionato wrote:
>> ... I wonder if there is a direct way to extract f
>> > from the classmethod ?
>>
>> Sure, c.im_func is f. That's how methods work.
>
> I mean, how can I extract the function if I don't have the class?
Who cares about the class? Just pass ANY type...
> If I convert a function
>
>>>> def f(x): return x
>
> to a staticmethod
>
>>>> s=staticmethod(f)
>
> I can have back the function via
>
>>>> s.__get__(None)
>
> If I have a classmethod
>
>>>> c=classmethod(f)
>
> I cannot do that. It seems I must define a dummy class
You can use almost any argument at your choice:
>>> def f(x): return x
...
>>> f
<function f at 0x4043725c>
>>> c = classmethod(f)
>>> c.__get__(int).im_func
<function f at 0x4043725c>
>>> c.__get__(c).im_func
<function f at 0x4043725c>
>>> c.__get__(23).im_func
<function f at 0x4043725c>
>>> c.__get__('as you like it').im_func
<function f at 0x4043725c>
>>>
You can't use None specifically, but thee is no need
for you to create a dummy class for the purpose, either.
Alex
Michele Simionato
>
> Although it is an underdocumented feature, e.g. the pure python version of
> super here
>
> http://www.python.org/2.2.2/descrintro.html#cooperation
>
> would not work. Here's how the feature was added:
>
> www.python.org/sf/535444
>
> and for a normal method:
>
> >>> class C(object):
> ... def f(self): pass
> ...
> >>> class D(C): pass
> ...
> >>> D.f
> <unbound method D.f>
> >>> super(D,D).f
> <bound method D.f of <class '__main__.D'>>
>
> super(D,D) appear to misbehave.
Another minor inconsistency (IMHO) is that "metamethods" i.e. regular
methods in the metaclass are not retrieved by "dir":
>>> class M(type):
... def mm(cls):
... "This is a metamethod"
... return cls
...
class C(object):
... __metaclass__=M
... def pm(self):
... "This is plain method"
... return self
...
>>> dir(C) #retrieves the plain method 'pm', not the metamethod 'mm'
['__class__', '__delattr__', '__dict__', '__doc__',
'__getattribute__', '__hash__', '__init__', '__metaclass__',
'__module__', '__new__', '__reduce__', '__repr__', '__setattr__',
'__str__', '__weakref__', 'pm']
In order to retrieve 'mm' I need to invoke dir(M): this is slightly
inconsistent because usually when 'dir' is invoked on an instance, it
automatically retrieves the methods of its class, therefore I would
expect that invoked on a class it would retrieve the methods of its
metaclass. On the other hand, this could be done by design, in order
to avoid cluttering of the output of 'dir'. 'dir' is not meant to
retrieve everything. It is also a documented behaviour:
>>> print dir.__doc__
dir([object]) -> list of strings
Return an alphabetized list of names comprising (some of) the
attributes
of the given object, and of attributes reachable from it:
No argument: the names in the current scope.
Module object: the module attributes.
Type or class object: its attributes, and recursively the attributes
of
its bases.
Otherwise: its attributes, its class's attributes, and recursively
the
attributes of its class's base classes.
Michele
Michele Simionato
> You can use almost any argument at your choice:
>
>
> >>> def f(x): return x
> ...
> >>> f
> <function f at 0x4043725c>
> >>> c = classmethod(f)
> >>> c.__get__(int).im_func
> <function f at 0x4043725c>
> >>> c.__get__(c).im_func
> <function f at 0x4043725c>
> >>> c.__get__(23).im_func
> <function f at 0x4043725c>
> >>> c.__get__('as you like it').im_func
> <function f at 0x4043725c>
> >>>
>
> You can't use None specifically, but thee is no need
> for you to create a dummy class for the purpose, either.
>
>
> Alex
WHAT A DIRTY TRICK! ;)
Thanks,
M.
Greg Ewing (using news.cis.dfn.de)
> it seems that the word 'inheritance' has been abused in this context:
I think what's being said is that the "is-a" relationship
is not transitive, i.e, isinstance(a, b) and isinstance(b, c)
does not imply insinstance(a, c).
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
Greg Ewing (using news.cis.dfn.de)
> It seems I must define a dummy class
>
>>>>class C(object): pass
>
> bound the classmethod to it
>
>>>>C.c=c
>
> and then retrieve the original function via im_func
>
>>>>C.c.im_func
>
> It there must be a better way!
It seems like you should be able to do
c.__get__(None).im_func. I just tried that
and I got
Python 2.2 (#1, Jul 11 2002, 14:19:37)
[GCC 3.0.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(x): return x
...
>>> c=classmethod(f)
>>> d = c.__get__(None)
Segmentation fault (core dumped)
Oops! But it would probably have worked if I had a less
buggy version of Python...
Greg Ewing (using news.cis.dfn.de)
> In order to retrieve 'mm' I need to invoke dir(M): this is slightly
> inconsistent because usually when 'dir' is invoked on an instance, it
> automatically retrieves the methods of its class, therefore I would
> expect that invoked on a class it would retrieve the methods of its
> metaclass.
The reasoning here is probably that when someone
does dir(C), they want to know what operations are
supported by *instances* of C, not C itself.
It makes sense when you think about it that way.
This is just another one of the ambiguitities
caused by Python lumping together class attributes
and instance methods in the same dict.
Greg Ewing (using news.cis.dfn.de)
> Aren't these reasons enough? How else would you code a method that
> you can indifferently call either on any instance or on the class
> object itself?
How often do people actually *need* that functionality,
though? Smalltalkers don't seem to be bothered by the
need to know whether to call a method on a class or an
instance. Can someone provide a compelling use case
for this in Python?
> And _having_ to write a custom metaclass as the
> only way to get classmethods would be somewhat of an overkill.
What if there were some syntactic support, such as
class C:
def class foo(cls):
...
which would trigger the automatic creation of a
suitable metaclass. Would there still be a pressing
need for classmethod()-style class methods then?
Michael Hudson
> Alex Martelli wrote:
> > Aren't these reasons enough? How else would you code a method that
> > you can indifferently call either on any instance or on the class
> > object itself?
>
> How often do people actually *need* that functionality, though?
I've done it, but I can't remember why :-)
> Can someone provide a compelling use case for this in Python?
Depends what you mean by "compelling" of course. I see little harm in
having them.
Cheers,
M.
--
The meaning of "brunch" is as yet undefined.
-- Simon Booth, ucam.chat
Thomas Heller
> Alex Martelli wrote:
> > Aren't these reasons enough? How else would you code a method that
> > you can indifferently call either on any instance or on the class
> > object itself?
>
> How often do people actually *need* that functionality,
> though? Smalltalkers don't seem to be bothered by the
> need to know whether to call a method on a class or an
> instance. Can someone provide a compelling use case
> for this in Python?
Although I use metaclasses and class-methods extensively in the ctypes
module, I haven't yet needed to call a class-method from an instance.
Maybe 'classmethod' is as weird as what some people aks for in this
forum (and they are overly happy if I point out that it is indeed
possible, while the 'right' answer would probably be 'you don't want to
desing your program in this way'): Have a method that binds to the class
if called from the class, and binds to the instance if called from an
instance.
>
> > And _having_ to write a custom metaclass as the
> > only way to get classmethods would be somewhat of an overkill.
While Guido doesn't like this notation, I actually find it nice:
class MyClass(object):
class __metaclass__(type):
def my_class_methods(cls):
.....
def my_normal_method(self):
....
(Donald Beaudry had a similar notation in his objectmodule.c, which
also provided class methods).
Thomas
Alex Martelli
> Alex Martelli wrote:
>> Aren't these reasons enough? How else would you code a method that
>> you can indifferently call either on any instance or on the class
>> object itself?
>
> How often do people actually *need* that functionality,
Not often! The same could be said of MANY other things, I
believe, including of course staticmethod. How many of the
features introduced in Python 1.6 and later are actually
NEEDED, considering that 1.5.2 was already quite usable to
write large and complicated applications in?
"NEED" is a very strong word, after all.
> though? Smalltalkers don't seem to be bothered by the
> need to know whether to call a method on a class or an
> instance. Can someone provide a compelling use case
> for this in Python?
If you provide any compelling use case for staticmethod's
ability to perform the same thing, I think a parallel can
surely be drawn that shows a similar use case for classmethod's
slightly richer abilities with exactly the same context.
>> And _having_ to write a custom metaclass as the
>> only way to get classmethods would be somewhat of an overkill.
>
> What if there were some syntactic support, such as
>
> class C:
>
> def class foo(cls):
> ...
>
> which would trigger the automatic creation of a
> suitable metaclass. Would there still be a pressing
> need for classmethod()-style class methods then?
I think that using a custom metaclass has implications that
may be considered a bit too strong to sneak one in "by black
magic" -- what happens if you're multiply inheriting from
a class C and a class D and both have class methods? Should
we have a FURTHER custom metaclass generated then? Magic
gets blacker and blacker...
What do you find so horrible with classmethod as it stands
that would come close to justifying such an excursion into
exhoterism in order to get rid of it?
Alex
Michele Simionato
>
> Maybe 'classmethod' is as weird as what some people aks for in this
> forum (and they are overly happy if I point out that it is indeed
> possible, while the 'right' answer would probably be 'you don't want to
> desing your program in this way'): Have a method that binds to the class
> if called from the class, and binds to the instance if called from an
> instance.
>
> >
> > > And _having_ to write a custom metaclass as the
> > > only way to get classmethods would be somewhat of an overkill.
>
> While Guido doesn't like this notation, I actually find it nice:
>
> class MyClass(object):
>
> class __metaclass__(type):
>
> def my_class_methods(cls):
> .....
>
> def my_normal_method(self):
> ....
>
> (Donald Beaudry had a similar notation in his objectmodule.c, which
> also provided class methods).
>
> Thomas
I have used that trick few days ago to define Frozen classes:
def frozen(self,name,value):
if hasattr(self,name):
object.__setattr__(self,name,value) # standard __setattr__
else:
raise AttributeError("You cannot add attributes to %s" % self)
class Frozen(object):
"""Subclasses of Frozen are frozen, i.e. it is impossible to add
new attributes to them and their instances, except via inheritance"""
__setattr__ = frozen
class __metaclass__(type):
__setattr__ = frozen
This kills Python dynamism. Also, it become impossible to use the
self.var=something idiom, and one is forced to write code as in a
static compiled language. It protects your code against run-time
modifications (unless one overrides __setattr__).
Michele
Greg Ewing (using news.cis.dfn.de)
> what happens if you're multiply inheriting from
> a class C and a class D and both have class methods? Should
> we have a FURTHER custom metaclass generated then? Magic
> gets blacker and blacker...
Hmmm, I hadn't considered that. You're right, it could
get messy. (Smalltalk avoids this problem by not having
multiple inheritance. :-)
> What do you find so horrible with classmethod as it stands
> that would come close to justifying such an excursion into
> exhoterism in order to get rid of it?
I don't find them horrible, I was just exploring whether
it was redundant or nearly so, given the metaclass mechanism.
It seemed as though the only extra thing it gave you was
the ability to call class methods through an instance,
which didn't seem startlingly useful to me.
But I can see now that it is better behaved in the
face of multiple inheritance, which is probably a pretty
good justification!
Aahz
Michele Simionato <mi...@pitt.edu> wrote:
>
>I have used that trick few days ago to define Frozen classes:
>
>def frozen(self,name,value):
> if hasattr(self,name):
> object.__setattr__(self,name,value) # standard __setattr__
> else:
> raise AttributeError("You cannot add attributes to %s" % self)
>
>class Frozen(object):
> """Subclasses of Frozen are frozen, i.e. it is impossible to add
> new attributes to them and their instances, except via inheritance"""
> __setattr__ = frozen
> class __metaclass__(type):
> __setattr__ = frozen
>
>This kills Python dynamism. Also, it become impossible to use the
>self.var=something idiom, and one is forced to write code as in a
>static compiled language. It protects your code against run-time
>modifications (unless one overrides __setattr__).
Why the heck didn't you use __slots__?!?!?!
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
Register for PyCon now! http://www.python.org/pycon/reg.html
Alex Martelli
...
>> """Subclasses of Frozen are frozen, i.e. it is impossible to add
>> new attributes to them and their instances, except via inheritance"""
> Why the heck didn't you use __slots__?!?!?!
Just using __slots__ (without a custom metaclass) would NOT "freeze" any
subclass: just by subclassing, you'd get instances to which attributes
could freely be added.
A custom metaclass might well be a more suitable approach, of course,
and it could, in particular. use __slots__ to optimize things.
Alex
Michael Hudson
> I don't find them horrible, I was just exploring whether
> it was redundant or nearly so, given the metaclass mechanism.
> It seemed as though the only extra thing it gave you was
> the ability to call class methods through an instance,
> which didn't seem startlingly useful to me.
I've remembered why I did this, now. I have a bunch of classes that
describe records, which is to say bunches of fields and their
associated descriptions (string, integer, that kind of thing). The
superclass has a class method "get_field_description" which is
usefully called both on instances and class objects:
if instance.get_field_description(f).acceptable(datum):
setattr(instance, f, datum)
s = 0
for f in fieldnames:
s += class_obj.get_field_description(f).datasize
(well, not quite like that because I'm trying to limit the length of
the post, but I hope you get the idea).
I probably wouldn't have wished for this ability were it not there,
but it's kind of nice to have it.
Cheers,
M.
--
This proposal, if accepted, will probably mean a heck of a lot of
work for somebody. But since I don't want it accepted, I don't
care. -- Laura Creighton, PEP 666
Michele Simionato
You have a point here. Not that I would be opposed to automatic generation
of metaclasses (in some future version of Python, maybe), but it is true that
for the time being classmethods are a much simpler solution. Still, I would
prefer to have them in the standard library, but this is a matter of
personal taste. I agree that this point on multiple inheritance is a quite
strong argument for using them over metamethods.
Michele
Phillip J. Eby
>
> I was not familiar with staticmethods neither (I do not come from Java or
> C++), nevertheless I am disturbed by the confusion between "metamethods"
> (regular methods in the metaclass) and classmethods. If Samuele is
> right (I have no experience in Smalltalk) class methods in Smalltalk
> are actually metamethods. And this make much more sense to me than
> Python classmethods. This is a puristic point of view, of course,
> and I know that "practicality beats purity" ;)
There is actually an important difference between static, metamethods,
and class methods here. Important to *me*, anyway...
Class methods (like Java "static" methods) can be overridden in a
subclass, directly. Metaclass methods require subclassing the
metaclass to override. This is important when I wish to have a
*class* implement a particular behavior, and it must delegate portions
of the behavior to other *class* methods. E.g.:
class Number(object):
def fromXML(klass, node):
return klass.fromString(node.text)
fromXML = classmethod(fromXML)
def fromString(klass, s):
return klass(int(s))
fromString = classmethod(fromString)
Now, if in a subclass I redefine 'fromString', then the 'fromXML'
method will still work correctly. If I do this with metaclass
methods, I must subclass the metaclass for every subclass where I wish
to override a class method.
I use *both* metaclass methods *and* class methods in my software, but
I use them for different things. Class methods are for when the
behavior will vary by class, and metaclass methods are for when the
behavior will vary by metaclass.
Personally, I have never bothered to use a staticmethod, as I have
never seen the point of using one. Sooner or later, I would find I
needed to refer to the class, so I prefer to just start out with
classmethod instead of having to change it later. __new__ is the only
staticmethod I've ever bothered with, and I'm not actually sure why
it's not a classmethod!
> I also was confused by the *syntactic* similarity between staticmethods
> and classmethods: at the beginning I was induced to think that they
> were similar concepts, but actually they are completely different
> beasts. I mantain that the introduction of classmethods make harder
> the life for newbies without giving a definitive advantage, whereas
> on the contrary I would welcome a ternary operator.
I think it would be better to say that staticmethod is the one that
doesn't give any functionality. I also think staticmethod is poorly
named, since it does not produce a method, and it definitely doesn't
do what a 'static method' does in Java or any other language I know
of.
Alex Martelli
...
> I think it would be better to say that staticmethod is the one that
> doesn't give any functionality. I also think staticmethod is poorly
> named, since it does not produce a method, and it definitely doesn't
> do what a 'static method' does in Java or any other language I know
> of.
Nolo contendere regarding the functionality, but I'm totally puzzled
regarding your objection to the name. Where's the "definitely doesn't
do" -- isn't a Python staticmethod just like e.g. a C++ static method?
class cpp
{
static void greet() {
std::cout << "Hello, world!\n";
}
}
vs
class py:
def greet(): print "Hello, world!"
greet = staticmethod(greet)
What "definite" difference is totally escaping my notice...?
Alex
Erik Max Francis
> I think it would be better to say that staticmethod is the one that
> doesn't give any functionality. I also think staticmethod is poorly
> named, since it does not produce a method, and it definitely doesn't
> do what a 'static method' does in Java or any other language I know
> of.
Sure it does, at least in some circumstances:
>>> class C:
... def f(x): print "See?", x
... f = staticmethod(f)
...
>>> C.f(123)
See? 123
>>> c = C()
>>> c.f(123)
See? 123
The latter example wouldn't work if f (a staticmethod) weren't acting as
a method.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, USA / 37 20 N 121 53 W / &tSftDotIotE
/ \ There is nothing wrong with sobriety in moderation.
\__/ John Ciardi
Physics reference / http://www.alcyone.com/max/reference/physics/
A physics reference.
Michele Simionato
I think he is referring to this:
>>> class C:
... m=lambda self,x: x
... s=staticmethod(lambda x:x)
...
>>> C.m
<unbound method C.<lambda>>
>>> C.s
<function <lambda> at 0x402c2d14>
>>> C.__dict__['m']
<function <lambda> at 0x402c272c>
>>> C.__dict__['s']
<staticmethod object at 0x402f1060>
A staticmethod is not a method, is a function, when called as in the example.
On the other hand, if you are doing some metaprogramming such as wrapping
the methods in a class by looking at C.__dict__, you must explicitly
distinguish static methods (which are descriptors objects in this context)
from regular methods(which now are functions). No difference at all in
the normal usage, but big difference in metaprogramming applications.
Michele
Alex Martelli
...
>> > doesn't give any functionality. I also think staticmethod is poorly
>> > named, since it does not produce a method, and it definitely doesn't
>> > do what a 'static method' does in Java or any other language I know
>> regarding your objection to the name. Where's the "definitely doesn't
>> do" -- isn't a Python staticmethod just like e.g. a C++ static method?
>> What "definite" difference is totally escaping my notice...?
> A staticmethod is not a method, is a function, when called as in the
Nor did I say otherwise. But I'm still totally puzzled by the
"definitely doesn't do what a 'static method' does in Java" and I
don't see how your answer addresses this. What am I missing?
> this context) from regular methods(which now are functions). No difference
> at all in the normal usage, but big difference in metaprogramming
In Java there are no "functions" distinct from static method, so that
can't have anything to do with whatever the OP meant by "definitely
doesn't do". In C++ there is no introspection / reflection, and a
static method IS also "a function" (you can take its address as a
pointer to function, NOT as a pointer to method). So, I'm still
totally in the dark about what the "definitely doesn't do" refers
to -- *how* does a Python staticmethod differ from Java's or C++'s???
Alex
Michele Simionato
I don't claim expertise nor in Java nor in C++, therefore let the
OP to clarify this issue :)