Are decorators really that different from metaclasses...

32 views
Skip to first unread message

Paul Morrow

unread,
Aug 24, 2004, 12:29:38 PM8/24/04
to pytho...@python.org
...that they warrant an entirely new syntax? It seems that they are
very similar in a very significant way --- they alter the default
behavior of something. IMO, it's not a stretch to say that they
'parallel' metaclasses; that they are to functions/methods what
metaclasses are to classes. So why don't they share a similar syntax?

class Foo:
""" This is the docstring for class Foo. """
__metaclass__ = M
# body of class goes here


def baz():
""" This is the docstring for function baz. """
__features__ = synchronized, memoized
# body of function goes here.


I personally find the above baz function aesthetically appealing. Does
anyone else feel this way? For contrast, below is the same function
written in the proposed A1 and J2 [*] syntaxes (respectively).


@synchronized
@memoized
@doc(""" This is the docstring for function baz. """)
def baz():
# body of function goes here.


decorate:
""" This is the docstring for function baz. """
synchronized
memoized
def baz():
# body of function goes here.


* if J2 is accepted, the name 'decorate' may be replaced with some other
keyword, but I believe that the structure of the syntax would stay the same.

Paul

Paolo Veronelli

unread,
Aug 24, 2004, 1:05:58 PM8/24/04
to Paul Morrow, pytho...@python.org

Paul Morrow wrote:
> ...that they warrant an entirely new syntax? It seems that they are
> very similar in a very significant way --- they alter the default
> behavior of something. IMO, it's not a stretch to say that they
> 'parallel' metaclasses; that they are to functions/methods what
> metaclasses are to classes. So why don't they share a similar syntax?
>
> class Foo:
> """ This is the docstring for class Foo. """
> __metaclass__ = M
> # body of class goes here
>
>
> def baz():
> """ This is the docstring for function baz. """
> __features__ = synchronized, memoized
> # body of function goes here.
>
>
> I personally find the above baz function aesthetically appealing. Does
> anyone else feel this way?
>

> Paul

Obviously me,but it's possible doors are closed,too many heads try to
solve something that I start to think show a "bug" in python
architecture.I mean,does ruby need to throw up so much dust to add such
a feature?Or it just allow people to "build" language syntaxes?

Paolino

Colin J. Williams

unread,
Aug 24, 2004, 2:25:38 PM8/24/04
to
Paul Morrow wrote:

Robert Brewer provided an excellent review of the options now on the
table, but he didn't explore the question of whether there is an
existing structure on which these new operations, which are mainly
transformations, can be based.
http://www.aminus.org/rbre/python/pydec.html

There seems to be an artificial deadline which is motivating the push to
implement something, even though the effect can already be achieved, in
a less desirable way, by placing the transformer aka decorator after the
function body.

I like the general thrust of this proposal - let's use existing
structures if this is possible. The idea of putting the modifier
__features__ after the thing named and given a signature also makes sense.

Colin W.

Larry Bates

unread,
Aug 24, 2004, 4:14:04 PM8/24/04
to
You are asking the same question I asked in a separate
posting to c.l.p (but apparently more clearly).

I thought about using:

class Foo:
""" This is the docstring for class Foo. """

__decorators__ = {'metaclass': 'M'}


# body of class goes here


def baz():
""" This is the docstring for function baz. """

__decorators__= {'features': ['synchronized', 'memoized'],
'docstring':'This is the docstring for function baz. '}


# body of function goes here.

This seems extendable (e.g. you can add keys to __decorators__
dictionary without disturbing existing keys) and "sort of"
parallels __dict__ which everyone understands.

Some of these discussions are so far over my head that I thought
my idea didn't somehow make any sense at all. Then I saw your post.

Thanks,
Larry Bates
Syscon, Inc.


"Paul Morrow" <pm_...@yahoo.com> wrote in message
news:mailman.2286.1093364...@python.org...

Steven Bethard

unread,
Aug 24, 2004, 6:55:09 PM8/24/04
to pytho...@python.org
Paul Morrow <pm_mon <at> yahoo.com> writes:
>
> ...that they warrant an entirely new syntax?
[snip]

> So why don't they share a similar syntax?
>
> class Foo:
> """ This is the docstring for class Foo. """
> __metaclass__ = M
> # body of class goes here
>
>
> def baz():
> """ This is the docstring for function baz. """
> __features__ = synchronized, memoized
> # body of function goes here.

As I'm sure someone has already mentioned, this has been brought up before.
I'd especially look at:

http://mail.python.org/pipermail/python-dev/2004-August/048176.html

That being said, I think the main objection to this syntax is the same
objection to any inside-def syntax: The code inside a funcdef block (with the
exception of docstrings, which Guido has repeatedly said he regrets putting
there) is executed as part of the /execution/ of the function, not
the /definition/.

Note a classdef is not really the same in this respect. The code inside a
classdef block is executed as part of the /definition/ of the class. Compare
what happens when you call Foo() with what happens when you call baz(). With
the call to Foo(), you don't execute the code in the classdef block; you
execute the code in __new__ and/or __init__. With the call to baz(), you /do/
execute the code in the funcdef block.

So the basic answer is that decorators are different from metaclasses because
the code inside the funcdef block is executed when the function is /called/,
while the code inside the classdef is executed when the class is /declared/.

It may be a subtle difference, but I believe this is what Guido disliked so
much about the inside-def syntaxes...

Steve

Anthony Baxter

unread,
Aug 24, 2004, 11:02:47 PM8/24/04
to pytho...@python.org
On Tue, 24 Aug 2004 12:29:38 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
> ....that they warrant an entirely new syntax? It seems that they are

> very similar in a very significant way --- they alter the default
> behavior of something. IMO, it's not a stretch to say that they
> 'parallel' metaclasses; that they are to functions/methods what
> metaclasses are to classes. So why don't they share a similar syntax?

No. Function decorators are to functions as class decorators are to classes.

Class decorators are not in 2.4, because you can do most things you'd
do with a class decorator by using a metaclass, but in a completely
different way. In my opinion, class decorators _should_ be in 2.4,
because they're a much saner way to handle many things that require a
metaclass today.

They don't "alter the default behaviour". A metaclass allows the user
to specify an object that builds classes in a new way. A decorator
allows the user to specify a way to transform the already built
function (or class, for class decorators).

Anthony

Anthony Baxter

unread,
Aug 24, 2004, 11:26:03 PM8/24/04
to pytho...@python.org
I forgot one other point I meant to add - think about when the body of
a class gets executed, vs when the body of a function gets executed.
The former is at compile time, the latter is when the function is
used.

Paul Morrow

unread,
Aug 24, 2004, 11:38:28 PM8/24/04
to pytho...@python.org
Steven Bethard wrote:

> Paul Morrow <pm_mon <at> yahoo.com> writes:
>

>>...that they warrant an entirely new syntax?
>

> [snip]


>
>>So why don't they share a similar syntax?
>>

>> class Foo:
>> """ This is the docstring for class Foo. """
>> __metaclass__ = M
>> # body of class goes here
>>
>>
>> def baz():
>> """ This is the docstring for function baz. """
>> __features__ = synchronized, memoized
>> # body of function goes here.
>
>
> As I'm sure someone has already mentioned, this has been brought up before.
> I'd especially look at:
>
> http://mail.python.org/pipermail/python-dev/2004-August/048176.html
>

Well, what I proposed there is more substantial than this, and requires
a more significant change to the Python system to accomodate.

> That being said, I think the main objection to this syntax is the same
> objection to any inside-def syntax: The code inside a funcdef block (with the
> exception of docstrings, which Guido has repeatedly said he regrets putting
> there) is executed as part of the /execution/ of the function, not
> the /definition/.
>

Yes, but we have the docstring precendent nonetheless. And I think that
Guido should stop regretting that decision and start recognizing the
natural beauty of it. To allow docstrings at the top of module/class
bodies, but not at the top of function/method bodies would be
asymmetrical, and we'd wonder why. Any alternative means of providing
docstrings for methods/functions would look like a wart (IMO).
Especially since the obvious place for them would still be there.

So, no, I for one like docstrings right where they are. And look at
what they create a precedent for.

When Python executes a def statement, it binds __doc__ to the docstring
it finds there. But what is __doc__? It's an attribute of the method
or function object being defined. Yep. Its metadata. It's information
*about* the method/function. __doc__ is *not* a local variable of the
method or function object being defined.

I believe that (virtually) all __xxx__ attributes have this metadata
aspect (semantics) to them. When a programmer says

def foo():
__author__ = 'Wilma Flintstone'
__version__ = '0.1'

She does not intend for __author__ nor __version__ to be local variables
of foo, used somehow by foo in the calculation of its return value. To
her they are foo /metadata/ --- attributes of the foo object itself ---
as they contain information that *describes* foo.

Likewise, when she defines __lt__, __getitem__, __init__, etc. as part
of a class, they will not typically be called by methods of the class or
users/consumers/clients of the class [*] the way that 'normal'
attributes will. They contain meta info that describes a deeper level
of class behavior.

And so if all __xxx__ attributes are really metadata attributes, let's
formally recognize that. Then, when processing a function/method def,
after Python binds __doc__ to the function's docstring, it can go ahead
and bind the other __xxx__ attributes too, making them function
attributes rather than local variables. And then after it's finished
with the function def, if the function defined the __features__
attribute, Python can pass the completed function in turn to each
decorator in the __features__ tuple (or some such).

No asymmetry. No new syntax. No warts.


* well, __init__ is a little odd in this respect as subclasses will
sometimes call a parent's __init__ directly, but that's an exception.

Paul

Steven Bethard

unread,
Aug 25, 2004, 12:08:47 AM8/25/04
to pytho...@python.org
Paul Morrow <pm_mon <at> yahoo.com> writes:
> I believe that (virtually) all __xxx__ attributes have this metadata
> aspect (semantics) to them. When a programmer says
>
> def foo():
> __author__ = 'Wilma Flintstone'
> __version__ = '0.1'
>
> She does not intend for __author__ nor __version__ to be local variables
> of foo, used somehow by foo in the calculation of its return value. To
> her they are foo /metadata/ --- attributes of the foo object itself ---
> as they contain information that *describes* foo.

I assume you mean that this is what you'd like a programmer to intend? If I
wrote that, I would intend (and expect) __author__ to be a local variable.
I'm not saying that I couldn't be retrained. I'm just saying that right now,
I would not expect it to be otherwise.

> Likewise, when she defines __lt__, __getitem__, __init__, etc. as part
> of a class, they will not typically be called by methods of the class or
> users/consumers/clients of the class [*] the way that 'normal'
> attributes will. They contain meta info that describes a deeper level
> of class behavior.

This seems a little misleading to me. The only reason these methods are
special is because they override operators (something like "<", "[]", and "()"
respectively). You could provide a class with /exactly/ the same
functionality without ever implementing any of these methods. (Well, minus
__init__, but that was a special case in your discussion too.) The only thing
that implementing these methods does is allows your user to access these
methods through an operator shorthand. A simple example:

class Identity:
def get(self, x):
return x
__getitem__ = get

How is __getitem__ any more "metadata" then get is? They provide exactly the
same functionality. The __getitem__ declaration just allows you to access the
get method using "[...]".

Are you trying to say that "metadata" is the same thing as "operator shortcut"?

Steve


Robert Brewer

unread,
Aug 25, 2004, 12:21:20 AM8/25/04
to Anthony Baxter, pytho...@python.org
Anthony Baxter wrote:
> Class decorators are not in 2.4, because you can do most things you'd
> do with a class decorator by using a metaclass, but in a completely
> different way. In my opinion, class decorators _should_ be in 2.4,
> because they're a much saner way to handle many things that require a
> metaclass today.

For various definitions of "should", I'd say at this point they should
be in 2.5. Nothing's stopping that, AFAICT. Give function decorators a
trial run and then try class decorators. I prefer that schedule even
though I personally have use cases for class decorators and none for
functions...


Robert Brewer
MIS
Amor Ministries
fuma...@amor.org

Bengt Richter

unread,
Aug 25, 2004, 6:08:06 AM8/25/04
to

Actually, the body of the class gets executed after whatever code
just precedes the class def (though with a different environment, with
bindings going into the class dict), but UIAM both of those are compiled
when the combined source is compiled?

if you put the class def inside a function and dis.dis it, you get:

>>> def foo():
... class C(object):
... __metaclass__ = MC
... def m(self): return 'method m'
... return C
...

Above we have *compiled* both foo and the class definition in its body, but we
have executed only the foo *definition*. (Note no complaint here about MC undefined)

>>> import dis
>>> dis.dis(foo)
2 0 LOAD_CONST 1 ('C')
3 LOAD_GLOBAL 0 (object)
6 BUILD_TUPLE 1
9 LOAD_CONST 2 (<code object C at 008FDF20, file "<stdin>", line 2>)

12 MAKE_FUNCTION 0
15 CALL_FUNCTION 0
18 BUILD_CLASS
19 STORE_FAST 0 (C)

5 22 LOAD_FAST 0 (C)
25 RETURN_VALUE
26 LOAD_CONST 0 (None)
29 RETURN_VALUE

Just to show when the class definition finishes:

>>> def MC(*args): print args; return type(*args)
...

Now call foo to make the class *definition* in the body of foo execute (BTW triggering MC):

>>> Cfoo = foo()
('C', (<type 'object'>,), {'m': <function m at 0x00903370>, '__module__': '__main__', '__metacla
ss__': <function MC at 0x008FD4F0>})

(the printed args tuple above wrapped, obviously)

What we got:
>>> Cfoo
<class '__main__.C'>
>>> Cfoo.m
<unbound method C.m>
>>> Cfoo().m
<bound method C.m of <__main__.C object at 0x00901FD0>>

Regards,
Bengt Richter

Paolo Veronelli

unread,
Aug 25, 2004, 6:59:04 AM8/25/04
to Steven Bethard, pytho...@python.org

Steven Bethard wrote:
> Paul Morrow <pm_mon <at> yahoo.com> writes:

>>So why don't they share a similar syntax?
>>
>> class Foo:
>> """ This is the docstring for class Foo. """
>> __metaclass__ = M
>> # body of class goes here
>>
>>
>> def baz():
>> """ This is the docstring for function baz. """
>> __features__ = synchronized, memoized
>> # body of function goes here.
>
>

>

> So the basic answer is that decorators are different from metaclasses because
> the code inside the funcdef block is executed when the function is /called/,
> while the code inside the classdef is executed when the class is /declared/.
>

This can be seen as an engine choice and IMO is very far from touching
the syntax issue.
Still I posted a question before:

Is not time to put order in the possible uses of __xxx__ with a job on
classifying them and stop looking for a better role for BDFL?

Proposals:

1) Ban a "just because I'm free of doing it" use of them as
settable/gettable attributes (in place of non special meanings attributes)
(optional actually)

2)One or more builtin classes (interfaces) useful to build their
significance,with a previous work on classifying possible levels of
interpreter instructioning:
a) Syntactical level:
_1) (re)defining operators code
_2) ......
b) Allocational level:
__init__,__new__,__metaclass__ ....????
c) Decorational:
.........
d) Globalish:
__name__,....
d) Whatelsetional ......

Good meditation to all Paolino


Paul Morrow

unread,
Aug 25, 2004, 7:20:17 AM8/25/04
to pytho...@python.org
Steven Bethard wrote:
> Paul Morrow <pm_mon <at> yahoo.com> writes:
>
>>I believe that (virtually) all __xxx__ attributes have this metadata
>>aspect (semantics) to them. When a programmer says
>>
>> def foo():
>> __author__ = 'Wilma Flintstone'
>> __version__ = '0.1'
>>
>>She does not intend for __author__ nor __version__ to be local variables
>>of foo, used somehow by foo in the calculation of its return value. To
>>her they are foo /metadata/ --- attributes of the foo object itself ---
>>as they contain information that *describes* foo.
>
>
> I assume you mean that this is what you'd like a programmer to intend? If I
> wrote that, I would intend (and expect) __author__ to be a local variable.
> I'm not saying that I couldn't be retrained. I'm just saying that right now,
> I would not expect it to be otherwise.
>

What I mean is that this is what (I believe) veteran Python programmers
always intend, because it's a Python convention (although I've never
actually seen it described quite like this --- as metadata).

Here's what PEP 8 (http://www.python.org/peps/pep-0008.html) has to say
about __xxx__ attributes:

__double_leading_and_trailing_underscore__: "magic" objects or
attributes that live in user-controlled namespaces,
e.g. __init__, __import__ or __file__. Sometimes these are
defined by the user to trigger certain magic behavior
(e.g. operator overloading); sometimes these are inserted by the
infrastructure for its own use or for debugging purposes. Since
the infrastructure (loosely defined as the Python interpreter
and the standard library) may decide to grow its list of magic
attributes in future versions, user code should generally
refrain from using this convention for its own use. User code
that aspires to become part of the infrastructure could combine
this with a short prefix inside the underscores,
e.g. __bobo_magic_attr__.


Here's a link that gives some examples of them:
http://cardboard.nu/blog/2003_09_18/module_level_magic_variables.html)

Sure veterans know that if it looks like a local variable definition,
then it is one. But my point is that (IMO) they don't normally rely on
that aspect of __xxx__ variables. You'll almost never see a function
that examines its own metadata in the computation of its result.


>
>>Likewise, when she defines __lt__, __getitem__, __init__, etc. as part
>>of a class, they will not typically be called by methods of the class or
>>users/consumers/clients of the class [*] the way that 'normal'
>>attributes will. They contain meta info that describes a deeper level
>>of class behavior.
>
>
> This seems a little misleading to me. The only reason these methods are
> special is because they override operators (something like "<", "[]", and "()"
> respectively). You could provide a class with /exactly/ the same
> functionality without ever implementing any of these methods. (Well, minus
> __init__, but that was a special case in your discussion too.) The only thing
> that implementing these methods does is allows your user to access these
> methods through an operator shorthand. A simple example:
>
> class Identity:
> def get(self, x):
> return x
> __getitem__ = get
>
> How is __getitem__ any more "metadata" then get is? They provide exactly the
> same functionality. The __getitem__ declaration just allows you to access the
> get method using "[...]".
>

Well, but that's a big difference. __getitem__ has a much deeper
meaning than your 'get'. Defining it actually changes which syntax
you're allowed to use on instances of your class. Now you can do things
like...

ident = Identity()
print "%(alpha)s, %(beta)s, %(gamma)s" % ident


> Are you trying to say that "metadata" is the same thing as "operator shortcut"?
>

Not exactly, but it would include defining those. One way to think
about metadata is that it includes all information *about* an object,
but no information *used by* the object.

If we were to have a conversation about this conversation, then we would
be having a "meta-conversation." /metadata/ is data that describes
data. metadata for a function would be data that describes the
function; data *about* the function as opposed to data *used by* the
function. For example:

def circumference(diameter):
""" This is a docstring. It's metadata for this function. """
__author__ = 'Paul Morrow' # more metadata
pi = 3.14 # not metadata
return pi * diameter

Paul

Paul Morrow

unread,
Aug 25, 2004, 7:39:10 AM8/25/04
to pytho...@python.org
Anthony Baxter wrote:
> On Tue, 24 Aug 2004 12:29:38 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>....that they warrant an entirely new syntax? It seems that they are
>>very similar in a very significant way --- they alter the default
>>behavior of something. IMO, it's not a stretch to say that they
>>'parallel' metaclasses; that they are to functions/methods what
>>metaclasses are to classes. So why don't they share a similar syntax?
>
>
> No. Function decorators are to functions as class decorators are to classes.
>

Maybe, if they ever show up. But until then metaclasses are the closest
thing.


> Class decorators are not in 2.4, because you can do most things you'd
> do with a class decorator by using a metaclass, but in a completely
> different way. In my opinion, class decorators _should_ be in 2.4,
> because they're a much saner way to handle many things that require a
> metaclass today.
>

I'd like to see a compelling example of Class decorators. Something
that justifies using them instead of metaclasses.

> They don't "alter the default behaviour". A metaclass allows the user
> to specify an object that builds classes in a new way. A decorator
> allows the user to specify a way to transform the already built
> function (or class, for class decorators).
>

If you want to think about them that way, go ahead. But it's easier
(and I believe just as valid) to think about them in terms of altering
the default behavior of an object. For example...

class Foo:
__metaclass__ = M # comment out for default behavior

When I use a metaclass, I don't want to "specify an object that builds
classes in a new way". That's an awkward way to think about it. I want
to change the behavior of *my* class. Period. Likewise, when I use a
decorator, say 'memoized', on my function, I most definitely *do not*
want to think about what's going on under the hood --- that my function
is first created, then 'transformed' by the memoized decorator. No. I
want to think that my function simply has the 'memoized' feature and let
the system worry about the details of making that happen.


Arthur

unread,
Aug 25, 2004, 10:17:51 AM8/25/04
to
On Tue, 24 Aug 2004 23:38:28 -0400, Paul Morrow <pm_...@yahoo.com>
wrote:


>Well, what I proposed there is more substantial than this, and requires
>a more significant change to the Python system to accomodate.
>

Someone opined, I think in this thread, that the fact that the only
current alternatives to "fix" the problem that alpha 2 syntax
addresses are so so-so, by almost everybody's standards, indicates a
deeper "bug" in Python.

Athony seems to consider the current placement of the doc string a
"bug".

So there is at least *something* here.

My sense is that you are trying to zero in on it, before Python
commits itself to a workaround.

It would be selling Guido short to think that we has not considered
all this.

On the other hand, he has not communicated much about it - so it is
hard to know for sure.

So I wish you luck in the digging.

Art

Steven Bethard

unread,
Aug 25, 2004, 12:13:25 PM8/25/04
to pytho...@python.org
Paul Morrow <pm_mon <at> yahoo.com> writes:
> Steven Bethard wrote:
> > Are you trying to say that "metadata" is the same thing as "operator
shortcut"?
>
> Not exactly, but it would include defining those.

Ahh. Well I at least sort of see where you're going with this now, thanks.

It's probably notable that I could write my example in two ways:

class Identity:
def get(self, x):
return x
__getitem__ = get

class Identity:
def __getitem__(self, x):
return x
get = __getitem__

I think I could agree that the use of things like __getitem__ can indicate
some metadata about the class, but I'd note that that is not all they do. In
the example here, __getitem__ defines a function that can be used like any
other function. At the same time, because it's *named* __getitem__, we get
some metadata about the function.

Using the same logic, I should expect that in something like:

def baz():
__foo__ = "bar"
# body of baz

__foo__ would be a string, and could be used like any other string. At the
same time, because it was *named* __foo__, we would get some metadata about
the string.

So I guess my point is that, while the *name* might give us some metadata, I'm
not convinced that the *use* is in any sense metadata.

STeve

Jess Austin

unread,
Aug 25, 2004, 4:48:34 PM8/25/04
to
Paul Morrow <pm_...@yahoo.com> wrote in message news:<mailman.2346.1093432...@python.org>...

>
> Not exactly, but it would include defining those. One way to think
> about metadata is that it includes all information *about* an object,
> but no information *used by* the object.
>
> If we were to have a conversation about this conversation, then we would
> be having a "meta-conversation." /metadata/ is data that describes
> data. metadata for a function would be data that describes the
> function; data *about* the function as opposed to data *used by* the
> function. For example:

Here I think you're making an unwarranted assumption. There is
currently nothing stopping an object from using information about
itself to do its job. And this is sometimes a useful freedom. As an
example, you might have subclasses whose only real purpose is to keep
track of their class and metadata, and have all functionality provided
by their superclass:

class supe(object):
"""this docstring is rarely frobnosticated"""
def do_something_based_on_metadata():
frobnosticate(self.__doc__)

class sub1(supe):
"""I am sub1"""

class sub2(supe):
"""I am sub2"""

Something like this could often be considered premature
objectification, but I've used it successfully. Also, you'll notice
that these are classes rather than simple functions - I admit I've
never had a need to think about function metadata before.

However, I know that many people have a specific view of metadata and
don't really see how decorators fit in. To be frank, I think pep318
shoehorned in metadata to pretend that there was a wider application
area for decoration than there really is. The current quite minimal
support for metadata is sufficient for 99% of metadata use cases. My
example has worked in Python for some time. I know that decorators
would allow different behaviors to be arbitrarily assigned to various
pieces of metadata, but that doesn't seem like a big win for code
maintainability. If anyone decided to support pep318 based solely on
a love of metadata, he made a mistake.

I'm intrigued by decorators anyway, and in no small part because it
seems that they should be able to completely replace metaclasses,
which are themselves totally unrelated to metadata. In fact, that's
what at first prompted me to read a thread with the title, "Are
decorators really that different from metaclasses..." Class
decorators wouldn't be so different from metaclasses, but J2 or even
@pie are much better syntaxes than the current metaclass syntax.

In another post in this thread Anthony Baxter said they were
different. I'll respond directly to that post.

later,
Jess

Paul Morrow

unread,
Aug 25, 2004, 7:47:35 PM8/25/04
to pytho...@python.org
Jess Austin wrote:
>
> Here I think you're making an unwarranted assumption. There is
> currently nothing stopping an object from using information about
> itself to do its job. And this is sometimes a useful freedom. As an
> example, you might have subclasses whose only real purpose is to keep
> track of their class and metadata, and have all functionality provided
> by their superclass:
>
> class supe(object):
> """this docstring is rarely frobnosticated"""
> def do_something_based_on_metadata():
> frobnosticate(self.__doc__)
>
> class sub1(supe):
> """I am sub1"""
>
> class sub2(supe):
> """I am sub2"""
>

What you're trying to illustrate (I believe) is a superclass doing
something based on the docstring of a subclass. Yes this certainly does
happen. But the superclass and subclass are separate objects. I was
talking about the situation where a function does something based on
/its own/ metadata. That is what I'm saying virtually never happens,
and therefore it's ok to make all assignments to __xxx__ attributes
inside of a function def create /function attributes/ rather than /local
variables/.


Paul

Paul Morrow

unread,
Aug 25, 2004, 7:49:23 PM8/25/04
to pytho...@python.org
Paul Morrow wrote:

> and therefore it's ok to make all assignments to __xxx__ attributes
> inside of a function def create /function attributes/ rather than /local
> variables/.

In My Opinion.


Jess Austin

unread,
Aug 25, 2004, 8:13:21 PM8/25/04
to
On Tue, 24 Aug 2004 12:29:38 -0400, Paul Morrow <pm_...@yahoo.com>
wrote:
>
> No. Function decorators are to functions as class decorators are to classes.
>
> Class decorators are not in 2.4, because you can do most things you'd
> do with a class decorator by using a metaclass, but in a completely
> different way. In my opinion, class decorators _should_ be in 2.4,
> because they're a much saner way to handle many things that require a
> metaclass today.

So far we agree.

> [snip] A metaclass allows the user


> to specify an object that builds classes in a new way. A decorator
> allows the user to specify a way to transform the already built
> function (or class, for class decorators).

This is effectively the same thing. When you call a class object,
whether it is a classic class, new-style class, class with a
metaclass, decorated class, or some legal combination of the
preceding, it returns an object. You can't have too many expectations
of that object, although typically it will have its __class__ variable
set to the class object you called originally. Anything a metaclass
can do to the objects it creates (which are typically classes), a
decorator could do to objects (whether functions or classes) it
decorates, if only by creating another object and returning it instead
of the original object. I'm assuming that the function returned by
the decorator must accept name, bases, dict as arguments just like a
metaclass must. That returned function will perform in a more
flexible manner since you can also pass arguments to the decorator to
control its behavior - something you couldn't do with a metaclass. A
factory and a filter are equivalent in the sense that once an object
has squirted out the back end, they don't have access to it anymore.

What differences are there? There could be a running-time difference,
but who cares about class instantiation time? Object instantiation
time is what matters. Also, an object the class of which has
__metaclass__ set currently also has __metaclass__ set to the same
thing. This seems like an ill-advised feature, through which
presumably you could do odd and unmaintainable things, although these
could also be done through __class__.__metaclass__. It's also an
inaccurate feature - can a simple object be said to have a metaclass?
To what would this be set for a class whose metaclass had a metaclass?

Anthony Baxter <anthon...@gmail.com> wrote in message news:<mailman.2329.1093405...@python.org>...

This is a difference between classes and functions. But maybe it
could be "swept under the rug" in the following way. Code like this:

using:
metaclass_decorator(baz)
class foo(bar):
a = 1
def moo(self):
pass

could cause something similar to the following:

foo = metaclass_decorator(baz)('foo', (<class '__main__.bar'>,), {'a':
1, 'moo': <function moo at 0x00879E70>, '__module__': '__main__'})

Granted, this is slightly different from what a function decorator
does. But the difference probably won't be felt by those who use
class decorators. Besides the mischief inherent in the introduction
of another namespace floating around behind the __metaclass__ variable
of normal objects, I don't see anything metaclasses can do that class
decorators couldn't. Class decorators can be passed arguments to
control what function they return (see "baz" above), but this doesn't
seem like a major difference. So, no, I don't think that class
decorators would be _that_ different from metaclasses.

later,
Jess

Anthony Baxter

unread,
Aug 26, 2004, 3:31:16 AM8/26/04
to pytho...@python.org
On Wed, 25 Aug 2004 19:47:35 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
> What you're trying to illustrate (I believe) is a superclass doing
> something based on the docstring of a subclass. Yes this certainly does
> happen. But the superclass and subclass are separate objects. I was
> talking about the situation where a function does something based on
> /its own/ metadata. That is what I'm saying virtually never happens,
> and therefore it's ok to make all assignments to __xxx__ attributes
> inside of a function def create /function attributes/ rather than /local
> variables/.

This is an extraordinarily complex idea - you're proposing that inside
a function there is now access to a brand new namespace, that of the
function object itself. I don't think you appreciate just _how_ much
work this would requre, nor the complexity of trying to explain this
to users. Remember, at the moment, you can't even get access to the
function object itself from inside the function, without using a hack
like sys._getframe() or raising an exception.

Putting aside nested scopes for the moment, there are three namespaces
that are used in Python - local, module level global, and builtins.
You're proposing that there is a new namespace, that of the function
object, and that it only be used by accessing any local variable that
starts or ends with two underscores. At the moment, the scoping rules
for Python are _extremely_ simple. Do you really want to add something
like this, that looks like a hack, smells like a hack, and who's
implementation would be a hack? I know I don't want to see something
like this.

Anthony Baxter

unread,
Aug 26, 2004, 3:43:35 AM8/26/04
to pytho...@python.org
On Wed, 25 Aug 2004 07:20:17 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
> Not exactly, but it would include defining those. One way to think
> about metadata is that it includes all information *about* an object,
> but no information *used by* the object.

See, for me, metadata is data about an object that is defined
explicitly as about the object. You can look at an object, and say
"Aha! It supports __getitem__, __iter__ and __len__, and is therefore
likely to be a sequence type object." This is not metadata - this is
something you've determined by looking at the object. __getitem__ and
friends *are* used by the object, and so can't be considered metadata.
I'd even say that __metaclass__ is not metadata in any sense of the
word, because it's *fundamental* to the class's workings.

Something like Zope3's Interfaces, which allow you to specify the
Interfaces that an object or class implements, are closer to metadata,
but note that even there, we've gone from an initial API of:

class Foo:
__implements__ = ( IFoo, )

to
class Foo:
implements(IFoo)

... moving away from the double-under form to a more readable and
useful form. The new form allows for far more powerful and useful
behaviour, such as inheritence of interfaces from base classes.
Twisted has also moved from using the former style to the latter.
These are two _very_ large bodies of code, with a lot of very clueful
people working on them - and they found that the magic double-under
name was inferior to an explicit call syntax.

> If we were to have a conversation about this conversation, then we would
> be having a "meta-conversation." /metadata/ is data that describes
> data. metadata for a function would be data that describes the
> function; data *about* the function as opposed to data *used by* the
> function. For example:
>

> def circumference(diameter):
> """ This is a docstring. It's metadata for this function. """
> __author__ = 'Paul Morrow' # more metadata
> pi = 3.14 # not metadata
> return pi * diameter

There are a small number of double-under names that are used for
metadata - most of them are automatically inserted by Python
automatically. The overwhelming number of double-under names are not
metadata in any way.

Paolo Veronelli

unread,
Aug 26, 2004, 4:28:14 AM8/26/04
to Anthony Baxter, pytho...@python.org

Anthony Baxter wrote:
> On Wed, 25 Aug 2004 19:47:35 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>What you're trying to illustrate (I believe) is a superclass doing
>>something based on the docstring of a subclass. Yes this certainly does
>>happen. But the superclass and subclass are separate objects. I was
>>talking about the situation where a function does something based on
>>/its own/ metadata. That is what I'm saying virtually never happens,
>>and therefore it's ok to make all assignments to __xxx__ attributes
>>inside of a function def create /function attributes/ rather than /local
>>variables/.
>
>
> This is an extraordinarily complex idea - you're proposing that inside
> a function there is now access to a brand new namespace, that of the
> function object itself. I don't think you appreciate just _how_ much
> work this would requre, nor the complexity of trying to explain this
> to users. Remember, at the moment, you can't even get access to the
> function object itself from inside the function, without using a hack
> like sys._getframe() or raising an exception.

It's not the ideas complexity that fears,but the feeling we are trying
to bring down to earth the supposed magic of some language solutions.
There is nothing good in magic,specially when you have to build robust
things on it.


> Putting aside nested scopes for the moment, there are three namespaces
> that are used in Python - local, module level global, and builtins.
> You're proposing that there is a new namespace, that of the function
> object, and that it only be used by accessing any local variable that
> starts or ends with two underscores. At the moment, the scoping rules
> for Python are _extremely_ simple. Do you really want to add something
> like this, that looks like a hack, smells like a hack, and who's
> implementation would be a hack? I know I don't want to see something
> like this.

If this blows away clouds on the language future and bring back the
useful features in a wider theory where metaclasses and decorators are
members of,this shouldn't be considered a hack.If it becomes a hack the
problem is to be searched and solved above generalizing the scope system.


Anthony Baxter

unread,
Aug 26, 2004, 7:05:46 AM8/26/04
to pytho...@python.org
On Thu, 26 Aug 2004 10:28:14 +0200, Paolo Veronelli
<paolo.v...@yahoo.it> wrote:
> It's not the ideas complexity that fears,but the feeling we are trying
> to bring down to earth the supposed magic of some language solutions.
> There is nothing good in magic,specially when you have to build robust
> things on it.

I have no idea what you're trying to say here. Are you saying that the
approach of double-under variables inside a function having some new
meaning, getting put into a new scope that did not previously exist,
is somehow _less_ magical that the syntactic sugar of decorators? If
so, how do you intend to handle the backwards compatibility issue,
where code that works on Python2.4 will do something entirely
different on Python2.3 (the double-under variables will be silently
ignored). Do you intend that the double-under names would also be
looked for in the same scopes? That is, what will this code do?

def foo():
__name__ = '%s_banana'%(__name__)

> If this blows away clouds on the language future and bring back the
> useful features in a wider theory where metaclasses and decorators are
> members of,this shouldn't be considered a hack.If it becomes a hack the
> problem is to be searched and solved above generalizing the scope system.

Hand waving is all well and good, but this isn't a matter of
"generalizing the scope system". This is a _radical_ change to the
scoping rules of Python, and I think it's safe to say that there's
*absolutely* *no* *way* something like this would be considered,
without an excellent reason - and bolting some sort of strange
decorator semantics doesn't cut it.

Anthony

Paul Morrow

unread,
Aug 26, 2004, 7:28:32 AM8/26/04
to pytho...@python.org
Anthony Baxter wrote:
> On Wed, 25 Aug 2004 19:47:35 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>What you're trying to illustrate (I believe) is a superclass doing
>>something based on the docstring of a subclass. Yes this certainly does
>>happen. But the superclass and subclass are separate objects. I was
>>talking about the situation where a function does something based on
>>/its own/ metadata. That is what I'm saying virtually never happens,
>>and therefore it's ok to make all assignments to __xxx__ attributes
>>inside of a function def create /function attributes/ rather than /local
>>variables/.
>
>
> This is an extraordinarily complex idea - you're proposing that inside
> a function there is now access to a brand new namespace, that of the
> function object itself. I don't think you appreciate just _how_ much
> work this would requre, nor the complexity of trying to explain this
> to users. Remember, at the moment, you can't even get access to the
> function object itself from inside the function, without using a hack
> like sys._getframe() or raising an exception.
>

No I'm not suggesting that. The function would continue to have exactly
the same namespace.

This is not a big change; quite the contrary...

The interpreter, when it is executing a def statement, after it binds
__doc__ to the docstring, would execute any assignments to __xxx__
attributes it finds immediately following the docstring (if present). So

def circumference(diameter):
""" This is a docstring. """

__author__ = 'Paul Morrow'
pi = 3.14
return pi * diameter

would be /exactly equivalent/ to

def circumference(diameter):
pi = 3.14
return pi * diameter

circumference.__doc__ = """ This is a docstring. """
circumference.__author__ = 'Paul Morrow'


See how simple? In fact, isn't this just as simple --- or even simpler
-- than what would be require to implement J2 or A1?

And remember, the assumption here is that these assignments (to __xxx__
attributes) are a form of declarations /about the function/, just as
docstrings are now. There's no need to support conditional declarations...

def foo():
""" I am a docstring. """
__author__ = 'Morrow'

# Just as there is no proper, 'definition time' way to
# conditionally change a docstring, the code below
# will generate an exception (at runtime).
if 1:
__version__ = '0.1'
else:
__version__ = '0.2'

Think of what I'm proposing here as just a convenience syntax. We can
already do the same thing after (outsidef of) the function def as the
second circumference example above shows.

Paul

Paul Morrow

unread,
Aug 26, 2004, 8:05:26 AM8/26/04
to pytho...@python.org
Anthony Baxter wrote:
> On Wed, 25 Aug 2004 07:20:17 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>Not exactly, but it would include defining those. One way to think
>>about metadata is that it includes all information *about* an object,
>>but no information *used by* the object.
>
>
> See, for me, metadata is data about an object that is defined
> explicitly as about the object. You can look at an object, and say
> "Aha! It supports __getitem__, __iter__ and __len__, and is therefore
> likely to be a sequence type object." This is not metadata - this is
> something you've determined by looking at the object. __getitem__ and
> friends *are* used by the object, and so can't be considered metadata.
> I'd even say that __metaclass__ is not metadata in any sense of the
> word, because it's *fundamental* to the class's workings.
>

Ok. Then let's jettison the baggage with the word metaclass and just
call them 'magic attributes'. It doesn't matter. The essential point
about them is that they almost /never have 'local variable' semantics/.
They are attributes of the object being defined. That's the point I'm
trying to drive home here. When we use __xxx__ attributes, we are not
intending to create local variables. No. We are making declarations.

> Something like Zope3's Interfaces, which allow you to specify the
> Interfaces that an object or class implements, are closer to metadata,
> but note that even there, we've gone from an initial API of:
>
> class Foo:
> __implements__ = ( IFoo, )
>
> to
> class Foo:
> implements(IFoo)
>
> ... moving away from the double-under form to a more readable and
> useful form. The new form allows for far more powerful and useful
> behaviour, such as inheritence of interfaces from base classes.
> Twisted has also moved from using the former style to the latter.
> These are two _very_ large bodies of code, with a lot of very clueful
> people working on them - and they found that the magic double-under
> name was inferior to an explicit call syntax.
>

Woah Man! I thought you wanted special functionality to have special
syntax, so that it really sticks out that we're doing something special?
It sounds like now you are /endorsing/ the use of normal looking syntax
that does special things.

Personally, the look of the old '__implements__' syntax was vastly
superior. In an instant, an experienced pythonista could see that
magic was being peformed on Foo. That fact is obscured in the new
'implements' syntax.

Wait. Did I just argue your point from an earlier thread? :-) See, we
do (or at least 'did') agree on some things...

>
>>If we were to have a conversation about this conversation, then we would
>>be having a "meta-conversation." /metadata/ is data that describes
>>data. metadata for a function would be data that describes the
>>function; data *about* the function as opposed to data *used by* the
>>function. For example:
>>

>> def circumference(diameter):


>> """ This is a docstring. It's metadata for this function. """
>> __author__ = 'Paul Morrow' # more metadata
>> pi = 3.14 # not metadata
>> return pi * diameter
>
>
> There are a small number of double-under names that are used for
> metadata - most of them are automatically inserted by Python
> automatically. The overwhelming number of double-under names are not
> metadata in any way.

I don't need to debate that for my position to be valid. How about from
now on we just call it magic data? That works fine for me. As long as
we're clear that it's not (intended to be) 'local data'. That it has
more of a declaration semantics.

Paul


Paul Morrow

unread,
Aug 26, 2004, 8:27:52 AM8/26/04
to pytho...@python.org
Anthony Baxter wrote:
> On Thu, 26 Aug 2004 10:28:14 +0200, Paolo Veronelli
> <paolo.v...@yahoo.it> wrote:
>
> ignored). Do you intend that the double-under names would also be
> looked for in the same scopes? That is, what will this code do?
>
> def foo():
> __name__ = '%s_banana'%(__name__)
>

/exactly/ the same thing as

def foo(): pass

foo.__name__ = '%s_banana'%(foo.__name__)


Paul

Arthur

unread,
Aug 26, 2004, 8:34:16 AM8/26/04
to


Though I am learning stuff by trying to keep up with this conversation
- I should probably stay out of it, directly.

But, as I publically demonstrated in my confusion about __name__

...to someone educated from within Python, mostly by chucking out
code, rather than reading documentation - that __name__ accessed
from within the function body is something totally different from
__name__ accessed from outside of it, was not what I think the
language taught me to expect. I might have intuited that to expect
otherwise would be "magic", but thought, somehow, that the __xxx__
implied the availability of that kind of magic.

In other words, it would have surprised me less to have been right
than to have benn wrong.

Though I have learned not to be shocked about being wrong ;)

And can understand the possiblity that others with other backgrounds
might be surprised if I had been right.

Does

"""
Namespaces are one honking great idea -- let's do more of those!
"""

come into play here at all?

Art

Arthur

unread,
Aug 26, 2004, 9:00:41 AM8/26/04
to
On Thu, 26 Aug 2004 21:05:46 +1000, Anthony Baxter
<anthon...@gmail.com> wrote:
>
>def foo():
> __name__ = '%s_banana'%(__name__)
>


I'm not sure what you are driving at here, but ...

Aa a practical matter, wouldn't it be nice to be able to use string
substitution on a docstring placed in its usual location under def
foo().

That I need to place my __doc__ under the function can't be considered
a good thing, or intutive thing.

Sort of like having to do transformations after and under the function
;)

Art

Arthur

unread,
Aug 26, 2004, 9:28:32 AM8/26/04
to
On Thu, 26 Aug 2004 13:00:41 GMT, Arthur <ajsi...@optonline.com>
wrote:

But you can perhaps respond that with a2 the flexibility to put the
doc string where you think it belongs, before and above the function
(and I don't necessarily disagree, in that that it is always where I
find myself putting #comments (though perhaps I am putting them there
in particular to *distinguish* them from a docstring (where would I
put comments if my docstring were above (perhaps below)))) -

that you and I will be placing are docstrings differently can be seen
both as a benefit *and as a cost*, of the a2 implmentation.

Art

Paul Morrow

unread,
Aug 26, 2004, 9:31:55 AM8/26/04
to pytho...@python.org
Arthur wrote:

Currently, you can change a doc string outside of a function, by
modifying the function's __doc__ attribute

>>> def foo():
... """ I am foo """
...
>>> foo.__doc__
' I am foo '
>>> foo.__doc__ = foo.__doc__ + 'indeed'
>>> foo.__doc__
' I am foo indeed'
>>>

IMO, to change it inside of a function def should be (but isn't) as easy
as...

>>> def foo():
... """ I am foo """
... __doc__ = __doc__ + 'indeed'

Paul

Paul Morrow

unread,
Aug 26, 2004, 9:50:43 AM8/26/04
to pytho...@python.org
Errata. Please change the paragraph...

>
> Ok. Then let's jettison the baggage with the word metaclass and just
> call them 'magic attributes'. It doesn't matter. The essential point
> about them is that they almost /never have 'local variable' semantics/.
> They are attributes of the object being defined. That's the point I'm
> trying to drive home here. When we use __xxx__ attributes, we are not
> intending to create local variables. No. We are making declarations.
>

...to...

Ok. Then let's jettison the baggage with the word metadata and just


call them 'magic attributes'. It doesn't matter. The essential point
about them is that they almost /never have 'local variable' semantics/.
They are attributes of the object being defined. That's the point I'm

trying to drive home here. When we define __xxx__ attributes, we are

not intending to create local variables. No. We are making declarations.


Paul


Arthur

unread,
Aug 26, 2004, 10:40:18 AM8/26/04
to
On Thu, 26 Aug 2004 09:31:55 -0400, Paul Morrow <pm_...@yahoo.com>
wrote:

>


>Currently, you can change a doc string outside of a function, by
>modifying the function's __doc__ attribute
>
> >>> def foo():
> ... """ I am foo """
> ...
> >>> foo.__doc__
> ' I am foo '
> >>> foo.__doc__ = foo.__doc__ + 'indeed'
> >>> foo.__doc__
> ' I am foo indeed'
> >>>
>
>IMO, to change it inside of a function def should be (but isn't) as easy
>as...
>
> >>> def foo():
> ... """ I am foo """
> ... __doc__ = __doc__ + 'indeed'
>
>Paul

Yes. Not only do I follow, but I think we came to exactly the same
place, from very different directions, and coming from what I sense is
very different backgrounds.

Its just that I don't think many others seem to find that as
interesting as I happen to.

Hopefully, for good enough reasons.

Art

>
>

Anthony Baxter

unread,
Aug 26, 2004, 10:53:51 AM8/26/04
to Arthur, pytho...@python.org
On Thu, 26 Aug 2004 14:40:18 GMT, Arthur <ajsi...@optonline.com> wrote:
> >IMO, to change it inside of a function def should be (but isn't) as easy
> >as...
> >
> > >>> def foo():
> > ... """ I am foo """
> > ... __doc__ = __doc__ + 'indeed'
> >
> >Paul
>
> Yes. Not only do I follow, b