Are decorators really that different from metaclasses...
Paul Morrow
very similar in a very significant way --- they alter the default
behavior of something. IMO, it's not a stretch to say that they
'parallel' metaclasses; that they are to functions/methods what
metaclasses are to classes. So why don't they share a similar syntax?
class Foo:
""" This is the docstring for class Foo. """
__metaclass__ = M
# body of class goes here
def baz():
""" This is the docstring for function baz. """
__features__ = synchronized, memoized
# body of function goes here.
I personally find the above baz function aesthetically appealing. Does
anyone else feel this way? For contrast, below is the same function
written in the proposed A1 and J2 [*] syntaxes (respectively).
@synchronized
@memoized
@doc(""" This is the docstring for function baz. """)
def baz():
# body of function goes here.
decorate:
""" This is the docstring for function baz. """
synchronized
memoized
def baz():
# body of function goes here.
* if J2 is accepted, the name 'decorate' may be replaced with some other
keyword, but I believe that the structure of the syntax would stay the same.
Paul
Paolo Veronelli
Paul Morrow wrote:
> ...that they warrant an entirely new syntax? It seems that they are
> very similar in a very significant way --- they alter the default
> behavior of something. IMO, it's not a stretch to say that they
> 'parallel' metaclasses; that they are to functions/methods what
> metaclasses are to classes. So why don't they share a similar syntax?
>
> class Foo:
> """ This is the docstring for class Foo. """
> __metaclass__ = M
> # body of class goes here
>
>
> def baz():
> """ This is the docstring for function baz. """
> __features__ = synchronized, memoized
> # body of function goes here.
>
>
> I personally find the above baz function aesthetically appealing. Does
> anyone else feel this way?
>
> Paul
Obviously me,but it's possible doors are closed,too many heads try to
solve something that I start to think show a "bug" in python
architecture.I mean,does ruby need to throw up so much dust to add such
a feature?Or it just allow people to "build" language syntaxes?
Paolino
Colin J. Williams
Robert Brewer provided an excellent review of the options now on the
table, but he didn't explore the question of whether there is an
existing structure on which these new operations, which are mainly
transformations, can be based.
http://www.aminus.org/rbre/python/pydec.html
There seems to be an artificial deadline which is motivating the push to
implement something, even though the effect can already be achieved, in
a less desirable way, by placing the transformer aka decorator after the
function body.
I like the general thrust of this proposal - let's use existing
structures if this is possible. The idea of putting the modifier
__features__ after the thing named and given a signature also makes sense.
Colin W.
Larry Bates
posting to c.l.p (but apparently more clearly).
I thought about using:
class Foo:
""" This is the docstring for class Foo. """
__decorators__ = {'metaclass': 'M'}
# body of class goes here
def baz():
""" This is the docstring for function baz. """
__decorators__= {'features': ['synchronized', 'memoized'],
'docstring':'This is the docstring for function baz. '}
# body of function goes here.
This seems extendable (e.g. you can add keys to __decorators__
dictionary without disturbing existing keys) and "sort of"
parallels __dict__ which everyone understands.
Some of these discussions are so far over my head that I thought
my idea didn't somehow make any sense at all. Then I saw your post.
Thanks,
Larry Bates
Syscon, Inc.
"Paul Morrow" <pm_...@yahoo.com> wrote in message
news:mailman.2286.1093364...@python.org...
Steven Bethard
> So why don't they share a similar syntax?
>
> class Foo:
> """ This is the docstring for class Foo. """
> __metaclass__ = M
> # body of class goes here
>
>
> def baz():
> """ This is the docstring for function baz. """
> __features__ = synchronized, memoized
> # body of function goes here.
As I'm sure someone has already mentioned, this has been brought up before.
I'd especially look at:
http://mail.python.org/pipermail/python-dev/2004-August/048176.html
That being said, I think the main objection to this syntax is the same
objection to any inside-def syntax: The code inside a funcdef block (with the
exception of docstrings, which Guido has repeatedly said he regrets putting
there) is executed as part of the /execution/ of the function, not
the /definition/.
Note a classdef is not really the same in this respect. The code inside a
classdef block is executed as part of the /definition/ of the class. Compare
what happens when you call Foo() with what happens when you call baz(). With
the call to Foo(), you don't execute the code in the classdef block; you
execute the code in __new__ and/or __init__. With the call to baz(), you /do/
execute the code in the funcdef block.
So the basic answer is that decorators are different from metaclasses because
the code inside the funcdef block is executed when the function is /called/,
while the code inside the classdef is executed when the class is /declared/.
It may be a subtle difference, but I believe this is what Guido disliked so
much about the inside-def syntaxes...
Steve
Anthony Baxter
> ....that they warrant an entirely new syntax? It seems that they are
> very similar in a very significant way --- they alter the default
> behavior of something. IMO, it's not a stretch to say that they
> 'parallel' metaclasses; that they are to functions/methods what
No. Function decorators are to functions as class decorators are to classes.
Class decorators are not in 2.4, because you can do most things you'd
do with a class decorator by using a metaclass, but in a completely
different way. In my opinion, class decorators _should_ be in 2.4,
because they're a much saner way to handle many things that require a
metaclass today.
They don't "alter the default behaviour". A metaclass allows the user
to specify an object that builds classes in a new way. A decorator
allows the user to specify a way to transform the already built
function (or class, for class decorators).
Anthony
Anthony Baxter
a class gets executed, vs when the body of a function gets executed.
The former is at compile time, the latter is when the function is
used.
Paul Morrow
> Paul Morrow <pm_mon <at> yahoo.com> writes:
>
>>...that they warrant an entirely new syntax?
>
> [snip]
>
>>So why don't they share a similar syntax?
>>
>> class Foo:
>> """ This is the docstring for class Foo. """
>> __metaclass__ = M
>> # body of class goes here
>>
>>
>> def baz():
>> """ This is the docstring for function baz. """
>> __features__ = synchronized, memoized
>> # body of function goes here.
>
>
> As I'm sure someone has already mentioned, this has been brought up before.
> I'd especially look at:
>
> http://mail.python.org/pipermail/python-dev/2004-August/048176.html
>
Well, what I proposed there is more substantial than this, and requires
a more significant change to the Python system to accomodate.
> That being said, I think the main objection to this syntax is the same
> objection to any inside-def syntax: The code inside a funcdef block (with the
> exception of docstrings, which Guido has repeatedly said he regrets putting
> there) is executed as part of the /execution/ of the function, not
> the /definition/.
>
Yes, but we have the docstring precendent nonetheless. And I think that
Guido should stop regretting that decision and start recognizing the
natural beauty of it. To allow docstrings at the top of module/class
bodies, but not at the top of function/method bodies would be
asymmetrical, and we'd wonder why. Any alternative means of providing
docstrings for methods/functions would look like a wart (IMO).
Especially since the obvious place for them would still be there.
So, no, I for one like docstrings right where they are. And look at
what they create a precedent for.
When Python executes a def statement, it binds __doc__ to the docstring
it finds there. But what is __doc__? It's an attribute of the method
or function object being defined. Yep. Its metadata. It's information
*about* the method/function. __doc__ is *not* a local variable of the
method or function object being defined.
I believe that (virtually) all __xxx__ attributes have this metadata
aspect (semantics) to them. When a programmer says
def foo():
__author__ = 'Wilma Flintstone'
__version__ = '0.1'
She does not intend for __author__ nor __version__ to be local variables
of foo, used somehow by foo in the calculation of its return value. To
her they are foo /metadata/ --- attributes of the foo object itself ---
as they contain information that *describes* foo.
Likewise, when she defines __lt__, __getitem__, __init__, etc. as part
of a class, they will not typically be called by methods of the class or
users/consumers/clients of the class [*] the way that 'normal'
attributes will. They contain meta info that describes a deeper level
of class behavior.
And so if all __xxx__ attributes are really metadata attributes, let's
formally recognize that. Then, when processing a function/method def,
after Python binds __doc__ to the function's docstring, it can go ahead
and bind the other __xxx__ attributes too, making them function
attributes rather than local variables. And then after it's finished
with the function def, if the function defined the __features__
attribute, Python can pass the completed function in turn to each
decorator in the __features__ tuple (or some such).
No asymmetry. No new syntax. No warts.
* well, __init__ is a little odd in this respect as subclasses will
sometimes call a parent's __init__ directly, but that's an exception.
Paul
Steven Bethard
> I believe that (virtually) all __xxx__ attributes have this metadata
> aspect (semantics) to them. When a programmer says
>
> def foo():
> __author__ = 'Wilma Flintstone'
> __version__ = '0.1'
>
> She does not intend for __author__ nor __version__ to be local variables
> of foo, used somehow by foo in the calculation of its return value. To
> her they are foo /metadata/ --- attributes of the foo object itself ---
> as they contain information that *describes* foo.
I assume you mean that this is what you'd like a programmer to intend? If I
wrote that, I would intend (and expect) __author__ to be a local variable.
I'm not saying that I couldn't be retrained. I'm just saying that right now,
I would not expect it to be otherwise.
> Likewise, when she defines __lt__, __getitem__, __init__, etc. as part
> of a class, they will not typically be called by methods of the class or
> users/consumers/clients of the class [*] the way that 'normal'
> attributes will. They contain meta info that describes a deeper level
> of class behavior.
This seems a little misleading to me. The only reason these methods are
special is because they override operators (something like "<", "[]", and "()"
respectively). You could provide a class with /exactly/ the same
functionality without ever implementing any of these methods. (Well, minus
__init__, but that was a special case in your discussion too.) The only thing
that implementing these methods does is allows your user to access these
methods through an operator shorthand. A simple example:
class Identity:
def get(self, x):
return x
__getitem__ = get
How is __getitem__ any more "metadata" then get is? They provide exactly the
same functionality. The __getitem__ declaration just allows you to access the
get method using "[...]".
Are you trying to say that "metadata" is the same thing as "operator shortcut"?
Steve
Robert Brewer
> Class decorators are not in 2.4, because you can do most things you'd
> do with a class decorator by using a metaclass, but in a completely
> different way. In my opinion, class decorators _should_ be in 2.4,
> because they're a much saner way to handle many things that require a
> metaclass today.
For various definitions of "should", I'd say at this point they should
be in 2.5. Nothing's stopping that, AFAICT. Give function decorators a
trial run and then try class decorators. I prefer that schedule even
though I personally have use cases for class decorators and none for
functions...
Robert Brewer
MIS
Amor Ministries
fuma...@amor.org
Bengt Richter
Actually, the body of the class gets executed after whatever code
just precedes the class def (though with a different environment, with
bindings going into the class dict), but UIAM both of those are compiled
when the combined source is compiled?
if you put the class def inside a function and dis.dis it, you get:
>>> def foo():
... class C(object):
... __metaclass__ = MC
... def m(self): return 'method m'
... return C
...
Above we have *compiled* both foo and the class definition in its body, but we
have executed only the foo *definition*. (Note no complaint here about MC undefined)
>>> import dis
>>> dis.dis(foo)
2 0 LOAD_CONST 1 ('C')
3 LOAD_GLOBAL 0 (object)
6 BUILD_TUPLE 1
9 LOAD_CONST 2 (<code object C at 008FDF20, file "<stdin>", line 2>)
12 MAKE_FUNCTION 0
15 CALL_FUNCTION 0
18 BUILD_CLASS
19 STORE_FAST 0 (C)
5 22 LOAD_FAST 0 (C)
25 RETURN_VALUE
26 LOAD_CONST 0 (None)
29 RETURN_VALUE
Just to show when the class definition finishes:
>>> def MC(*args): print args; return type(*args)
...
Now call foo to make the class *definition* in the body of foo execute (BTW triggering MC):
>>> Cfoo = foo()
('C', (<type 'object'>,), {'m': <function m at 0x00903370>, '__module__': '__main__', '__metacla
ss__': <function MC at 0x008FD4F0>})
(the printed args tuple above wrapped, obviously)
What we got:
>>> Cfoo
<class '__main__.C'>
>>> Cfoo.m
<unbound method C.m>
>>> Cfoo().m
<bound method C.m of <__main__.C object at 0x00901FD0>>
Regards,
Bengt Richter
Paolo Veronelli
Steven Bethard wrote:
> Paul Morrow <pm_mon <at> yahoo.com> writes:
>>So why don't they share a similar syntax?
>>
>> class Foo:
>> """ This is the docstring for class Foo. """
>> __metaclass__ = M
>> # body of class goes here
>>
>>
>> def baz():
>> """ This is the docstring for function baz. """
>> __features__ = synchronized, memoized
>> # body of function goes here.
>
>
>
> So the basic answer is that decorators are different from metaclasses because
> the code inside the funcdef block is executed when the function is /called/,
> while the code inside the classdef is executed when the class is /declared/.
>
This can be seen as an engine choice and IMO is very far from touching
the syntax issue.
Still I posted a question before:
Is not time to put order in the possible uses of __xxx__ with a job on
classifying them and stop looking for a better role for BDFL?
Proposals:
1) Ban a "just because I'm free of doing it" use of them as
settable/gettable attributes (in place of non special meanings attributes)
(optional actually)
2)One or more builtin classes (interfaces) useful to build their
significance,with a previous work on classifying possible levels of
interpreter instructioning:
a) Syntactical level:
_1) (re)defining operators code
_2) ......
b) Allocational level:
__init__,__new__,__metaclass__ ....????
c) Decorational:
.........
d) Globalish:
__name__,....
d) Whatelsetional ......
Good meditation to all Paolino
Paul Morrow
>>aspect (semantics) to them. When a programmer says
>>
>> def foo():
>> __author__ = 'Wilma Flintstone'
>> __version__ = '0.1'
>>
>>She does not intend for __author__ nor __version__ to be local variables
>>of foo, used somehow by foo in the calculation of its return value. To
>>her they are foo /metadata/ --- attributes of the foo object itself ---
>>as they contain information that *describes* foo.
>
>
> I assume you mean that this is what you'd like a programmer to intend? If I
> wrote that, I would intend (and expect) __author__ to be a local variable.
> I'm not saying that I couldn't be retrained. I'm just saying that right now,
> I would not expect it to be otherwise.
>
What I mean is that this is what (I believe) veteran Python programmers
always intend, because it's a Python convention (although I've never
actually seen it described quite like this --- as metadata).
Here's what PEP 8 (http://www.python.org/peps/pep-0008.html) has to say
about __xxx__ attributes:
__double_leading_and_trailing_underscore__: "magic" objects or
attributes that live in user-controlled namespaces,
e.g. __init__, __import__ or __file__. Sometimes these are
defined by the user to trigger certain magic behavior
(e.g. operator overloading); sometimes these are inserted by the
infrastructure for its own use or for debugging purposes. Since
the infrastructure (loosely defined as the Python interpreter
and the standard library) may decide to grow its list of magic
attributes in future versions, user code should generally
refrain from using this convention for its own use. User code
that aspires to become part of the infrastructure could combine
this with a short prefix inside the underscores,
e.g. __bobo_magic_attr__.
Here's a link that gives some examples of them:
http://cardboard.nu/blog/2003_09_18/module_level_magic_variables.html)
Sure veterans know that if it looks like a local variable definition,
then it is one. But my point is that (IMO) they don't normally rely on
that aspect of __xxx__ variables. You'll almost never see a function
that examines its own metadata in the computation of its result.
>
>>Likewise, when she defines __lt__, __getitem__, __init__, etc. as part
>>of a class, they will not typically be called by methods of the class or
>>users/consumers/clients of the class [*] the way that 'normal'
>>attributes will. They contain meta info that describes a deeper level
>>of class behavior.
>
>
> This seems a little misleading to me. The only reason these methods are
> special is because they override operators (something like "<", "[]", and "()"
> respectively). You could provide a class with /exactly/ the same
> functionality without ever implementing any of these methods. (Well, minus
> __init__, but that was a special case in your discussion too.) The only thing
> that implementing these methods does is allows your user to access these
> methods through an operator shorthand. A simple example:
>
> class Identity:
> def get(self, x):
> return x
> __getitem__ = get
>
> How is __getitem__ any more "metadata" then get is? They provide exactly the
> same functionality. The __getitem__ declaration just allows you to access the
> get method using "[...]".
>
Well, but that's a big difference. __getitem__ has a much deeper
meaning than your 'get'. Defining it actually changes which syntax
you're allowed to use on instances of your class. Now you can do things
like...
ident = Identity()
print "%(alpha)s, %(beta)s, %(gamma)s" % ident
> Are you trying to say that "metadata" is the same thing as "operator shortcut"?
>
Not exactly, but it would include defining those. One way to think
about metadata is that it includes all information *about* an object,
but no information *used by* the object.
If we were to have a conversation about this conversation, then we would
be having a "meta-conversation." /metadata/ is data that describes
data. metadata for a function would be data that describes the
function; data *about* the function as opposed to data *used by* the
function. For example:
def circumference(diameter):
""" This is a docstring. It's metadata for this function. """
__author__ = 'Paul Morrow' # more metadata
pi = 3.14 # not metadata
return pi * diameter
Paul
Paul Morrow
> On Tue, 24 Aug 2004 12:29:38 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>....that they warrant an entirely new syntax? It seems that they are
>>very similar in a very significant way --- they alter the default
>>behavior of something. IMO, it's not a stretch to say that they
>>'parallel' metaclasses; that they are to functions/methods what
>>metaclasses are to classes. So why don't they share a similar syntax?
>
>
> No. Function decorators are to functions as class decorators are to classes.
>
Maybe, if they ever show up. But until then metaclasses are the closest
thing.
> Class decorators are not in 2.4, because you can do most things you'd
> do with a class decorator by using a metaclass, but in a completely
> different way. In my opinion, class decorators _should_ be in 2.4,
> because they're a much saner way to handle many things that require a
> metaclass today.
>
I'd like to see a compelling example of Class decorators. Something
that justifies using them instead of metaclasses.
> They don't "alter the default behaviour". A metaclass allows the user
> to specify an object that builds classes in a new way. A decorator
> allows the user to specify a way to transform the already built
> function (or class, for class decorators).
>
If you want to think about them that way, go ahead. But it's easier
(and I believe just as valid) to think about them in terms of altering
the default behavior of an object. For example...
class Foo:
__metaclass__ = M # comment out for default behavior
When I use a metaclass, I don't want to "specify an object that builds
classes in a new way". That's an awkward way to think about it. I want
to change the behavior of *my* class. Period. Likewise, when I use a
decorator, say 'memoized', on my function, I most definitely *do not*
want to think about what's going on under the hood --- that my function
is first created, then 'transformed' by the memoized decorator. No. I
want to think that my function simply has the 'memoized' feature and let
the system worry about the details of making that happen.
Arthur
wrote:
>Well, what I proposed there is more substantial than this, and requires
>a more significant change to the Python system to accomodate.
>
Someone opined, I think in this thread, that the fact that the only
current alternatives to "fix" the problem that alpha 2 syntax
addresses are so so-so, by almost everybody's standards, indicates a
deeper "bug" in Python.
Athony seems to consider the current placement of the doc string a
"bug".
So there is at least *something* here.
My sense is that you are trying to zero in on it, before Python
commits itself to a workaround.
It would be selling Guido short to think that we has not considered
all this.
On the other hand, he has not communicated much about it - so it is
hard to know for sure.
So I wish you luck in the digging.
Art
Steven Bethard
> > Are you trying to say that "metadata" is the same thing as "operator
shortcut"?
>
> Not exactly, but it would include defining those.
Ahh. Well I at least sort of see where you're going with this now, thanks.
It's probably notable that I could write my example in two ways:
class Identity:
def get(self, x):
return x
__getitem__ = get
class Identity:
def __getitem__(self, x):
return x
get = __getitem__
I think I could agree that the use of things like __getitem__ can indicate
some metadata about the class, but I'd note that that is not all they do. In
the example here, __getitem__ defines a function that can be used like any
other function. At the same time, because it's *named* __getitem__, we get
some metadata about the function.
Using the same logic, I should expect that in something like:
def baz():
__foo__ = "bar"
# body of baz
__foo__ would be a string, and could be used like any other string. At the
same time, because it was *named* __foo__, we would get some metadata about
the string.
So I guess my point is that, while the *name* might give us some metadata, I'm
not convinced that the *use* is in any sense metadata.
STeve
Jess Austin
>
> Not exactly, but it would include defining those. One way to think
> about metadata is that it includes all information *about* an object,
> but no information *used by* the object.
>
> If we were to have a conversation about this conversation, then we would
> be having a "meta-conversation." /metadata/ is data that describes
> data. metadata for a function would be data that describes the
> function; data *about* the function as opposed to data *used by* the
> function. For example:
Here I think you're making an unwarranted assumption. There is
currently nothing stopping an object from using information about
itself to do its job. And this is sometimes a useful freedom. As an
example, you might have subclasses whose only real purpose is to keep
track of their class and metadata, and have all functionality provided
by their superclass:
class supe(object):
"""this docstring is rarely frobnosticated"""
def do_something_based_on_metadata():
frobnosticate(self.__doc__)
class sub1(supe):
"""I am sub1"""
class sub2(supe):
"""I am sub2"""
Something like this could often be considered premature
objectification, but I've used it successfully. Also, you'll notice
that these are classes rather than simple functions - I admit I've
never had a need to think about function metadata before.
However, I know that many people have a specific view of metadata and
don't really see how decorators fit in. To be frank, I think pep318
shoehorned in metadata to pretend that there was a wider application
area for decoration than there really is. The current quite minimal
support for metadata is sufficient for 99% of metadata use cases. My
example has worked in Python for some time. I know that decorators
would allow different behaviors to be arbitrarily assigned to various
pieces of metadata, but that doesn't seem like a big win for code
maintainability. If anyone decided to support pep318 based solely on
a love of metadata, he made a mistake.
I'm intrigued by decorators anyway, and in no small part because it
seems that they should be able to completely replace metaclasses,
which are themselves totally unrelated to metadata. In fact, that's
what at first prompted me to read a thread with the title, "Are
decorators really that different from metaclasses..." Class
decorators wouldn't be so different from metaclasses, but J2 or even
@pie are much better syntaxes than the current metaclass syntax.
In another post in this thread Anthony Baxter said they were
different. I'll respond directly to that post.
later,
Jess
Paul Morrow
>
> Here I think you're making an unwarranted assumption. There is
> currently nothing stopping an object from using information about
> itself to do its job. And this is sometimes a useful freedom. As an
> example, you might have subclasses whose only real purpose is to keep
> track of their class and metadata, and have all functionality provided
> by their superclass:
>
> class supe(object):
> """this docstring is rarely frobnosticated"""
> def do_something_based_on_metadata():
> frobnosticate(self.__doc__)
>
> class sub1(supe):
> """I am sub1"""
>
> class sub2(supe):
> """I am sub2"""
>
What you're trying to illustrate (I believe) is a superclass doing
something based on the docstring of a subclass. Yes this certainly does
happen. But the superclass and subclass are separate objects. I was
talking about the situation where a function does something based on
/its own/ metadata. That is what I'm saying virtually never happens,
and therefore it's ok to make all assignments to __xxx__ attributes
inside of a function def create /function attributes/ rather than /local
variables/.
Paul
Paul Morrow
> and therefore it's ok to make all assignments to __xxx__ attributes
> inside of a function def create /function attributes/ rather than /local
> variables/.
In My Opinion.
Jess Austin
wrote:
>
>
> do with a class decorator by using a metaclass, but in a completely
> different way. In my opinion, class decorators _should_ be in 2.4,
> because they're a much saner way to handle many things that require a
> metaclass today.
So far we agree.
> [snip] A metaclass allows the user
> to specify an object that builds classes in a new way. A decorator
> allows the user to specify a way to transform the already built
> function (or class, for class decorators).
This is effectively the same thing. When you call a class object,
whether it is a classic class, new-style class, class with a
metaclass, decorated class, or some legal combination of the
preceding, it returns an object. You can't have too many expectations
of that object, although typically it will have its __class__ variable
set to the class object you called originally. Anything a metaclass
can do to the objects it creates (which are typically classes), a
decorator could do to objects (whether functions or classes) it
decorates, if only by creating another object and returning it instead
of the original object. I'm assuming that the function returned by
the decorator must accept name, bases, dict as arguments just like a
metaclass must. That returned function will perform in a more
flexible manner since you can also pass arguments to the decorator to
control its behavior - something you couldn't do with a metaclass. A
factory and a filter are equivalent in the sense that once an object
has squirted out the back end, they don't have access to it anymore.
What differences are there? There could be a running-time difference,
but who cares about class instantiation time? Object instantiation
time is what matters. Also, an object the class of which has
__metaclass__ set currently also has __metaclass__ set to the same
thing. This seems like an ill-advised feature, through which
presumably you could do odd and unmaintainable things, although these
could also be done through __class__.__metaclass__. It's also an
inaccurate feature - can a simple object be said to have a metaclass?
To what would this be set for a class whose metaclass had a metaclass?
Anthony Baxter <anthon...@gmail.com> wrote in message news:<mailman.2329.1093405...@python.org>...
This is a difference between classes and functions. But maybe it
could be "swept under the rug" in the following way. Code like this:
using:
metaclass_decorator(baz)
class foo(bar):
a = 1
def moo(self):
pass
could cause something similar to the following:
foo = metaclass_decorator(baz)('foo', (<class '__main__.bar'>,), {'a':
1, 'moo': <function moo at 0x00879E70>, '__module__': '__main__'})
Granted, this is slightly different from what a function decorator
does. But the difference probably won't be felt by those who use
class decorators. Besides the mischief inherent in the introduction
of another namespace floating around behind the __metaclass__ variable
of normal objects, I don't see anything metaclasses can do that class
decorators couldn't. Class decorators can be passed arguments to
control what function they return (see "baz" above), but this doesn't
seem like a major difference. So, no, I don't think that class
decorators would be _that_ different from metaclasses.
later,
Jess
Anthony Baxter
> What you're trying to illustrate (I believe) is a superclass doing
> something based on the docstring of a subclass. Yes this certainly does
> happen. But the superclass and subclass are separate objects. I was
> talking about the situation where a function does something based on
> /its own/ metadata. That is what I'm saying virtually never happens,
> inside of a function def create /function attributes/ rather than /local
> variables/.
This is an extraordinarily complex idea - you're proposing that inside
a function there is now access to a brand new namespace, that of the
function object itself. I don't think you appreciate just _how_ much
work this would requre, nor the complexity of trying to explain this
to users. Remember, at the moment, you can't even get access to the
function object itself from inside the function, without using a hack
like sys._getframe() or raising an exception.
Putting aside nested scopes for the moment, there are three namespaces
that are used in Python - local, module level global, and builtins.
You're proposing that there is a new namespace, that of the function
object, and that it only be used by accessing any local variable that
starts or ends with two underscores. At the moment, the scoping rules
for Python are _extremely_ simple. Do you really want to add something
like this, that looks like a hack, smells like a hack, and who's
implementation would be a hack? I know I don't want to see something
like this.
Anthony Baxter
> Not exactly, but it would include defining those. One way to think
> about metadata is that it includes all information *about* an object,
> but no information *used by* the object.
See, for me, metadata is data about an object that is defined
explicitly as about the object. You can look at an object, and say
"Aha! It supports __getitem__, __iter__ and __len__, and is therefore
likely to be a sequence type object." This is not metadata - this is
something you've determined by looking at the object. __getitem__ and
friends *are* used by the object, and so can't be considered metadata.
I'd even say that __metaclass__ is not metadata in any sense of the
word, because it's *fundamental* to the class's workings.
Something like Zope3's Interfaces, which allow you to specify the
Interfaces that an object or class implements, are closer to metadata,
but note that even there, we've gone from an initial API of:
class Foo:
__implements__ = ( IFoo, )
to
class Foo:
implements(IFoo)
... moving away from the double-under form to a more readable and
useful form. The new form allows for far more powerful and useful
behaviour, such as inheritence of interfaces from base classes.
Twisted has also moved from using the former style to the latter.
These are two _very_ large bodies of code, with a lot of very clueful
people working on them - and they found that the magic double-under
name was inferior to an explicit call syntax.
> If we were to have a conversation about this conversation, then we would
> be having a "meta-conversation." /metadata/ is data that describes
> data. metadata for a function would be data that describes the
> function; data *about* the function as opposed to data *used by* the
> function. For example:
>
> def circumference(diameter):
> """ This is a docstring. It's metadata for this function. """
> __author__ = 'Paul Morrow' # more metadata
> pi = 3.14 # not metadata
> return pi * diameter
There are a small number of double-under names that are used for
metadata - most of them are automatically inserted by Python
automatically. The overwhelming number of double-under names are not
metadata in any way.
Paolo Veronelli
Anthony Baxter wrote:
> On Wed, 25 Aug 2004 19:47:35 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>What you're trying to illustrate (I believe) is a superclass doing
>>something based on the docstring of a subclass. Yes this certainly does
>>happen. But the superclass and subclass are separate objects. I was
>>talking about the situation where a function does something based on
>>/its own/ metadata. That is what I'm saying virtually never happens,
>>and therefore it's ok to make all assignments to __xxx__ attributes
>>inside of a function def create /function attributes/ rather than /local
>>variables/.
>
>
> This is an extraordinarily complex idea - you're proposing that inside
> a function there is now access to a brand new namespace, that of the
> function object itself. I don't think you appreciate just _how_ much
> work this would requre, nor the complexity of trying to explain this
> to users. Remember, at the moment, you can't even get access to the
> function object itself from inside the function, without using a hack
> like sys._getframe() or raising an exception.
It's not the ideas complexity that fears,but the feeling we are trying
to bring down to earth the supposed magic of some language solutions.
There is nothing good in magic,specially when you have to build robust
things on it.
> Putting aside nested scopes for the moment, there are three namespaces
> that are used in Python - local, module level global, and builtins.
> You're proposing that there is a new namespace, that of the function
> object, and that it only be used by accessing any local variable that
> starts or ends with two underscores. At the moment, the scoping rules
> for Python are _extremely_ simple. Do you really want to add something
> like this, that looks like a hack, smells like a hack, and who's
> implementation would be a hack? I know I don't want to see something
> like this.
If this blows away clouds on the language future and bring back the
useful features in a wider theory where metaclasses and decorators are
members of,this shouldn't be considered a hack.If it becomes a hack the
problem is to be searched and solved above generalizing the scope system.
Anthony Baxter
<paolo.v...@yahoo.it> wrote:
> It's not the ideas complexity that fears,but the feeling we are trying
> to bring down to earth the supposed magic of some language solutions.
> There is nothing good in magic,specially when you have to build robust
> things on it.
I have no idea what you're trying to say here. Are you saying that the
approach of double-under variables inside a function having some new
meaning, getting put into a new scope that did not previously exist,
is somehow _less_ magical that the syntactic sugar of decorators? If
so, how do you intend to handle the backwards compatibility issue,
where code that works on Python2.4 will do something entirely
different on Python2.3 (the double-under variables will be silently
ignored). Do you intend that the double-under names would also be
looked for in the same scopes? That is, what will this code do?
def foo():
__name__ = '%s_banana'%(__name__)
> If this blows away clouds on the language future and bring back the
> useful features in a wider theory where metaclasses and decorators are
> members of,this shouldn't be considered a hack.If it becomes a hack the
> problem is to be searched and solved above generalizing the scope system.
Hand waving is all well and good, but this isn't a matter of
"generalizing the scope system". This is a _radical_ change to the
scoping rules of Python, and I think it's safe to say that there's
*absolutely* *no* *way* something like this would be considered,
without an excellent reason - and bolting some sort of strange
decorator semantics doesn't cut it.
Anthony
Paul Morrow
> On Wed, 25 Aug 2004 19:47:35 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>What you're trying to illustrate (I believe) is a superclass doing
>>something based on the docstring of a subclass. Yes this certainly does
>>happen. But the superclass and subclass are separate objects. I was
>>talking about the situation where a function does something based on
>>/its own/ metadata. That is what I'm saying virtually never happens,
>>and therefore it's ok to make all assignments to __xxx__ attributes
>>inside of a function def create /function attributes/ rather than /local
>>variables/.
>
>
> This is an extraordinarily complex idea - you're proposing that inside
> a function there is now access to a brand new namespace, that of the
> function object itself. I don't think you appreciate just _how_ much
> work this would requre, nor the complexity of trying to explain this
> to users. Remember, at the moment, you can't even get access to the
> function object itself from inside the function, without using a hack
> like sys._getframe() or raising an exception.
>
No I'm not suggesting that. The function would continue to have exactly
the same namespace.
This is not a big change; quite the contrary...
The interpreter, when it is executing a def statement, after it binds
__doc__ to the docstring, would execute any assignments to __xxx__
attributes it finds immediately following the docstring (if present). So
def circumference(diameter):
""" This is a docstring. """
__author__ = 'Paul Morrow'
pi = 3.14
return pi * diameter
would be /exactly equivalent/ to
def circumference(diameter):
pi = 3.14
return pi * diameter
circumference.__doc__ = """ This is a docstring. """
circumference.__author__ = 'Paul Morrow'
See how simple? In fact, isn't this just as simple --- or even simpler
-- than what would be require to implement J2 or A1?
And remember, the assumption here is that these assignments (to __xxx__
attributes) are a form of declarations /about the function/, just as
docstrings are now. There's no need to support conditional declarations...
def foo():
""" I am a docstring. """
__author__ = 'Morrow'
# Just as there is no proper, 'definition time' way to
# conditionally change a docstring, the code below
# will generate an exception (at runtime).
if 1:
__version__ = '0.1'
else:
__version__ = '0.2'
Think of what I'm proposing here as just a convenience syntax. We can
already do the same thing after (outsidef of) the function def as the
second circumference example above shows.
Paul
Paul Morrow
> On Wed, 25 Aug 2004 07:20:17 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>Not exactly, but it would include defining those. One way to think
>>about metadata is that it includes all information *about* an object,
>>but no information *used by* the object.
>
>
> See, for me, metadata is data about an object that is defined
> explicitly as about the object. You can look at an object, and say
> "Aha! It supports __getitem__, __iter__ and __len__, and is therefore
> likely to be a sequence type object." This is not metadata - this is
> something you've determined by looking at the object. __getitem__ and
> friends *are* used by the object, and so can't be considered metadata.
> I'd even say that __metaclass__ is not metadata in any sense of the
> word, because it's *fundamental* to the class's workings.
>
Ok. Then let's jettison the baggage with the word metaclass and just
call them 'magic attributes'. It doesn't matter. The essential point
about them is that they almost /never have 'local variable' semantics/.
They are attributes of the object being defined. That's the point I'm
trying to drive home here. When we use __xxx__ attributes, we are not
intending to create local variables. No. We are making declarations.
> Something like Zope3's Interfaces, which allow you to specify the
> Interfaces that an object or class implements, are closer to metadata,
> but note that even there, we've gone from an initial API of:
>
> class Foo:
> __implements__ = ( IFoo, )
>
> to
> class Foo:
> implements(IFoo)
>
> ... moving away from the double-under form to a more readable and
> useful form. The new form allows for far more powerful and useful
> behaviour, such as inheritence of interfaces from base classes.
> Twisted has also moved from using the former style to the latter.
> These are two _very_ large bodies of code, with a lot of very clueful
> people working on them - and they found that the magic double-under
> name was inferior to an explicit call syntax.
>
Woah Man! I thought you wanted special functionality to have special
syntax, so that it really sticks out that we're doing something special?
It sounds like now you are /endorsing/ the use of normal looking syntax
that does special things.
Personally, the look of the old '__implements__' syntax was vastly
superior. In an instant, an experienced pythonista could see that
magic was being peformed on Foo. That fact is obscured in the new
'implements' syntax.
Wait. Did I just argue your point from an earlier thread? :-) See, we
do (or at least 'did') agree on some things...
>
>>If we were to have a conversation about this conversation, then we would
>>be having a "meta-conversation." /metadata/ is data that describes
>>data. metadata for a function would be data that describes the
>>function; data *about* the function as opposed to data *used by* the
>>function. For example:
>>
>> def circumference(diameter):
>> """ This is a docstring. It's metadata for this function. """
>> __author__ = 'Paul Morrow' # more metadata
>> pi = 3.14 # not metadata
>> return pi * diameter
>
>
> There are a small number of double-under names that are used for
> metadata - most of them are automatically inserted by Python
> automatically. The overwhelming number of double-under names are not
> metadata in any way.
I don't need to debate that for my position to be valid. How about from
now on we just call it magic data? That works fine for me. As long as
we're clear that it's not (intended to be) 'local data'. That it has
more of a declaration semantics.
Paul
Paul Morrow
> On Thu, 26 Aug 2004 10:28:14 +0200, Paolo Veronelli
> <paolo.v...@yahoo.it> wrote:
>
> ignored). Do you intend that the double-under names would also be
> looked for in the same scopes? That is, what will this code do?
>
> def foo():
> __name__ = '%s_banana'%(__name__)
>
/exactly/ the same thing as
def foo(): pass
foo.__name__ = '%s_banana'%(foo.__name__)
Paul
Arthur
Though I am learning stuff by trying to keep up with this conversation
- I should probably stay out of it, directly.
But, as I publically demonstrated in my confusion about __name__
...to someone educated from within Python, mostly by chucking out
code, rather than reading documentation - that __name__ accessed
from within the function body is something totally different from
__name__ accessed from outside of it, was not what I think the
language taught me to expect. I might have intuited that to expect
otherwise would be "magic", but thought, somehow, that the __xxx__
implied the availability of that kind of magic.
In other words, it would have surprised me less to have been right
than to have benn wrong.
Though I have learned not to be shocked about being wrong ;)
And can understand the possiblity that others with other backgrounds
might be surprised if I had been right.
Does
"""
Namespaces are one honking great idea -- let's do more of those!
"""
come into play here at all?
Art
Arthur
<anthon...@gmail.com> wrote:
>
>def foo():
> __name__ = '%s_banana'%(__name__)
>
I'm not sure what you are driving at here, but ...
Aa a practical matter, wouldn't it be nice to be able to use string
substitution on a docstring placed in its usual location under def
foo().
That I need to place my __doc__ under the function can't be considered
a good thing, or intutive thing.
Sort of like having to do transformations after and under the function
;)
Art
Arthur
wrote:
But you can perhaps respond that with a2 the flexibility to put the
doc string where you think it belongs, before and above the function
(and I don't necessarily disagree, in that that it is always where I
find myself putting #comments (though perhaps I am putting them there
in particular to *distinguish* them from a docstring (where would I
put comments if my docstring were above (perhaps below)))) -
that you and I will be placing are docstrings differently can be seen
both as a benefit *and as a cost*, of the a2 implmentation.
Art
Paul Morrow
Currently, you can change a doc string outside of a function, by
modifying the function's __doc__ attribute
>>> def foo():
... """ I am foo """
...
>>> foo.__doc__
' I am foo '
>>> foo.__doc__ = foo.__doc__ + 'indeed'
>>> foo.__doc__
' I am foo indeed'
>>>
IMO, to change it inside of a function def should be (but isn't) as easy
as...
>>> def foo():
... """ I am foo """
... __doc__ = __doc__ + 'indeed'
Paul
Paul Morrow
>
> Ok. Then let's jettison the baggage with the word metaclass and just
> call them 'magic attributes'. It doesn't matter. The essential point
> about them is that they almost /never have 'local variable' semantics/.
> They are attributes of the object being defined. That's the point I'm
> trying to drive home here. When we use __xxx__ attributes, we are not
> intending to create local variables. No. We are making declarations.
>
...to...
Ok. Then let's jettison the baggage with the word metadata and just
call them 'magic attributes'. It doesn't matter. The essential point
about them is that they almost /never have 'local variable' semantics/.
They are attributes of the object being defined. That's the point I'm
trying to drive home here. When we define __xxx__ attributes, we are
not intending to create local variables. No. We are making declarations.
Paul
Arthur
wrote:
>
>Currently, you can change a doc string outside of a function, by
>modifying the function's __doc__ attribute
>
> >>> def foo():
> ... """ I am foo """
> ...
> >>> foo.__doc__
> ' I am foo '
> >>> foo.__doc__ = foo.__doc__ + 'indeed'
> >>> foo.__doc__
> ' I am foo indeed'
> >>>
>
>IMO, to change it inside of a function def should be (but isn't) as easy
>as...
>
> >>> def foo():
> ... """ I am foo """
> ... __doc__ = __doc__ + 'indeed'
>
>Paul
Yes. Not only do I follow, but I think we came to exactly the same
place, from very different directions, and coming from what I sense is
very different backgrounds.
Its just that I don't think many others seem to find that as
interesting as I happen to.
Hopefully, for good enough reasons.
Art
>
>
Anthony Baxter
> >IMO, to change it inside of a function def should be (but isn't) as easy
> >as...
> >
> > >>> def foo():
> > ... """ I am foo """
> > ... __doc__ = __doc__ + 'indeed'
> >
> >Paul
>
> Yes. Not only do I follow, but I think we came to exactly the same
> place, from very different directions, and coming from what I sense is
> very different backgrounds.
>
> Its just that I don't think many others seem to find that as
> interesting as I happen to.
Not so much that, as running out of ways to restate myself. The
proposed syntax above still requires magic handling of double-under
variables in a function, and a new namespace. I can't see how you can
think that this is a _good_ thing.
Anthony Baxter
> Ok. Then let's jettison the baggage with the word metadata and just
> call them 'magic attributes'. It doesn't matter. The essential point
> about them is that they almost /never have 'local variable' semantics/.
> They are attributes of the object being defined. That's the point I'm
> trying to drive home here. When we define __xxx__ attributes, we are
> not intending to create local variables. No. We are making declarations.
Explain to me how __getitem__ is a declaration while getMonkey is not,
assuming both are methods.
The only magic attribute that fits your case of "magic attribute" is
__metaclass__.
The overwhelming number of double-under names are not magic
attributes, or metadata, or whatever you want to call them.
Paul Morrow
>>>as...
>>>
>>> >>> def foo():
>>> ... """ I am foo """
>>> ... __doc__ = __doc__ + 'indeed'
>>>
>>>Paul
>>
>>Yes. Not only do I follow, but I think we came to exactly the same
>>place, from very different directions, and coming from what I sense is
>>very different backgrounds.
>>
>>Its just that I don't think many others seem to find that as
>>interesting as I happen to.
>
>
> proposed syntax above still requires magic handling of double-under
> variables in a function, and a new namespace. I can't see how you can
> think that this is a _good_ thing.
The function does *not* get a new namespace! Let me stress this point a
little further. We would simply be moving __xxx__ variables *out of*
the function's local variable namespace to where they belong, the
namespace of the function itself --- /the same namespace that __doc__
lives in./
As such, the function *body* would have no more access to __doc__ than
it does now. Consider the following
def foo():
""" foo's docstring """
__author__ = 'Paul Morrow'
__version__ = '0.1'
__features__ = memoized, synchronized
# Above this line is the function's declaration section.
# Below this line is where the function's body starts.
var1 = 10
print __doc__ # error! __doc__ is not in local namespace
print var1 # ok.
Paul
Paul Morrow
__getitem__ is most certainly magical! Defining it 'declares'
(implicitly, but we'll ignore that governing zen rule for the moment)
that instances of the containing class have dictionary semantics (that
they can be used, in some degree, like dictionaries). That's magic.
That's meta. That's profoundly deeper than anything defining getMonkey
does.
Anthony Baxter
> The function does *not* get a new namespace! Let me stress this point a
> little further. We would simply be moving __xxx__ variables *out of*
> the function's local variable namespace to where they belong, the
> namespace of the function itself --- /the same namespace that __doc__
> lives in./
There _is_ _no_ _such_ _namespace_. __doc__ is an attribute of the
function object.
You are proposing to add one. How else is __doc__ = __doc__ + 'extra' to work?
Please, please look into how python's namespaces work a bit more. This
isn't super-complex stuff, and the lovely thing about Python is that
it's _really_ easy to use introspection to see how these things work.
Arthur
Forgive me, that I am not necessarily able to grasp why it is
necessarily a bad thing. or a badder thing than other things. But do
not discount at all your advice that we are in pie-in-the-sky land.
The weight of evidence - being the general lack of interest in this
thread, being taken into account.
But as I explained earlier, I was hoping some fortipation (or other
mojo someone might provide) might make this come together some.
Ain't to be if it ain't to be.
Art
Anthony Baxter
> __getitem__ is most certainly magical! Defining it 'declares'
> (implicitly, but we'll ignore that governing zen rule for the moment)
> that instances of the containing class have dictionary semantics (that
> they can be used, in some degree, like dictionaries). That's magic.
> That's meta. That's profoundly deeper than anything defining getMonkey
> does.
What? There is *nothing* that __getitem__ "declares". __getitem__ is
used by the interpreter. When do make a call like:
someobj[key]
the interpreter turns that into
someobj.__getitem__(key)
That's all. I can make an object that acts like a dictionary _without_
using a __getitem__, watch:
class Foo:
def __init__(self):
self.d = dict(ape=False,spidermonkey=True)
def getMonkey(self, key):
return self.d[key]
def __getattr__(self, name):
if name == "__getitem__":
return self.getMonkey
else:
raise AttributeError, name
f = Foo()
print f['ape']
print f['spidermonkey']
Paolo Veronelli
Anthony Baxter wrote:
> On Thu, 26 Aug 2004 10:28:14 +0200, Paolo Veronelli
> <paolo.v...@yahoo.it> wrote:
>
>>It's not the ideas complexity that fears,but the feeling we are trying
>>to bring down to earth the supposed magic of some language solutions.
>>There is nothing good in magic,specially when you have to build robust
>>things on it.
>
>
> I have no idea what you're trying to say here. Are you saying that the
> approach of double-under variables inside a function having some new
> meaning, getting put into a new scope that did not previously exist,
> is somehow _less_ magical that the syntactic sugar of decorators?
I would like to erase magic from code and have some kind of
specification for the double-under variables.
if __metaclass__ is not a keyword I suppose it's a reference
but.....this is not enough:assigning it is a meta-coding action.
So let's classify these actions and see what kind of meta-coding
decorators are .
This should be done before the creation of a new syntax,a good syntax
for all meta-coding.
> so, how do you intend to handle the backwards compatibility issue,
> where code that works on Python2.4 will do something entirely
> different on Python2.3 (the double-under variables will be silently
> ignored).
This is not a problem for me.If it's the case I don't really care of
zoombies,I'm a researcher not a politician,
I'm trying to develope a thought around a pragmatic good language,and
possibly evolve my knowledge of it.
Next generation will measure better our efforts.
> Do you intend that the double-under names would also be
> looked for in the same scopes? That is, what will this code do?
>
> def foo():
> __name__ = '%s_banana'%(__name__)
>
Has it anything to do with our talking?Sharp features are sharp.If
somebody want to obfuscate code he can do it,he is free of playing this
game.
>>If this blows away clouds on the language future and bring back the
>>useful features in a wider theory where metaclasses and decorators are
>>members of,this shouldn't be considered a hack.If it becomes a hack the
>>problem is to be searched and solved above generalizing the scope system.
>
>
> Hand waving is all well and good, but this isn't a matter of
> "generalizing the scope system". This is a _radical_ change to the
> scoping rules of Python, and I think it's safe to say that there's
> *absolutely* *no* *way* something like this would be considered,
> without an excellent reason - and bolting some sort of strange
> decorator semantics doesn't cut it.
We can use decorators issue to see how python successors will kill it
in the long term or how the python will be
after changing skin.
In my enfant views knowledge is fractal and a the three scoping rules is
hiding patterns which will be seen with the right lens.
Possibly we can learn form other languages approches to the same
problems but no way I matter to be considered from the higher spheres,I
am a farmer taking some notes on how the plant is growing.
Paolino
Arthur
Googling on "function namespace" is interesting.
Subject to my interpretation of what I am accessing:
Common Lisp has 'em, Scheme don't.
Talked about a lot in the context of XML in ways I am not sure are
relevant.
Nested scopes is the beginning of their introduction into Python - as
I am reading a cite from Dive Into Python. So of course what I have
been driving at all along, is some extension to the the nested scope
mechanism. Yes. I'm talking through my backside. Darts at the wall
language design, I call it.
Art
Bengt Richter
No, it's convention. The convention is that you should write __getitem__
methods for YourClass so that if ycinst = YourClass() then ycinst[key_or_index]
will not break expectation of dict-like or sequence-like access too badly.
But you are free to break the convention.
The __getitem__ name serves as a hook into implementation of ycinst[key_or_index]
which has been designed to use __getitem__ in a predictable way. Similarly with
other double-underscore methods. The names don't declare anything. Their use in
a predictable way by generated code just gives you the opportunity to hook in
your own magic for source that translates to use of such methods.
Your proposal exposes the peculiar sematics of the doc string, which really should
become a discarded string expression where it now appears. IMO it might have been as
good or better to capture text from the first block of comments inside a function
and bind __doc__ to that. Then statement semantics would have been more consistent.
But you are going in the other direction, expanding on special statement semantics
in special contexts, and making double underscore names into special sugar for
something more than names, not just special names with special conventional bindings.
I.e., your __xxx__ becomes a spelling for foo.__xxx__ etc. and an implied moving of
the code to post-definition execution context. I'd rather do that with a prefixed
using: block than with special interpretation of names.
I would rather see see some way to access the current object and its environment
without using its post-definition name binding, and controlling _when_ statements
execute. The decorator syntax provide a when of post-def-execution-pre-name-binding
and it provides current-object binding via the binding to the parameter in the
decorator call. The _when_ for statements in a function body is during-call-execution.
I'd like some of the internal whens to be exposed and hookable. E.g., expanding on
try/finally to provide keyword:suite syntax for what is e.g. currently done with
default-argument hack -- i.e., def-execution-time initialization of values
for every-call-time initialization of local bindings to those values. A version of
that might provide persistent local bindings -- like a private writable closure.
<musing>
A __self__ magic binding could provide access to the current object, and __self__.__enclosing__
could be magic for accessing the textually enclosing entity, which might have another enclosing
entity, so __self__.__enclosing__.__enclosing__ would then be legal. Dynamic spaces similarly,
perhaps locals() and outwards to locals(1) etc.
</musing>
>That's meta. That's profoundly deeper than anything defining getMonkey
>does.
ISTM you are looking at __getitem__ from the wrong end of the magic.
The name is a standard method name that various specific source code texts
will generate code to access, e.g. x[3] or x.__class__.__dict__['__getitem__'](x, 3).
The magic is in what people have done with the names, not the names themselves.
The source spelling for generating code using the __metaclass__ name is admittedly
a bit more mysterious than x[y] generating code using __getitem__ ;-)
Regards,
Bengt Richter
Paul Morrow
Yes, it doesn't seem all that complex, although I'm not sure that
everyone reading this understands them and their subtleties. The
following is an excerpt from
http://docs.python.org/tut/node11.html#SECTION0011200000000000000000
"A namespace is a mapping from names to objects. Most namespaces are
currently implemented as Python dictionaries, but that's normally not
noticeable in any way (except for performance), and it may change in the
future. Examples of namespaces are: the set of built-in names (functions
such as abs(), and built-in exception names); the global names in a
module; and the local names in a function invocation. In a sense the set
of attributes of an object also form a namespace."
When I talk about namespaces, I include all of the above, including the
sense mentioned in the last line. So an object's attributes constitute
a namespace too. Therefore __doc__, being an attribute of the function
object, is in the function object's /namespace/. And note that this is
*not* a new namespace; it's been there all along.
So now to answer your question about how is __doc__ = __doc__ + 'extra'
going to work. It's simple. But first, we need to define a term. For
the sake of this discussion, let's say that 1) specifying a docstring,
and 2) assigning to a top-level [*] __xxx__ variable inside of a
function def is making a *'declaration'*. So in the following function
def, there are three declarations and one simple assignment.
def foo():
""" I am a docstring. """ # declaration
__doc__ = __doc__ + 'extra' # declaration
__author__ = 'Morrow' # declaration
pi = 3.14 # not a declaration
Ok? Now declarations would *not* be like local variable definitions.
Declarations would be executed in the namespace of the object being
defined.
"Declarations encountered during the execution of a def
statement are executed in the namespace of the function
or method object being defined."
So in the above example, the three declarations would be executed in the
namespace of the foo function object. They would *not* create local
variables [**]. To help visualize this, the following would be exactly
equivalent:
def foo():
pi = 3.14 # not a declaration
foo.__doc__ = """ I am a docstring. """
foo.__doc__ = foo.__doc__ + 'extra'
foo.__author__ = 'Morrow'
Paul
* ie. not within a subordinate code block, such as a branch of an if
statement.
** if the Python system was so modified.
Jess Austin
> This is a difference between classes and functions. But maybe it
> could be "swept under the rug" in the following way. Code like this:
>
> using:
> metaclass_decorator(baz)
> class foo(bar):
> a = 1
> def moo(self):
> pass
>
> could cause something similar to the following:
>
> foo = metaclass_decorator(baz)('foo', (<class '__main__.bar'>,), {'a':
> 1, 'moo': <function moo at 0x00879E70>, '__module__': '__main__'})
So I thought a bit about this, and I think a better equivalent would
be:
foo = metaclass_decorator(baz)(type('foo', (<class '__main__.bar'>,),
{'a': 1,
'moo': <function moo at
0x00879E70>,
'__module__': '__main__'}))
Smart pythonistas have probably already thought of this. It just took
me a day or two.
later,
Jess
Jess Austin
> Currently, you can change a doc string outside of a function, by
> modifying the function's __doc__ attribute
[snip]
> IMO, to change it inside of a function def should be (but isn't) as easy
> as...
>
> >>> def foo():
> ... """ I am foo """
> ... __doc__ = __doc__ + 'indeed'
I'm not trying to be mean, but why not use a class for this? This is
the whole point of "self" - provide a persistent, updateable namespace
for an object. (Note: explicit use of self in Python is a GOOD
THING!) An object that needs such a namespace should be a class. I'm
not sure why or how a function could use this anyway - why not just
set the docstring to " I am foo indeed" to start with? If what you're
looking for is a "configured function", which contains some data (or
metadata) that is set at the beginning of the function's lifecycle, to
me that would be a closure, which could be written like this:
def closure_maker(postfix):
def foo():
""" I am foo """
pass
foo.__doc__ += postfix
return foo
Or perhaps there is another application you had in mind, for which
this wouln't be sufficient? Paul: I admit I'm a bit confused by your
wanting a function to have access to its metadata, because I could
have sworn that earlier in the thread you defined metadata as data
about an object that that object would never use itself. I could be
thinking of someone else though...
later,
Jess
Paul Morrow
> Paul Morrow <pm_...@yahoo.com> wrote in message news:<mailman.2423.1093527...@python.org>...
>
>>Currently, you can change a doc string outside of a function, by
>>modifying the function's __doc__ attribute
>
>
> [snip]
>
>
>>IMO, to change it inside of a function def should be (but isn't) as easy
>>as...
>>
>> >>> def foo():
>> ... """ I am foo """
>> ... __doc__ = __doc__ + 'indeed'
>
>
> I'm not trying to be mean, but why not use a class for this?
This is a contrived example; we probably wouldn't do this in practice.
But some of us would like to do something like this
def foo():
__features__ = synchronized, memoized
...so that the function foo get's some special abilities from the
'decorators' (or whatever we end up calling them) assigned to the
__features__ attribute.
But before anyone is really going to accept that way of specifying a
function's decorators, they need to be convinced that it's ok to think
of __features__ as an attribute of the function foo (i.e. foo metadata),
rather than a local variable.
> Paul: I admit I'm a bit confused by your
> wanting a function to have access to its metadata, because I could
> have sworn that earlier in the thread you defined metadata as data
> about an object that that object would never use itself. I could be
> thinking of someone else though...
>
Nope, that was me alright. I don't want a function to have access to
it's metadata. Nothing's changed about that. I want to be able to
specify a function's metadata inside the function def, where it seems to
most appropriately belong.
Others seem to want to specify a function's metadata outside of the
function def, which just doesn't seem pythonic (IMO).
Arthur
wrote:
>Paul Morrow <pm_...@yahoo.com> wrote in message news:<mailman.2423.1093527...@python.org>...
>> Currently, you can change a doc string outside of a function, by
>> modifying the function's __doc__ attribute
>
>[snip]
>
>> IMO, to change it inside of a function def should be (but isn't) as easy
>> as...
>>
>> >>> def foo():
>> ... """ I am foo """
>> ... __doc__ = __doc__ + 'indeed'
>
>I'm not trying to be mean, but why not use a class for this?
I believe he was only being illustrative.
My use case:
(unless I'm being stupid)
IDLE 1.0.3
>>>class foo:
hint ="boo"
def bar(self):
"bar using %s" %hint
pass
>>> f=foo()
>>> print f.bar.__doc__
None
vs.
>>> class foo:
hint ="boo"
def bar(self):
pass
bar.__doc__="bar using %s" %hint
>>> f=foo()
>>> print f.bar.__doc__
bar using boo
But to me more important, in the context of the overall discussion, is
in seeing some good connectedness between the problem of being forced
outside and below the function to use string substitution on its
docstring, and the limited range of options that seem to be available
to try to solve the percieved syntax problems related to
transformations (pre a2) being stuck outside and below.
Is the apparent connectedness only skin deep?
It's not making sense to me that is.
Art
Paul Morrow
> On Thu, 26 Aug 2004 11:15:46 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>__getitem__ is most certainly magical! Defining it 'declares'
>>(implicitly, but we'll ignore that governing zen rule for the moment)
>>that instances of the containing class have dictionary semantics (that
>>they can be used, in some degree, like dictionaries). That's magic.
>>That's meta. That's profoundly deeper than anything defining getMonkey
>>does.
>
>
> What? There is *nothing* that __getitem__ "declares". __getitem__ is
> used by the interpreter.
Sure the interpreter uses __getitem__, but you use it too, right? I
mean, you don't need the interpreter to tell you whether instances of
the following class can act like a dictionary, do you?
class Foo:
def __getitem__(self, x): pass
No you don't. Nor do you need to pretend that you're the interpreter,
and simulate a call to an instance of Foo. That's because the mere
presence of def __getitem__ tells you what you need to know. In this
way, __getitem__ *does* serve as a declaration (of dictionary semantics)
[*] to you and anyone reading your code.
* But an implicit declaration of course. I guess the only way of
*explicitly* stating that instances of a class can act like dictionaries
is to have the class inherit from dict or UserDict.
> used by the interpreter. When do make a call like:
>
> someobj[key]
>
> the interpreter turns that into
>
> someobj.__getitem__(key)
>
> That's all. I can make an object that acts like a dictionary _without_
> using a __getitem__, watch:
>
> class Foo:
> def __init__(self):
> self.d = dict(ape=False,spidermonkey=True)
>
> def getMonkey(self, key):
> return self.d[key]
>
> def __getattr__(self, name):
> if name == "__getitem__":
> return self.getMonkey
> else:
> raise AttributeError, name
>
> f = Foo()
> print f['ape']
> print f['spidermonkey']
Now hold on just a minute. If I squint a little, I can still see
__getitem__ in there somewhere.
Though I do believe that you are illustrating a good point; that you can
mangle your code badly enough that it no longer lends itself to a
declarative reading. And then, the only way for your poor reader to
figure out what the heck it's doing is for him/her to act like the
interpreter and walk through your code (or run it under the debugger).
But I guess if you had a good enough reason...
Andrew Durdin
>
> Sure the interpreter uses __getitem__, but you use it too, right? I
> mean, you don't need the interpreter to tell you whether instances of
> the following class can act like a dictionary, do you?
>
> class Foo:
> def __getitem__(self, x): pass
>
> No you don't. Nor do you need to pretend that you're the interpreter,
> and simulate a call to an instance of Foo. That's because the mere
> presence of def __getitem__ tells you what you need to know. In this
> way, __getitem__ *does* serve as a declaration (of dictionary semantics)
> [*] to you and anyone reading your code.
That's because the dictionary interface/protocol requires a
__getitem__ method; there's no magical/technical reason why it should
be called __getitem__ rather than, say, dict_get_item.
As an example, the iterator protocol requires two methods: __iter__()
and next() -- the second doesn't have the double underscores, yet
still allows you to do:
class Foo:
def __iter__(self):
return self
def next(self):
return 42
for v in Foo():
print v
(yes, that's an infinite loop, but that's irrelevant). All the "magic"
__X__ methods are just those methods required by certain built-in
interfaces/protocols. And there's syntactic sugar to make them
convenient. They don't affect the creation of the class at all, as you
can (if necessary) add them in after the fact:
>>> class Foo:
... pass
...
>>> for i in Foo():
... pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence
>>> def bar(self):
... return self
...
>>> def baz(self):
... raise StopIteration()
...
>>> Foo.__iter__ = bar
>>> Foo.next = baz
>>> for i in Foo():
... pass
...
>>>
Anthony Baxter
> Anthony Baxter wrote:
> Sure the interpreter uses __getitem__, but you use it too, right? I
> mean, you don't need the interpreter to tell you whether instances of
> the following class can act like a dictionary, do you?
>
> class Foo:
> def __getitem__(self, x): pass
Well, actually, no. The presence or absence of __getitem__ "tells" you nothing.
It depends on the implementation.
def __getitem__(self, key): raise AttributeError, '__getitem__'
> No you don't. Nor do you need to pretend that you're the interpreter,
> and simulate a call to an instance of Foo. That's because the mere
> presence of def __getitem__ tells you what you need to know. In this
> way, __getitem__ *does* serve as a declaration (of dictionary semantics)
> [*] to you and anyone reading your code.
No. It. Does. Not. "dictionary semantics" includes a hell of a lot
more than a "__getitem__". And list-like objects _also_ include a
__getitem__.
> * But an implicit declaration of course. I guess the only way of
> *explicitly* stating that instances of a class can act like dictionaries
> is to have the class inherit from dict or UserDict.
Or use Interfaces. Or put something in the docstring.
> > class Foo:
> > def __init__(self):
> > self.d = dict(ape=False,spidermonkey=True)
> >
> > def getMonkey(self, key):
> > return self.d[key]
> >
> > def __getattr__(self, name):
> > if name == "__getitem__":
> > return self.getMonkey
> > else:
> > raise AttributeError, name
> >
> > f = Foo()
> > print f['ape']
> > print f['spidermonkey']
>
> Now hold on just a minute. If I squint a little, I can still see
> __getitem__ in there somewhere.
As a *string*, buried away in the innards. So I make that string get
loaded from another object, or from a database, or whatever. Your
point is _wrong_. You cannot "declare" anything by simply the presence
or absence of magic double-under methods.
> Though I do believe that you are illustrating a good point; that you can
> mangle your code badly enough that it no longer lends itself to a
> declarative reading. And then, the only way for your poor reader to
> figure out what the heck it's doing is for him/her to act like the
> interpreter and walk through your code (or run it under the debugger).
Or read the docstring for the class.
Anthony, giving up.
Paul Morrow
> On Thu, 26 Aug 2004 20:54:22 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>Sure the interpreter uses __getitem__, but you use it too, right? I
>>mean, you don't need the interpreter to tell you whether instances of
>>the following class can act like a dictionary, do you?
>>
>> class Foo:
>> def __getitem__(self, x): pass
>>
>>and simulate a call to an instance of Foo. That's because the mere
>>presence of def __getitem__ tells you what you need to know. In this
>>way, __getitem__ *does* serve as a declaration (of dictionary semantics)
>>[*] to you and anyone reading your code.
>
>
> __getitem__ method; there's no magical/technical reason why it should
> be called __getitem__ rather than, say, dict_get_item.
Um. Actually there is a reason why __getitem__ should be named
something that starts and ends with double underscores. It's magical.
By defining that method, instances of your class have special behavior.
The *convention* is to surround magic methods with double underscores.
So no, dict_get_item would not be a proper substitute name.
> As an example, the iterator protocol requires two methods: __iter__()
> and next() -- the second doesn't have the double underscores
Ok I'll just say it (ban me from the newsgroup if this stings too much).
The 'next' method is poorly named. It doesn't follow conventions for
magic methods. It should be fixed. It should be __next__
>
> All the "magic"
> __X__ methods are just those methods required by certain built-in
> interfaces/protocols. And there's syntactic sugar to make them
> convenient. They don't affect the creation of the class at all, as you
> can (if necessary) add them in after the fact:
>
Of course you can. In fact, you could do all of your programming that
way, where you create an empty class, then add the methods and
attributes after the fact. Even more fun would be to not even
explicitly create the class (with a class statement), but instead call a
function that returns an empty class, and then add your attributes to
the function result.
But that's not a very effective way to program if you care about program
maintenance. As a reader of your code, I'll have a much harder time
understanding what you're doing than if you adhered to basic conventions
(like defining all of your methods and attributes inside of a class
statement).
But my point was that the presence of certain magic methods/attributes
'says' [*] something to the reader about our code.
Paul
* implicitly 'declares'.
Paul Morrow
>>Sure the interpreter uses __getitem__, but you use it too, right? I
>>mean, you don't need the interpreter to tell you whether instances of
>>the following class can act like a dictionary, do you?
>>
>> class Foo:
>> def __getitem__(self, x): pass
>
>
> It depends on the implementation.
>
> def __getitem__(self, key): raise AttributeError, '__getitem__'
>
Oh my.
Following conventions helps readers tremendously. Once a pythonista
learns a convention, he doesn't spend as long looking at code that
(appears to) follow that convention. He *assumes* that the convention
means the same thing in the code he's reading as it does every else.
Good developers are sensitive to this. Good developers write code that
follows conventions.
One convention is that classes whose instances can return values via
indexing (e.g. foo['x']) define the __getitem__ method. /Because of
this convention/, if we create a class that defines the __getitem__
method, our readers will expect that instances of that class can return
values via indexing.
Of course this means that we can have some fun with our readers too. We
can create classes that define the __getitem__ method but which raise an
exception when we try to use the syntax __getitem__ enables [*]. But
while it may be fun to confuse readers of our code [**], its generally a
bad idea.
We all know that conventions have meaning. When we properly use
conventions in our code, we are telling the reader a little (or in some
cases a lot) about what is going on in our code, without us having to
come right out and say it. This saves us time. This saves the reader
time. This improves the maintainability of our code.
So conventions are good. And because of that, I propose the following
as the 20th Zen principle:
Conventional is better than unconventional.
Paul
* as you did in your example.
** by writing code that follows conventions but doesn't exhibit the
associated conventional behavior.
Colin J. Williams
Paul Morrow wrote:
[snip]
>
>
> Yes, it doesn't seem all that complex, although I'm not sure that
> everyone reading this understands them and their subtleties. The
> following is an excerpt from
> http://docs.python.org/tut/node11.html#SECTION0011200000000000000000
>
> "A namespace is a mapping from names to objects. Most namespaces are
> currently implemented as Python dictionaries, but that's normally not
> noticeable in any way (except for performance), and it may change in the
> future. Examples of namespaces are: the set of built-in names (functions
> such as abs(), and built-in exception names); the global names in a
> module; and the local names in a function invocation. In a sense the set
> of attributes of an object also form a namespace."
>
> When I talk about namespaces, I include all of the above, including the
> sense mentioned in the last line. So an object's attributes constitute
> a namespace too. Therefore __doc__, being an attribute of the function
> object, is in the function object's /namespace/. And note that this is
> *not* a new namespace; it's been there all along.
>
Could you elaborate on the last sentence please? Is the namespace not
created when the def ... line(s) is/are executed?
Colin W.
Paul Morrow
If Python waits until then, where does it put the docstring in the
meantime? I haven't actually looked at the code [*], but it would make
sense for the function's namespace to be created at the beginning of the
def execution (so that it had a place for the __doc__ attribute when the
docstring was encountered).
* Oh Anthony will want to jump on that one :-)
Anthony Baxter
another piece of misinformation before I go, in case the 3 or 4 people
who've stayed with this thread get the wrong idea <wink>]
On Thu, 26 Aug 2004 16:09:42 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
> Yes, it doesn't seem all that complex, although I'm not sure that
> everyone reading this understands them and their subtleties. The
> following is an excerpt from
> http://docs.python.org/tut/node11.html#SECTION0011200000000000000000
>
> "A namespace is a mapping from names to objects. Most namespaces are
> currently implemented as Python dictionaries, but that's normally not
> noticeable in any way (except for performance), and it may change in the
> future. Examples of namespaces are: the set of built-in names (functions
> such as abs(), and built-in exception names); the global names in a
> module; and the local names in a function invocation. In a sense the set
> of attributes of an object also form a namespace."
> When I talk about namespaces, I include all of the above, including the
> sense mentioned in the last line. So an object's attributes constitute
> a namespace too. Therefore __doc__, being an attribute of the function
> object, is in the function object's /namespace/. And note that this is
> *not* a new namespace; it's been there all along.
"In a sense" is the bit you're missing here. You can't just hand-wave
and say that it's a namespace. It's *not* a namespace. If it was, you
could do any of these:
>>> def foo():
... "a docstring"
...
>>>
>>> eval('__doc__', foo)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: globals must be a dict
>>> eval('__doc__', globals(), foo)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: locals must be a mapping
>>> exec '__doc__ = __doc__ + "extra"' in foo
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: exec: arg 2 must be a dictionary or None
Note that if I use a real dict, I get the correct result:
>>> d=dict(__doc__ = 'docstring')
>>> exec '__doc__ = __doc__ + "extra"' in d
>>> d['__doc__']
'docstringextra'
If and when you can make the above work, then come back and re-visit
your idea. Better yet, play around a bit and figure out _why_ they
don't work - it's a good learning exercise.
Anthony
Paul Morrow
> On Thu, 26 Aug 2004 16:09:42 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
>
>>Yes, it doesn't seem all that complex, although I'm not sure that
>>everyone reading this understands them and their subtleties. The
>>following is an excerpt from
>>http://docs.python.org/tut/node11.html#SECTION0011200000000000000000
>>
>>"A namespace is a mapping from names to objects. Most namespaces are
>>currently implemented as Python dictionaries, but that's normally not
>>noticeable in any way (except for performance), and it may change in the
>>future. Examples of namespaces are: the set of built-in names (functions
>>such as abs(), and built-in exception names); the global names in a
>>module; and the local names in a function invocation. In a sense the set
>>of attributes of an object also form a namespace."
>
>
>>When I talk about namespaces, I include all of the above, including the
>>sense mentioned in the last line. So an object's attributes constitute
>>a namespace too. Therefore __doc__, being an attribute of the function
>>object, is in the function object's /namespace/. And note that this is
>>*not* a new namespace; it's been there all along.
>
>
> "In a sense" is the bit you're missing here. You can't just hand-wave
> and say that it's a namespace. It's *not* a namespace. If it was, you
> could do any of these:
>
No I didn't miss it (I read that bit). In a sense, every object has
it's own namespace. So that means that every class, every module, every
function has a namespace (in a sense). The namespace (the mapping from
names to objects) is stored in each object's special __dict__ attribute.
Ok, and this is certainly apparent with classes.
>>> class Foo:
... """docstring for Foo"""
...
>>> Foo.__doc__
'docstring for Foo'
>>> Foo.__dict__.keys()
['__module__', '__doc__']
>>>
And because of the parallels between the above class definition of Foo
and the following function definition of baz, I would expect the same
behavior.
>>> def baz():
... """docstring of baz"""
...
>>> baz.__doc__
'docstring of baz'
>>> baz.__dict__.keys()
[]
Say what? Why didn't the Python system put baz's docstring into it's
namespace (__dict__)? And where did it put it?
I would like to understand the answers to these questions. Can you
answer them (will you)? If not, can you please point me at something
that documents what's going on here?
Thanks.
Paul
Jess Austin
> Nope, that was me alright. I don't want a function to have access to
> it's metadata. Nothing's changed about that. I want to be able to
> specify a function's metadata inside the function def, where it seems to
> most appropriately belong.
So you have:
def foo(x):
__meta_variable__ = 42
variable = 5
return dir()
And you're volunteering to explain to a newbie, or even a nearly
intermediate programmer like myself, why the list that is returned
from this function contains 'x' and 'variable' but not
'__meta_variable__'? I'm sorry, but this is unreadable and is a
dramatic break from Python convention.
A variable binding inside a block is available in that block and in
contained blocks (method blocks aren't properly understood as being
contained within the class block), after the statement setting the
variable has executed. A variable defined inside a block is not
available outside it without indexing through another binding created
by an import or class statement. This makes Python code easy to read.
You propose to turn this on its head for a particular class of
variables defined within function blocks. No amount of reasoning or
argumentation will justify such a maneuver.
> Others seem to want to specify a function's metadata outside of the
> function def, which just doesn't seem pythonic (IMO).
If you really understood Python's scoping rules, you would know that
this opinion is the opposite of the truth. Of all the current
incarnations of function "metadata", only one may be set within the
function, and that one is not set using a standard binding statement:
>>> def bar():
... """docstrings are the exception not the rule"""
... __meta_doc__ = """Please don't assume otherwise"""
...
>>> dir(bar)
['__call__', '__class__', '__delattr__', '__dict__', '__doc__',
'__get__', '__getattribute__', '__hash__', '__init__', '__module__',
'__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__', '__str__', 'func_closure', 'func_code',
'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']
In the returned list, only '__doc__' was affected by characters in the
function suite, and those characters did not comprise a binding
statement within the function block. So in this respect '__doc__' is
not an exception to the rules about name binding in function blocks
either.
I think you're getting confused via a false analogy between function
blocks and class blocks. It's true that bindings set within a class
block are accessible through class or instance objects. This stateful
binding access is what is fundamental about classes, while
operator-associated method names are sugar, or "magic" if you prefer.
Function block binding statements are instructions for function
_execution_, class block binding statements are instructions for class
_creation_, and there's the difference. Any meta-meaning that the
actual name used in a binding statement might have is purely
incidental, in that the function or class is created in the same
fashion whether a given name has a magic meaning or not. Certain
multiple-underscore names do change the behavior of a class, but only
one relatively hackish name (__metaclass__) changes how it is created.
Class statements have to deal with creation because they have state,
thus they must have an initial state. Functions do not have state,
although the objects we use to access them (function or generator
objects) from the _outside_ do. It seems hard to support pushing
bindings with persistent state into the function block when it has
never had this before.
I appreciate your persistence in error; it has provided me with the
opportunity to gain knowledge in an area about which I previously had
only intuition. b-) In broader commentary, as I've said, pep318
isn't really about metadata. One reason is that metadata isn't that
important. Data only needed by an outside system should be kept
there. If any layer of code uses particular data, that layer should
be responsible for maintaining such. Python already accomodates this
more correct style of information management, and pep318 might
increase this accomodation.
yours,
Jess
Paul Morrow
> Paul Morrow <pm_...@yahoo.com> wrote in message news:<mailman.2478.1093564...@python.org>...
>
>>Nope, that was me alright. I don't want a function to have access to
>>it's metadata. Nothing's changed about that. I want to be able to
>>specify a function's metadata inside the function def, where it seems to
>>most appropriately belong.
>
>
> So you have:
>
> def foo(x):
> __meta_variable__ = 42
> variable = 5
> return dir()
>
> And you're volunteering to explain to a newbie, or even a nearly
> intermediate programmer like myself, why the list that is returned
> from this function contains 'x' and 'variable' but not
> '__meta_variable__'?
Sure. Magic attributes defined at the top level of a function (e.g.
your __meta_variable__) aren't intended to be local variables. You'll
see that as you start looking at more experienced programmer's code.
Therfore, they shouldn't be treated as local variables. For example,
suppose we changed your function a little...
def foo(x):
__author__ = 'Jess Austin'
__version__ = '1.0'
variable = 5
return dir()
See the difference?
>
>>Others seem to want to specify a function's metadata outside of the
>>function def, which just doesn't seem pythonic (IMO).
>
>
> If you really understood Python's scoping rules, you would know that
> this opinion is the opposite of the truth.
A decorator is a kind of meta information. So are docstrings. The
proposals I've been seeing (A1 and J2 in particular) declare decorators
and docstrings outside of the function def (not inside of the function def).
> Of all the current
> incarnations of function "metadata", only one may be set within the
> function, and that one is not set using a standard binding statement:
>
True and True, provided that you believe that docstrings are the only
kind of function metadata. I believe though that attributes like
__author__ and __version__ above are also function metadata.
> In broader commentary, as I've said, pep318
> isn't really about metadata. One reason is that metadata isn't that
> important.
By this you mean "in your opinion", right?
Paul
Bengt Richter
It looks to me like __doc__ is a descriptor (or c-code that acts like one)
(i.e., like a property) of the function class:
>>> def foo():
... 'foo doc string'
...
Find __doc__ in the class dict:
>>> type(foo).__dict__['__doc__']
<member '__doc__' of 'function' objects>
Check on its nature a little:
>>> type(type(foo).__dict__['__doc__'])
<type 'member_descriptor'>
It should have a __get__ method
>>> type(foo).__dict__['__doc__'].__get__
<method-wrapper object at 0x00901410>
Call that with foo as object instance:
>>> type(foo).__dict__['__doc__'].__get__(foo,None)
'foo doc string'
Try writing the __doc__ property (check for __set__ first):
>>> type(foo).__dict__['__doc__'].__set__
<method-wrapper object at 0x00901490>
Do long version of foo.__doc__ = 'new docstring'
>>> type(foo).__dict__['__doc__'].__set__(foo, 'new docstring') # foo.__doc__ = ...
Look at it using the short spelling:
>>> foo.__doc__
'new docstring'
And the long:
>>> type(foo).__dict__['__doc__'].__get__(foo,None)
'new docstring'
foo.func_doc seems to be implemented in a similar way:
>>> type(foo).__dict__['func_doc'].__get__(foo,None)
'new docstring'
>>> foo.func_doc
'new docstring'
Even the dict of a function instance appears to be implemented as a property:
>>> type(type(foo).__dict__['__dict__'])
<type 'getset_descriptor'>
>>> type(foo).__dict__['__dict__'].__get__(foo,None)
{}
>>> foo.x=123
>>> type(foo).__dict__['__dict__'].__get__(foo,None)
{'x': 123}
Short spelling:
>>> foo.__dict__
{'x': 123}
I haven't been into the implementation code, but this interpretation
of surface appearances seems to fit.
Regards,
Bengt Richter
Paul Morrow
Cool! Thanks! Now I need to ponder why a function's docstring needs a
different implementation than a class's docstring (i.e. why not just
make it a straigtforward attribute of the function object).
Jess Austin
> Jess Austin wrote:
>
> > So you have:
> >
> > def foo(x):
> > __meta_variable__ = 42
> > variable = 5
> > return dir()
> >
> > And you're volunteering to explain to a newbie, or even a nearly
> > intermediate programmer like myself, why the list that is returned
> > from this function contains 'x' and 'variable' but not
> > '__meta_variable__'?
>
> Sure. Magic attributes defined at the top level of a function (e.g.
> your __meta_variable__) aren't intended to be local variables. You'll
> see that as you start looking at more experienced programmer's code.
> Therfore, they shouldn't be treated as local variables. For example,
> suppose we changed your function a little...
>
> def foo(x):
> __author__ = 'Jess Austin'
> __version__ = '1.0'
> variable = 5
> return dir()
>
> See the difference?
I see that some of these variables have underscores and some don't.
Why didn't you respond to my detailed explanation of how binding and
scoping work in Python? Would it be worth my time to restate the
explanation in simpler terms?
> >>Others seem to want to specify a function's metadata outside of the
> >>function def, which just doesn't seem pythonic (IMO).
> >
> >
> > If you really understood Python's scoping rules, you would know that
> > this opinion is the opposite of the truth.
>
> A decorator is a kind of meta information. So are docstrings. The
> proposals I've been seeing (A1 and J2 in particular) declare decorators
> and docstrings outside of the function def (not inside of the function def).
I'll be more specific. "Others seem to want to specify a function's
metadata outside of the function def": truth. "which just doesn't
seem pythonic": opposite of truth. "(IMO)": apparently truth.
Pythonic is a word with a meaning (in this newsgroup anyway), and
modifying an object's namespace from outside _is_ pythonic: we've
always been able to do so (at least since mid-1999 when I started
using python) and we always will be able to do so. Your proposed
arbitrary character-driven change to Python's conventional scoping
rules would _not_ be pythonic. It may in fact be your opinion that
such a change would be pythonic, but I correctly classified that
opinion as the opposite of truth.
> > Of all the current
> > incarnations of function "metadata", only one may be set within the
> > function, and that one is not set using a standard binding statement:
> >
>
> True and True, provided that you believe that docstrings are the only
> kind of function metadata. I believe though that attributes like
> __author__ and __version__ above are also function metadata.
Metadata is a programmer's convention, or perhaps in this case
religious belief. I'm not certain what your precise convention is
since you haven't explained it. You evidently expect all
"experienced" programmers to know it and respect it. Apparently it
has something to do with underscore characters. Python accomodates
this particular convention, but it will never enforce it on the rest
of us, and that is a GOOD THING.
> > In broader commentary, as I've said, pep318
> > isn't really about metadata. One reason is that metadata isn't that
> > important.
>
> By this you mean "in your opinion", right?
Of course. The interesting thing about my opinions is that I've
provided reasonable prima facie justification for them. Thoughtful
interlocutors would respond to that justification.
yours,
Jess
Paul Morrow
> Paul Morrow <pm_...@yahoo.com> wrote in message news:<mailman.2550.1093655...@python.org>...
>
>>Jess Austin wrote:
>>
> Why didn't you respond to my detailed explanation of how binding and
> scoping work in Python? Would it be worth my time to restate the
> explanation in simpler terms?
>
>
I was going to, but couldn't really find one main idea in there to
respond to. It was a long paragraph full of lots and lots of complex
ideas, where many sentences had multiple interpretations. It seemed as
if, for almost every statement you made in that section, I would either
1) beg to differ, 2) reply that you were being imprecise in your
assertions, or 3) just not understand what you were talking about and
ask for you to explain what you meant.
If you would clearly restate the point you're trying to make, I'll
gladly try to respond to it.
>
>>>>Others seem to want to specify a function's metadata outside of the
>>>>function def, which just doesn't seem pythonic (IMO).
>>>
>>>
>>>If you really understood Python's scoping rules, you would know that
>>>this opinion is the opposite of the truth.
>>
>>A decorator is a kind of meta information. So are docstrings. The
>>proposals I've been seeing (A1 and J2 in particular) declare decorators
>>and docstrings outside of the function def (not inside of the function def).
>
>
> I'll be more specific. "Others seem to want to specify a function's
> metadata outside of the function def": truth. "which just doesn't
> seem pythonic": opposite of truth. "(IMO)": apparently truth.
> Pythonic is a word with a meaning (in this newsgroup anyway), and
> modifying an object's namespace from outside _is_ pythonic:
Really? Can you show me an example of this?
> we've
> always been able to do so (at least since mid-1999 when I started
> using python) and we always will be able to do so.
Just because we *can* do it doesn't make it Pythonic. To be Pythonic,
it must be beautiful (IMO). We can do lot's of things to an object from
outside of it's definition that are not Pythonic (IMO). Directly
updating it's __dict__ would be one example (IMO).
> Your proposed
> arbitrary character-driven change to Python's conventional scoping
> rules would _not_ be pythonic.
In your opinion.
> It may in fact be your opinion that
> such a change would be pythonic, but I correctly classified that
> opinion as the opposite of truth.
>
I don't know what classifying an "opinion as the opposite of truth"
means. Opinions are subjective assessments.
>
>
>>>Of all the current
>>>incarnations of function "metadata", only one may be set within the
>>>function, and that one is not set using a standard binding statement:
>>>
>>
>>True and True, provided that you believe that docstrings are the only
>>kind of function metadata. I believe though that attributes like
>>__author__ and __version__ above are also function metadata.
>
>
> Metadata is a programmer's convention, or perhaps in this case
> religious belief.
What? Metadata is a convention? Metadata is a belief?
> I'm not certain what your precise convention is
> since you haven't explained it.
I'm not suggesting a new coding convention. I'm suggesting that we
continue to us magic (__xxx__) attributes for all magic behavior.
> You evidently expect all
> "experienced" programmers to know it and respect it. Apparently it
> has something to do with underscore characters.
Yes I do. Yes it does.
Paul
Alex Martelli
...
> Cool! Thanks! Now I need to ponder why a function's docstring needs a
> different implementation than a class's docstring (i.e. why not just
> make it a straigtforward attribute of the function object).
Historically, functions had docstrings before they had straightforward
attributes. Maybe it could be that.
Alex
Bengt Richter
[...]
>
>Cool! Thanks! Now I need to ponder why a function's docstring needs a
>different implementation than a class's docstring (i.e. why not just
>make it a straigtforward attribute of the function object).
>
so don't take this as developer's documention of internals. I'm just reading
between the lines, trying to recognize the underlying ideas ;-)
So, depending on your idea of 'straightforward' ISTM docstrings do look like
'straightforward' attributes in the way they behave:
>>> def func():
... 'func docstring'
...
>>> class CNew(object):
... 'CNew docstring'
...
>>> class COld:
... 'COld docstring'
...
>>> func.__doc__, CNew.__doc__, COld.__doc__
('func docstring', 'CNew docstring', 'COld docstring')
But you were wondering about different _implementation_ of
class and function docstrings. But implementation is a layered
thing that evolves, even while surface behavior stays the same.
So your question comes down to how implementation differs for
getting the three __doc__ attributes in the last tuple above.
As far as the language is concerned, writing func.__doc__ is
no different than x.y -- we don't know what func is bound to,
and we don't know whether __doc__ is an object attribute or
a method in the object's class or a method somewhere in the
chain of base classes, or whether func is a class and __doc__
is a class variable, etc. So we have to look into how and when
those differences are discovered and have their effect.
<aside>
Implementation is layered. High level functionality is composed
of primitives. A really nice thing about Python is that so much of
the dynamic primitive infrastructure is accessible and expressible
in python source. Much work has obviously gone into making primitives
that are actually implemented in C behave and be inspectable "as-if"
they were implemented in python. IOW, there is a kind of primitive
abstractions layer that is platform independent and source-representable.
But whatever the representation, it is (IMO) the world of abstract ideas
that is the most important, the world where sets and numbers and mappings
and maybe deities live. That's where we find the real beauty that our
graven source code sometimes distracts from when we forget what's what ;-)
C implementation can make optimized shortcuts to define as-if
behaviour that looks identical to general behavior with various
limitations that when encountered act as-if they were exceptions
of general behaviour.
</aside>
Ok, focusing on x.y, what do we need to find out?
It looks like the __getattribute__ method (or its C behavioural equivalent)
is the key to getting x's y attribute. So where do we find __getattribute__?
Well, it's a method, and methods would be found in x's class dict, and if not there,
it would be looked for in the chain of base classes. But how do we find x's class dict?
x.__class__.__dict__ doesn't work if x happens to be an old style class.
So type(x) serves now as the logical equivalent of x.__class__ :
>>> for t in map(type, [func, CNew, CNew(), COld, COld()]): print t
...
<type 'function'>
<type 'type'>
<class '__main__.CNew'>
<type 'classobj'>
<type 'instance'>
>>> for t in map(lambda o: type(type(o).__dict__), [func, CNew, CNew(), COld, COld()]): print t
...
<type 'dictproxy'>
<type 'dictproxy'>
<type 'dictproxy'>
<type 'dictproxy'>
<type 'dictproxy'>
In every case we have a dict proxy, that we can now look for attributes in,
so we can look for __getattribute__:
>>> for t in map(lambda o:type(o).__dict__.get('__getattribute__'),[func,CNew,CNew(),COld,COld()]): print t
...
<slot wrapper '__getattribute__' of 'function' objects>
<slot wrapper '__getattribute__' of 'type' objects>
None
<slot wrapper '__getattribute__' of 'classobj' objects>
<slot wrapper '__getattribute__' of 'instance' objects>
Kind of interesting. Now if we let the the attribute chase continue for CNew vs CNew():
>>> type(CNew).__getattribute__
<slot wrapper '__getattribute__' of 'type' objects>
>>> type(CNew()).__getattribute__
<slot wrapper '__getattribute__' of 'object' objects>
IOW, it looks like the None got replaced by object.__getattribute__
found in the base class of class CNew(object) for CNew() (since there
was no override defined in CNew) but for CNew itself, the search started
in type(CNew), so found type.__getattribute__
If we look in a thing's __dict__ we are apparently avoiding the attribute chase,
but if we look via thing.__dict__ we are using the __dict__ attribute to get
the __dict__, so to allow us to see what __dict__ is without attribute processing,
we have to do thing.__dict__['__dict__'] or thing.__dict__.get('__dict__').
We can then look at what kind of thing that is:
>>> for ob in [func,CNew,CNew(),COld,COld()]:
... print type(ob).__dict__.get('__dict__')
...
<attribute '__dict__' of 'function' objects>
<attribute '__dict__' of 'type' objects>
<attribute '__dict__' of 'CNew' objects>
None
None
What kinds of things are these?
>>> for ob in [func,CNew,CNew(),COld,COld()]:
... print type(type(ob).__dict__.get('__dict__'))
...
<type 'getset_descriptor'>
<type 'getset_descriptor'>
<type 'getset_descriptor'>
<type 'NoneType'>
<type 'NoneType'>
Looks to me like a mechanism to unify access to attributes at a particular
infrastructure level, being used in this case to deal with a bunch of things
that can act like dictionaries, but which might have very different implementations
and limitations. E.g., IWT you could make dictionaries lazily implemented for objects
whose .__dict__'s are almost always empty, and just create them on first need.
Even if they start with standard name content, one could imagine creating a full dict
only on update. But those are optimization games.
Bottom line, comparing implementation of __doc__ in functions and new style classes
to answer your question, they don't seem that different. I.e., both seem to be
implemented via descriptor/property-like things:
>>> type(func).__dict__['__doc__']
<member '__doc__' of 'function' objects>
>>> type(func).__dict__['__doc__'].__get__
<method-wrapper object at 0x00918890>
>>> type(func).__dict__['__doc__'].__get__(func)
'func docstring'
>>> type(CNew).__dict__['__doc__']
<attribute '__doc__' of 'type' objects>
>>> type(CNew).__dict__['__doc__'].__get__
<method-wrapper object at 0x00918890>
>>> type(CNew).__dict__['__doc__'].__get__(CNew)
'CNew docstring'
Note that old style classes and objects don't use a __doc__ descriptor:
>>> type(COld)
<type 'classobj'>
>>> type(type(COld).__dict__['__doc__'])
<type 'str'>
I.e., the above is the actual doc string of classobj, not a descriptor
>>> type(COld)
<type 'classobj'>
>>> type(COld).__dict__['__doc__']
'classobj(name, bases, dict)\n\nCreate a class object. The name must be a string; the second ar
gument\na tuple of classes, and the third a dictionary.'
(Of course, normally it is not retrieved this way).
vs.
>>> type(CNew)
<type 'type'>
>>> type(CNew).__dict__['__doc__']
<attribute '__doc__' of 'type' objects>
>>> type(type(CNew).__dict__['__doc__'])
<type 'getset_descriptor'>
So there is a difference between func and old-style-class docstring implementations.
Were you using old style classes for your question?
I don't know why I did this...
Regards,
Bengt Richter
Paul Morrow
> On Sat, 28 Aug 2004 08:45:20 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
> [...]
>
>>Cool! Thanks! Now I need to ponder why a function's docstring needs a
>>different implementation than a class's docstring (i.e. why not just
>>make it a straigtforward attribute of the function object).
>>
>
> Disclaimer: I am just speculating from appearances accessible interactively,
> so don't take this as developer's documention of internals. I'm just reading
> between the lines, trying to recognize the underlying ideas ;-)
>
> So, depending on your idea of 'straightforward' ISTM docstrings do look like
> 'straightforward' attributes in the way they behave:
>
By straightforward here I mean stored directly in the object's __dict__.
It's such an obvious place to keep the docstring, and it's the
expected place, since that's where it's (apparently) stored for classes
and modules.
> So there is a difference between func and old-style-class docstring implementations.
> Were you using old style classes for your question?
>
I'm hoping that it doesn't matter. What I'm really trying to get at is
a function's namespace --- a dictionary that maps function attribute
names to objects --- so that the interpreter has a context in which to
evaluate __xxx__ variables defined at the outermost level of a function
def.
Remember, I'm asserting that when we define __xxx__ attributes inside of
a function def, they are not intended to be local variables, used in the
computation of the function's result. Rather they are intended to be
function meta data, and therefore should be evaluated (at function
definition time, not function execution time), in the context of the
function being defined.
For example, in the following function def, the comments specify (what I
believe is) the author's intention behind each assignment.
def circu(diameter):
"""Info about circu.""" # defines circu.__doc__
__author__ = 'Paul Morrow' # defines circu.__author__
__features__ = memoized # defines circu.__features__
pi = 3.14 # local variable definition
return pi * diameter
> I don't know why I did this...
>
You were trying to help us understand what's going on under the hood,
and I really appreciate that research and explanation. Thank you for
taking such an interest in this issue.
Paul
Paul Morrow
/*
I have stared at your reply for three or four minutes now. On the
surface, it seems that you are being genuinely helpful, and so I want to
reply "Thanks. Of course. That makes perfect sense."
But then a lot of smart people say things that (on the surface) appear
honestly helpful but in reality are suggestions that the reader do more
research before asking questions that have such an obvious answer.
I know that you're a smart person Mr. Martelli. So I flipped a coin to
decide which intreprtation of your reply to use, which came up tails,
indicating the 2nd one.
Bahh. I prefer the superficial one (plz correct me if I'm wrong).
*/
Thanks. That does make good sense. They've had no reason to simplify
the implementation for function docstrings, so "why fix it if it ain't
broke."
Paul
Anthony Baxter
> For example, in the following function def, the comments specify (what I
> believe is) the author's intention behind each assignment.
>
> def circu(diameter):
> """Info about circu.""" # defines circu.__doc__
> __author__ = 'Paul Morrow' # defines circu.__author__
> __features__ = memoized # defines circu.__features__
> pi = 3.14 # local variable definition
> return pi * diameter
Not in any Python I know of. I look at that and I think "oo, local variables".
That you have a different belief doesn't make it right - I remain
_extremely_ unconvinced.
Anthony
Paul Morrow
My belief doesn't make it right, that's true. But I think that a
preponderance of supporting evidence does.
Have you seen a significant number of cases where a function uses
__xxx__ attributes as local variables? Have you seen any?
Can you craft a function definition that uses __xxx__ attributes as
local variables, where the majority of experienced Pythonistas would
agree that the double underscores are warranted?
Paul
Paul Morrow
Ugh... Sorry, I wrote that too hastily. Let me restate.
Have you seen a significant number of cases where a function uses
__xxx__ names for local variables? Have you seen any?
Can you craft a function definition that uses __xxx__ names for
Steven Bethard
> Have you seen a significant number of cases where a function uses
> __xxx__ names for local variables? Have you seen any?
Are there any examples currently of functions that use __xxx__ names in their
body at all? I haven't seen this, but I haven't really looked...
STeve
David Eppstein
Paul Morrow <pm_...@yahoo.com> wrote:
> Anthony Baxter wrote:
> > On Sun, 29 Aug 2004 12:10:13 -0400, Paul Morrow <pm_...@yahoo.com> wrote:
> >
> >>For example, in the following function def, the comments specify (what I
> >>believe is) the author's intention behind each assignment.
> >>
> >> def circu(diameter):
> >> """Info about circu.""" # defines circu.__doc__
> >> __author__ = 'Paul Morrow' # defines circu.__author__
> >> __features__ = memoized # defines circu.__features__
> >> pi = 3.14 # local variable definition
> >> return pi * diameter
> >
> >
> > Not in any Python I know of. I look at that and I think "oo, local
> > variables".
> >
> > That you have a different belief doesn't make it right - I remain
> > _extremely_ unconvinced.
> >
> > Anthony
>
> My belief doesn't make it right, that's true. But I think that a
> preponderance of supporting evidence does.
>
> Have you seen a significant number of cases where a function uses
> __xxx__ attributes as local variables? Have you seen any?
So we really need to return to the FORTRAN days where variables have
different semantics depending on their first character?
Anyway, regardless of whether it's a local variable or an attribute, if
it's in the function body I expect it to be executed at call time not
def time.
--
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/
Paul Morrow
>
>
> Are there any examples currently of functions that use __xxx__ names in their
> body at all? I haven't seen this, but I haven't really looked...
>
> STeve
>
I've seen some that define a few of the pydoc variables (__author__,
__version__, etc), although pydoc doesn't recognizes them there. I
remember reading a whitepaper on a web templating framework that made
use of them too, where each function corresponded to an html tag and the
__xxx__ variables controlled caching (memoizing), which database tables
were involved, etc.
For classes and modules of course you'll see more examples of __xxx__
variables describing meta aspects of the class/module (__metaclass__,
__future__, etc.), so it seems consistent that they could describe meta
aspects of functions too.
Paul Morrow
>
> So we really need to return to the FORTRAN days where variables have
> different semantics depending on their first character?
A little of that is a good thing (IMO). The alternative is that we
explicitly declare each variable's semantics, which makes for a much
more wordy language.
> Anyway, regardless of whether it's a local variable or an attribute, if
> it's in the function body I expect it to be executed at call time not
> def time.
>
I believe that we should think of assignments to __xxx__ attributes as
not being part of the function's body, but instead part of its
declaration, just as we do with its docstring.
def circum(diameter):
"""This describe's foo."""
__author__ = 'Paul Morrow'
__version__ = '0.1'
__features__ = synchronized, memoized
# Everything above here is declaration.
# Everything below here is in the body.
pi = 3.14
return pi * diameter
Paul
Paul Rubin
> I believe that we should think of assignments to __xxx__ attributes as
> not being part of the function's body, but instead part of its
> declaration, just as we do with its docstring.
>
> def circum(diameter):
> """This describe's foo."""
> __author__ = 'Paul Morrow'
> __version__ = '0.1'
That's fundamentally a pretty good idea, but conflicts with how we
currently use __xxx__ attributes:
def fcn(self, x, y, r):
...
class circle(shape):
__init__ = fcn
should work about the same way as
class circle(shape):
def __init__(self, x, y, r):
... # same function as fcn above
Paul Morrow
I'm not seeing the conflict. Would you please say a few more words
about that?
Just so that I'm clear on what I'm saying, I believe that all assignment
to __xxx__ variables (including assignments to __init__) should be
executed at object 'defininition time' (i.e. when an object is defined),
within the context (namespace) of the object being defined.
So in your example [*], the '__init__ = fcn' statement would be executed
when the interpreter processes the definition of your circle class (when
it executes the class statement). When it is finished, circle.__init__
would be bound to your fcn function.
Paul
[*] If I'm reading it right --- in my newsreader, the class statement
appears inside of the def fcn, but I'm assuming that wasn't your intention.
Jeff Epler
def f():
__var__ = 3
return __var__
f.__var__ += 1
print f()
What about any of the following:
def g():
if True:
__var__ = 4
print g.__var__
x = 3
def h(x):
__var__ = x*x
return x*x
print h(2), h.__var__
def j(cls):
def __radd__(self, other):
return self.__add__(other)
cls.__radd__ = __radd__
def k(x):
__created__ = Date(1, 1, 2004)
__expires__ = __created__ + Interval("1 year")
def l(x):
__now__ = time.time()
return __now__
print l.__now__, l()
x = 3
def m():
__x__ = x
return __x__
x = 4
print m(), m.__x__
def n(f, a):
__doc__ = gettext.gettext(f.__doc__)
__author__ = a.replace("@", " AT ").replace(".", " DOT ")
l = locals()
del l['f'], l['d'], l['a'], l['l']
f.__dict__.__update__(l)
def p():
import os as __os__
def q(): # BUG x doesn't get the proper metaclass in 2.3!
class __metaclass__(type): pass
class x: pass
# assert x's metaclass is __metaclass__
I gave it a few minutes thought, and I can't figure out a simple rule
that would *not* break code that happened to use __name__s as function
locals (since this has been allowed in all the ten years I've written
Python programs), would not add a new level of name lookup (lookup of
unqualified name on function object), would not introduce confusion
about when lines of code *in the function body* would be evaluated,
would not require that assignment statements be treated differently from
"def", "class", and "__import__" statements, etc, etc.
If you clear some or all of these things up, and turn it into a complete
proposal, wake me up.
Jeff
Anthony Baxter
> > Have you seen a significant number of cases where a function uses
Locals, no. But builtins, yes. For instance, __import__.
So your proposal suddently has to deal with looking up _only_ locals,
not globals or builtins.
I have asked this indirectly, and Jeff Epler's asked it more
obviously, so, to restate it:
Have you actually got a concrete, written down proposal? Have you
worked through how it might work, and what the impact on the language
might be? I was hoping that you'd sit down and do what Bengt Richter
did and posted to the newsgroup - worked out how this stuff works,
under the hood, so that you weren't just hand-waving. Oh well.
Alex Martelli
> Alex Martelli wrote:
> > Paul Morrow <pm_...@yahoo.com> wrote:
> > ...
> >
> >>Cool! Thanks! Now I need to ponder why a function's docstring needs a
> >>different implementation than a class's docstring (i.e. why not just
> >>make it a straigtforward attribute of the function object).
> >
> >
> > Historically, functions had docstrings before they had straightforward
> > attributes. Maybe it could be that.
> >
> >
> > Alex
>
> /*
> I have stared at your reply for three or four minutes now. On the
> surface, it seems that you are being genuinely helpful, and so I want to
> reply "Thanks. Of course. That makes perfect sense."
Heh, I think you've wasted a few minutes here.
>
> But then a lot of smart people say things that (on the surface) appear
> honestly helpful but in reality are suggestions that the reader do more
> research before asking questions that have such an obvious answer.
Doing your own research is good, of course. But once you have, there's
no harm checking your conclusions on mailing lists. Functions had a few
_non_-straightforward attributes (such as func_doc AKA __doc__ -- and I
don't recall which one came first) for a long time before they got
straightforward (==implemented with a dict) ones, after all.
>
> I know that you're a smart person Mr. Martelli. So I flipped a coin to
> decide which intreprtation of your reply to use, which came up tails,
> indicating the 2nd one.
>
> Bahh. I prefer the superficial one (plz correct me if I'm wrong).
> */
>
> Thanks. That does make good sense. They've had no reason to simplify
> the implementation for function docstrings, so "why fix it if it ain't
> broke."
Many object types have 'slots' ("non straightforward attributes") as
well as possibly a per-instance dict in which they may keep other
(straightforward) attributes. I see no reason why moving attributes
from dedicated slots to a dictionary should be a simplification. Take
attribute __name__: that's not in the dict for either functions or
classes (in functions, it's also known as func_name). Why not?
Alex
Paul Morrow
Where would I put such a proposal? Post it here? And would it need to
include an implementation? I'm new to the bowels of the Python system,
plus I've got a big project due this time next week, so it would be next
week before I could really dig into something like that.
I guess I'm just a hand-waiver for now.
Paul
Hallvard B Furuseth
> >>> def baz():
> ... """docstring of baz"""
> ...
> >>> baz.__doc__
> 'docstring of baz'
> >>> baz.__dict__.keys()
> []
>
> Say what? Why didn't the Python system put baz's docstring into it's
> namespace (__dict__)? And where did it put it?
I'm fairly new to Python, but I think you are just using the wrong
interface to access the namespace. __dict__ isn't necessarily
something's namespace, even if it has a __dict__:
>>> class C(object): __slots__ = 'hide'
...
>>> class D(C): pass
...
>>> o = D()
>>> o.show, o.hide = 3, 5
>>> o.__dict__
{'show': 3}
OTOH,
>>> dir(o)
['__class__', '__delattr__', '__dict__', ... 'hide', 'show']
>>> getattr(o, 'hide')
5
>>> def foo(): """docstring"""
...
>>> dir(foo)
['__call__', '__class__', '__delattr__', '__dict__', '__doc__', ...]
>>> setattr(foo, '__doc__', "nope")
>>> help(foo)
Help on function foo in module __main__:
foo()
nope
--
Hallvard
Hallvard B Furuseth
> I believe that we should think of assignments to __xxx__ attributes as
> not being part of the function's body, but instead part of its
> declaration, just as we do with its docstring.
>
> def circum(diameter):
> """This describe's foo."""
> __author__ = 'Paul Morrow'
> __version__ = '0.1'
> __features__ = synchronized, memoized
I really, really don't like this idea. Assignment in Python has quite
enough pitfalls already. Please don't make it worse. If it looks like
an assignment to a local variable, it should be an assignment to a local
variable. If we are going to invent a syntax for declaring function
attributes inside the function, why confuse the issue by making it look
like it does something else?
It's true that it makes it look more like what __*__ attribute
assignment in class bodies does, but the simple fact is that class
bodies are executed when the class statement is executed, and function
bodies are not executed when the def statement is executed. Now you
want part of the function bodies to be executed at def time, and part at
call time.
__*__ assigments in class bodies can look a little confusing if you
don't think of how class declarations work, but it's perfectly simple
once you think of how they do work: Execute the class body - no magic
involved (as far as I know). Then pass the resulting __dict__ to the
class creation machine - and _that_ recognizes __metaclass__, __slots__
and I don't know what else and does magic with them.
Once you remember that, __*__ variables in both class bodies and
function bodies do exactly what one would expect them to do. Stronger
magic, like grabbing the doc string and putting it in __doc__, has its
own syntax.
--
Hallvard
Paul Morrow
> Paul Morrow wrote:
>
>>I believe that we should think of assignments to __xxx__ attributes as
>>not being part of the function's body, but instead part of its
>>declaration, just as we do with its docstring.
>>
>> def circum(diameter):
>> """This describe's foo."""
>> __author__ = 'Paul Morrow'
>> __version__ = '0.1'
>> __features__ = synchronized, memoized
>
>
> I really, really don't like this idea. Assignment in Python has quite
> enough pitfalls already. Please don't make it worse. If it looks like
> an assignment to a local variable, it should be an assignment to a local
> variable.
Actually, these look like assignments to local *magic* variables.
There's the difference. The double underscores before and after each
name loudly proclaims that these variables are not like 'normal'
variables. They're special in some way. That's a Python convention.
> If we are going to invent a syntax for declaring function
> attributes inside the function, why confuse the issue by making it look
> like it does something else?
>
Because 1) adding new syntax should be resisted as much as possible
(IMO), and 2) this style has parallels in both class definitions and
module definitions. Compare this class statement with the function def
above.
class Oval(Circle):
"""This describe's Oval."""
__author__ = 'Paul Morrow'
__version__ = '0.1'
__metaclass__ = M
> It's true that it makes it look more like what __*__ attribute
> assignment in class bodies does, but the simple fact is that class
> bodies are executed when the class statement is executed, and function
> bodies are not executed when the def statement is executed. Now you
> want part of the function bodies to be executed at def time, and part at
> call time.
>
Maybe we just need to be clear as to which lines under the def statement
constitute the function's /body/. The function's docstring gets
assigned during processing of the def statement, therefore the docstring
is apparently not part of the function body; I think of it as part of
the function's declaration. So why not have assignments to __xxx__
variables also be part of the function's declaration. The distinctive
appearance of these names helps remind us that they have special semantics.
And note that if __xxx__ variables/attributes inside of function defs
*don't* have special semantics (i.e. if they are being used as normal
local variables), then they are improperly named (in accordance with our
naming conventions).
> __*__ assigments in class bodies can look a little confusing if you
> don't think of how class declarations work, but it's perfectly simple
> once you think of how they do work: Execute the class body - no magic
> involved (as far as I know). Then pass the resulting __dict__ to the
> class creation machine - and _that_ recognizes __metaclass__, __slots__
> and I don't know what else and does magic with them.
>
> Once you remember that, __*__ variables in both class bodies and
> function bodies do exactly what one would expect them to do. Stronger
> magic, like grabbing the doc string and putting it in __doc__, has its
> own syntax.
>
Perhaps its just my experience with declarative programming languages,
but I prefer thinking of __xxx__ assignments in a class as setting
'properties' of the class. It's a higher level of abstraction than
thinking in terms of the procedural mechanics of what happens under the
hood. And the syntax used to set properties in classes/modules should
be the same one used to set properties in functions and methods (IMO).
Paul
Steven Bethard
> There's the difference. The double underscores before and after each
> name loudly proclaims that these variables are not like 'normal'
> variables. They're special in some way. That's a Python convention.
Note that you're asking for __xxx__ variables to be assignments to the
function's attributes even though they're in the part of the function that
gets executed when the function is *called*. Let's look at what examples of
this behavior might look like for some different objects.
# in the part of a function that is executed when the function gets called
def f():
__author__ = 'Steve'
# in the part of a class that is executed when the class gets called
class S(object):
def __new__(cls):
__author__ = 'Steve'
# in the part of an object that is executed when the object gets called
class T(object):
def __call__(self):
__author__ = 'Steve'
Note that the parallel I'm trying to draw here is that in each of these cases,
the __author__ is assigned to in the part of the definition that gets executed
when the object is *called*. Thus f(), S(), and T()() all execute the code
block containing the __author__ assignment. Presumably you wouldn't want the
__author__ statements to assign to the class or the object in the second two
examples?
Of course, what you really want is for these things to be assigned to in the
part that gets executed when the object is *defined*. For a class, this is
clearly:
class S(object):
__author__ = 'Steve'
But what gets executed when a function is defined? Just the def statement,
the binding of the function name to its body. To be consistent with classes
then, your assignment to __author__ should be *in* the def statement. So what
you're suggesting, basically, is that the def statement should be extended to
include any __xxx__ assignments following the docstring.
Extending the def statement is not unreasonable, but IMHO it should be clear
that the extensions to the def statement are part of that statement. There's
nothing in current Python semantics that would suggest that any assignments in
a funcdef are part of the def statement. Note that even docstrings, which
could be reasonably argued to be part of the function definition, cannot be
included in the def statement by assignment:
>>> def f():
... __doc__ = "f"
...
>>> print f.__doc__
None
Certainly __xxx__ variables are special in some way. But are they special in
the way you want them to be? AFAICT, *assignment* to __xxx__ variables is
never what does the magic. The magic is done when the Python interpreter
reads the *values* of those variables. The variables continue to exist in
whatever namespace they were created in -- AFAICT, current Python __xxx__
variable assignments never cause __xxx__ variables to change namespace.
I seems like you want to introduce a substantially new semantics to an already
existing syntax. This creates ambiguity... Ambiguity bad. Bad, Ambiguitiy,
bad! No biscuit! ;)
Steve
Paul Morrow
In all of your examples, the __author__ assignments apply to the object
being defined. In the first example, that's the function f. In the
second example that's the method __new__, and the method __call__ in the
3rd example.
Note that this is true, /even in today's Python/. Your intention behind
each __author__ definition in your examples is *not* to create a local
variable. Your function/method (most likely) is not going to use
__author__ when it executes. Instead you are adding to the definition
of the function/method.
What I'm wondering is whether we can stop thinking of assignments to
magic attributes that immediately follow the docstring (if present) as
something that gets executed when the function is called. I think that
we can. I think that --- because of the names of these variables
(author, version, features) --- they will obviously and intuitively be
associated with the function's definition, not it's execution. That's a
point a lot of us have been forgetting here. These variables don't
sound like something the function would use in it's calculation. So
that, plus the fact that they are specially decorated (leading and
trailing underscores), plus the fact that they immediately follow the
docstring categorize them as part of the function's definition, not
execution.
But why speculate. Let's see if this is really going to be unclear in
practice. Ask someone who has never seen Python before which are the
local variables in the following def.
def circumference(diameter):
"""Calculate the diameter of a circle."""
__author__ = 'Paul Morrow'
__version__ = '0.1'
pi = 3.14
return pi * diameter
> Of course, what you really want is for these things to be assigned to in the
> part that gets executed when the object is *defined*. For a class, this is
> clearly:
>
> class S(object):
> __author__ = 'Steve'
>
> But what gets executed when a function is defined? Just the def statement,
> the binding of the function name to its body.
And the binding of the __doc__ variable, if a docstring is present.
> To be consistent with classes
> then, your assignment to __author__ should be *in* the def statement. So what
> you're suggesting, basically, is that the def statement should be extended to
> include any __xxx__ assignments following the docstring.
>
Yes.
> Extending the def statement is not unreasonable, but IMHO it should be clear
> that the extensions to the def statement are part of that statement. There's
> nothing in current Python semantics that would suggest that any assignments in
> a funcdef are part of the def statement. Note that even docstrings, which
> could be reasonably argued to be part of the function definition, cannot be
> included in the def statement by assignment:
>
>
>>>>def f():
>
> ... __doc__ = "f"
> ...
>
>>>>print f.__doc__
>
> None
>
> Certainly __xxx__ variables are special in some way. But are they special in
> the way you want them to be?
Not yet, /formally/. But /in practice/ this is how they are used (to
make meta statements about the thing being defined).
There's no need for a new syntax here. We don't need more words in our
language, or anything that makes our code less readable [*].
[*] Have you seen the decorator proposals?
> AFAICT, *assignment* to __xxx__ variables is
> never what does the magic. The magic is done when the Python interpreter
> reads the *values* of those variables. The variables continue to exist in
> whatever namespace they were created in -- AFAICT, current Python __xxx__
> variable assignments never cause __xxx__ variables to change namespace.
>
The assignments to __xxx__ variables (immediately following the
docstring, if present) would occur in the namespace of the object
(function/method) being defined. The assignments would not cause them
to change namespaces.
> I seems like you want to introduce a substantially new semantics to an already
> existing syntax. This creates ambiguity... Ambiguity bad. Bad, Ambiguitiy,
> bad! No biscuit! ;)
>
I maintain that the author's intention behind these assignments is what
we should focus on. When we define a __xxx__ variable inside of a
function def, we are (most likely) making a meta statement about the
function. We are not defining a local variable to be used later in the
execution of the function. So this proposal would not be changing the
meaning of the __xxx__ assignments in practice. It would merely be
formalizing them.
Paul
Paul Morrow
immediately following the docstring (if present) would cease to become
local variables. This of course has the possiblity of breaking existing
code. But I don't believe that developers routinely use such names for
local variables, so I don't believe that there will actually be much
broken code should this change be implemented.
Jeff Epler wrote:
> In your proposed model, what does the following code do?
> def f():
> __var__ = 3
> return __var__
> f.__var__ += 1
> print f()
>
Only assignments to __xxx__ variables at the very start of a function
def would have this special semantics. So your return statement would
be referencing a name (__var__) that doesn't exist in the function's
local variable namespace.
Easier to accept if we use a more likely variable name for the magic
variable.
def f():
__version__ = 3
return __version__
f.__version__ += 1
print f()
> What about any of the following:
> def g():
> if True:
> __var__ = 4
> print g.__var__
>
__var__ = 4 # creates a local variable because
# the assignment isn't at top
# of function def. But probably
# should generate a warning for
# giving a magic name to a local
# variable.
> x = 3
> def h(x):
> __var__ = x*x
> return x*x
> print h(2), h.__var__
>
In __var__ = x*x, x is not in the function's namespace (it's a local
variable), so x is undefined. Easier to visualize if we make it
__version__ = x*x
Same basic idea for your other examples.
>
> def p():
> import os as __os__
>
__os__ would be a local variable (since this is not a simple assignment,
i.e. using an equal sign). __os__ is not a good name for a local
variable, so interpretor should probably generate a warning.
> def q(): # BUG x doesn't get the proper metaclass in 2.3!
> class __metaclass__(type): pass
> class x: pass
> # assert x's metaclass is __metaclass__
>
This would do whatever it does now.
Paul
Paul Morrow
>
> But why speculate. Let's see if this is really going to be unclear in
> practice. Ask someone who has never seen Python before which are the
> local variables in the following def.
>
> def circumference(diameter):
> """Calculate the diameter of a circle."""
> __author__ = 'Paul Morrow'
> __version__ = '0.1'
> pi = 3.14
> return pi * diameter
>
>
And while you're at it, ask them which lines get executed at function
definition time, and which get executed when the function is called.
Paul
Steven Bethard
> The assignments to __xxx__ variables (immediately following the
> docstring, if present) would occur in the namespace of the object
> (function/method) being defined. The assignments would not cause them
> to change namespaces.
What you're suggesting is that given:
def f1():
__author__ = 'Steve'
and
def f2():
author = 'Steve'
in f1, the assignment to __author__ occurs in the function's namespace, but in
f2, the assignment to author occurs in the local namespace. Clearly then, the
__xxx__ format (if at the beginning of a function) changes the namespace to
which an assignment applies. How is this not causing assignments to change
namespaces?
Steve
Steven Bethard
> What you're suggesting is that given:
>
> def f1():
> __author__ = 'Steve'
>
> and
>
> def f2():
> author = 'Steve'
>
> in f1, the assignment to __author__ occurs in the function's namespace, but
> in f2, the assignment to author occurs in the local namespace. Clearly
> then, the __xxx__ format (if at the beginning of a function) changes the
> namespace to which an assignment applies.
Note that the parallel for classes would be if, given:
class F1:
__author__ = 'Steve'
and
class F2:
author = 'Steve'
that the __author__ assignment in F1 occured in one namespace, while the
author assignment in F2 occured in another namespace. (Something like
__author__ only being available from the class object, while author was only
available from class instance objects.) Python could almost certainly be
treat __xxx__ variables to work this way, but it's not the way it works
*now*. This is why I say that you're introducing a totally new semantics to
__xxx__ assignments.
Steve
Paul Morrow
> Paul Morrow <pm_mon <at> yahoo.com> writes:
>
>>The assignments to __xxx__ variables (immediately following the
>>docstring, if present) would occur in the namespace of the object
>>(function/method) being defined. The assignments would not cause them
>>to change namespaces.
>
>
>
>
> def f1():
> __author__ = 'Steve'
>
> and
>
> def f2():
> author = 'Steve'
>
> in f1, the assignment to __author__ occurs in the function's namespace, but in
> f2, the assignment to author occurs in the local namespace. Clearly then, the
> __xxx__ format (if at the beginning of a function) changes the namespace to
> namespaces?
>
def f3():
"""I am a docstring."""
__author__ = 'Steve'
y = 10
When the interpreter processes a function def, it places the name of the
function and its docstring into the namespace of the function being
defined. So instead of stopping at the docstring, it would look ahead
to see if there were any subsequent " __xxx__ = " statements, and if so,
would evaluate them in the same namespace [*]. After the def was
finished, the function's namespace would include not only its name and
docstring, but an attribute for each __xxx__ assignment as well.
Later, when the function is executed, its local, global, and built-in
namespaces would be exactly what they are now, except that its local
namespace would not include any __xxx__ assignments processed when the
def statement was executed.
So in f3 above, its local namespace would include 'y' only.
Paul
[*] The namespace of the function being defined; the same place it put
the function's name and docstring.
Paul Morrow
> I wrote:
>
>>What you're suggesting is that given:
>>
>>def f1():
>> __author__ = 'Steve'
>>
>>and
>>
>>def f2():
>> author = 'Steve'
>>
>>in f1, the assignment to __author__ occurs in the function's namespace, but
>>in f2, the assignment to author occurs in the local namespace. Clearly
>>then, the __xxx__ format (if at the beginning of a function) changes the
>>namespace to which an assignment applies.
>
>
> Note that the parallel for classes would be if, given:
>
> class F1:
> __author__ = 'Steve'
>
> and
>
> class F2:
> author = 'Steve'
>
> that the __author__ assignment in F1 occured in one namespace, while the
> author assignment in F2 occured in another namespace. (Something like
> __author__ only being available from the class object, while author was only
> available from class instance objects.) Python could almost certainly be
> treat __xxx__ variables to work this way, but it's not the way it works
> *now*. This is why I say that you're introducing a totally new semantics to
> __xxx__ assignments.
>
From the interpreter's or system programmer's point of view, you are
absolutely correct. There would be a profoundly different thing going
on under the covers. Specifically, __xxx__ assignments that appear at
the top of function defs would no longer result in local variables.
That is totally different than it is now, you're right.
But I believe that from the application programmer's point of view, not
much will change. That's because of the way (I believe) the majority of
us view __xxx__ assignments (inside of functions) now. When we say
def foo():
__author__ = 'Morrow'
__version__ = '0.1'
we don't care that __author__ and __version__ become local variables
because foo is most likely not going to use them in its calculation. So
we won't care if later they become something else (attributes of foo).
The semantics of assigning to __author__ or __version__ is the same, in
the eyes of the application programmer, whether the proposed changes are
implemented or not.
Paul
Bengt Richter
IMO you will have better luck promoting your functionality if you make
your proposed magic syntactically distinct using something other than underscores,
which are just name characters in any legal name that matches [a-zA-Z_][a-zA-Z0-9_]*
I think your magic becomes more interesting if we separate the two issues of
referring to attributes of the current object and doing it at def-time as opposed
to normal execution time (i.e., call-time for function body code), and get some
orthogonal general primitives going ;-)
To make things simple, let's say '$' means the current object, and let's say that
we use '..' in various ways to indicate execution or evaluation at def-time. E.g.,
prefixed to an assignment '=' it would mean do the binding (and evaluation of the
whole right hand side) at def-time. Thus your example becomes
def circumference(diameter):
"""Calculate the diameter of a circle."""
$.author ..= 'Paul Morrow'
$.version ..= '0.1'
pi = 3.14
return pi * diameter
But notice that the assignment of pi is a wasteful repeated action for every call.
If instead if we wrote (additionally illustrating constant folding effect):
pi ..= 3.14 # or __import__('math').pi for better accuracy ;-)
mask7 ..= 2**7-1 # net binding as if mask7=127
the effect would be a def-time pre-binding of pi and mask7, visible locally as if
def circumference(diameter, pi=3.14, mask7=2**7-1):
# etc
but without using the default arg hack. Of course, a default local arg name binding is
refreshed to the original rhs at every call, so even if you set pi=0.0 it will appear
as 3.14 next time. Unlike if you used $.pi ..= 3.14 and then set $.pi=0.0 -- that would
be preset once and rebindings would persist. $.xxx would also provide good opportunity
for optimizing access, and the $ attribute name space would be a handy substitute for
a mutable closure object named something other than '$'.
BTW, another interesting thing would be if the function class were modified so that it would
recognize descriptors when attributes of its instances. This would make assignments of
properties sticky unless they had a self-deleting method.
E.g.,
def foo(): $.prop ..= property(lambda self: 'hi from %s.prop'%self.__name__)
would mean foo.prop would return 'hi from foo.prop' and foo.prop = 123 would raise an
exception. Or the function class could be made subclassable.
BTW, an expression should also be able to have a def-time-evaluated term, without the rest
being def-time, e.g.,
def area(r): return ..(__import__('math').pi)*r*r
Note also what you could do with a mutable when you don't need the default arg hack:
def pow10(x):
cache ..={}
try: return cache[x]
except KeyError: return cache.setdefault(x, 10**x)
In classes, if there is a classvar ..= something, the resulting code
for the class definition should theoretically not have the assignment
as byte codes when the class body executes, but the binding should be
visible as-if. I don't know if there's an efficient way to do that.
Seems like it would have to be pre-bound in the class dict (that gets
passed to a metaclass if there is one), but I guess a fresh copy would
be needed. So that would wind up way more expensive than just a plain
class-def-time assignment. Hm, probably easiest just not to allow it.
Inside methods, of course, ..= would be fine. Method def-times are when
the class body executes (unless they are later added, in which case the
..= applies wherever the function/method was defined and its def executed.
Too many ideas in one post?
Ok, enough lunch fun.
Regards,
Bengt Richter
Hallvard B Furuseth
>Hallvard B Furuseth wrote:
>>Paul Morrow wrote:
>>>
>>>I believe that we should think of assignments to __xxx__ attributes as
>>>not being part of the function's body, but instead part of its
>>>declaration, just as we do with its docstring.
>>>
>>> def circum(diameter):
>>> """This describe's foo."""
>>> __author__ = 'Paul Morrow'
>>> __version__ = '0.1'
>>> __features__ = synchronized, memoized
>>
>>
>> I really, really don't like this idea. Assignment in Python has quite
>> enough pitfalls already. Please don't make it worse. If it looks like
>> an assignment to a local variable, it should be an assignment to a local
>> variable.
>
> Actually, these look like assignments to local *magic* variables.
> There's the difference.
No, that's not the difference. It's still different from how magic
variables behave in all other places. (Class bodies, module bodies,
object attributes). The difference is that magic variables in this
particular localtion is actually parsed differently from use of
variables other places, so the '=' operator (and what other operators?)
means something else.
> The double underscores before and after each name loudly proclaims
> that these variables are not like 'normal' variables. They're special
> in some way.
No other magic variables are that special. They all behave normally _as
variables_. Other parts of Python react to them being set or reset, the
very variable assigments are quite normal. Except for the name mangling
of __*__ instance variables, but that is a lot smaller exception than
changing the namespace of the assignment operator and the time at which
that operator is executed.
>> If we are going to invent a syntax for declaring function
>> attributes inside the function, why confuse the issue by making it look
>> like it does something else?
>
> Because 1) adding new syntax should be resisted as much as possible
> (IMO),
If one can extend old syntax in an intuitive way, it's fine to reuse old
syntax. When you extend old syntax to mean something counterintuitive,
it's far better to find a new syntax. And while it may seem intuitive
to you, you may have noticed that it seems counterintuitive to a lot of
others.
> and 2) this style has parallels in both class definitions and
> module definitions.
It parallels them as long as you don't think of how it works. Once you
do think of how it works, it's completely different. And you get a lot
of subtle differences to confuse the issue.
> Compare this class statement with the function def
> above.
>
> class Oval(Circle):
> """This describe's Oval."""
> __author__ = 'Paul Morrow'
> __version__ = '0.1'
> __metaclass__ = M
Yes, I can compare that. It's fine in a class body, which is executed
when the class definition is executed. When I see it in a function
definition, I see a strange use of the assignment operator, which is
normally a 'run-time' operator, in a place where it not being executed
at run-time.
You keep displaying such examples as if they should make your point
self-evident. Already several people have pointed out that they don't
see it at self-evident. That might give you a clue that it isn't
self-evident.
>> It's true that it makes it look more like what __*__ attribute
>> assignment in class bodies does, but the simple fact is that class
>> bodies are executed when the class statement is executed, and function
>> bodies are not executed when the def statement is executed. Now you
>> want part of the function bodies to be executed at def time, and part at
>> call time.
>
> Maybe we just need to be clear as to which lines under the def statement
> constitute the function's /body/.
That's perfectly clear without your change. Your change is confusing
the issue, by also using an operator which one (well, at least many of
us - not you, obviously) expects to be executed in the function body.
> The function's docstring gets
> assigned during processing of the def statement, therefore the docstring
> is apparently not part of the function body;
The function's doc string doesn't look like a statement, and even if it
one thinks it is one (which documentation handling somehow picks out
specially) it would be a no-op inside the function, so there is nothing
confusing about it.
BTW, I was all for having the function body extend further down - my
first decorator choice (before J2) would have been the
def foo(...):
...decorators and doc string...
some_keyword:
...body...
syntax. Like you point out about __*__ assigments it stands out, but
it's even clearer, but it doesn't look like it would normally
dosomething else, so the uninitiated will have far less reason to wonder
what kind of voodoo is going on to make it do what it does.
> So why not have assignments to __xxx__
> variables also be part of the function's declaration.
Because it looks confusing.
> The distinctive appearance of these names helps remind us that they
> have special semantics.
And if we use another syntax, we won't need to be reminded about
anything, and there is nothing to be confused about.
>> __*__ assigments in class bodies can look a little confusing if you
>> don't think of how class declarations work, but it's perfectly simple
>> once you think of how they do work: (...)
>
> Perhaps its just my experience with declarative programming languages,
> but I prefer thinking of __xxx__ assignments in a class as setting
> 'properties' of the class. (...)
I prefer to try to think a bit like the language I'm using does.
At least for me it makes it far easier to keep things straight.
--
Hallvard
Hallvard B Furuseth
>Jeff Epler wrote:
>> In your proposed model, what does the following code do?
>> def f():
>> __var__ = 3
>> return __var__
>> f.__var__ += 1
>> print f()
>>
>
> Only assignments to __xxx__ variables at the very start of a function
> def would have this special semantics. So your return statement would
> be referencing a name (__var__) that doesn't exist in the function's
> local variable namespace.
No, that would silently return a surprising result if __var__ existed in
the global namespace. Better make it a syntax error or something.
--
Hallvard
Hallvard B Furuseth
Of course that's obvious with your simple example, since it clearly
doesn't even use those variables and you can even tell from the function
name what the function ought to do. What is not obvious is how it
works.
OK, I asked my neighbor: She suggested some kind of default variables.
She noticed that the variables were not being used; if the function had
been longer she would apparently have needed to read through the entire
function to be sure.
I think my guess would have been something like C's static variables.
The assigment would be initialization, but of a variable in the same
namespace. Which seems similar to her guess, even though we thought
of it very differently.
--
Hallvard
Paul Morrow
> Paul Morrow wrote:
>
>>Hallvard B Furuseth wrote:
>>
>>>Paul Morrow wrote:
>>>
>>>>I believe that we should think of assignments to __xxx__ attributes as
>>>>not being part of the function's body, but instead part of its
>>>>declaration, just as we do with its docstring.
>>>>
>>>> def circum(diameter):
>>>> """This describe's foo."""
>>>> __author__ = 'Paul Morrow'
>>>> __version__ = '0.1'
>>>> __features__ = synchronized, memoized
>>>
>>>
>>>I really, really don't like this idea. Assignment in Python has quite
>>>enough pitfalls already. Please don't make it worse. If it looks like
>>>an assignment to a local variable, it should be an assignment to a local
>>>variable.
>>
>>Actually, these look like assignments to local *magic* variables.
>>There's the difference.
>
>
> No, that's not the difference. It's still different from how magic
> variables behave in all other places. (Class bodies, module bodies,
> object attributes). The difference is that magic variables in this
> particular localtion is actually parsed differently from use of
> variables other places, so the '=' operator (and what other operators?)
> means something else.
>
Only the '=' operator, no others.
They [*] would be parsed when the function definition is processed and
then assigned to the function. Just as in the following, they are
parsed when the class definition is processed and then assigned to the
class. Nothing special happens in either case. Just names being added
to a namespace.
class Circle(Shape):
"""This describe's Circle."""
__author__ = 'Paul Morrow'
__version__ = '0.1'
__metaclass__ = M
[*] the magic variables
>
>>The double underscores before and after each name loudly proclaims
>>that these variables are not like 'normal' variables. They're special
>>in some way.
>
>
> No other magic variables are that special. They all behave normally _as
> variables_. Other parts of Python react to them being set or reset, the
> very variable assigments are quite normal. Except for the name mangling
> of __*__ instance variables, but that is a lot smaller exception than
> changing the namespace of the assignment operator and the time at which
> that operator is executed.
>
Assignments that pertain to the definition of something are processed
when the thing is defined. When used as I've been describing here,
__xxx__ variables pertain to the definition of the function. Therefore,
they are processed when the function is defined. That's the *right*
time to process them.
>
>>>If we are going to invent a syntax for declaring function
>>>attributes inside the function, why confuse the issue by making it look
>>>like it does something else?
>>
>>Because 1) adding new syntax should be resisted as much as possible
>>(IMO),
>
>
> If one can extend old syntax in an intuitive way, it's fine to reuse old
> syntax. When you extend old syntax to mean something counterintuitive,
> it's far better to find a new syntax. And while it may seem intuitive
> to you, you may have noticed that it seems counterintuitive to a lot of
> others.
>
I have no idea how the lurkers feel about this. Maybe you are right.
Maybe this only makes sense to me.
>
>>and 2) this style has parallels in both class definitions and
>>module definitions.
>
>
> It parallels them as long as you don't think of how it works. Once you
> do think of how it works, it's completely different. And you get a lot
> of subtle differences to confuse the issue.
>
You need to show me the subtle differences. I'm not aware of any (that
matter).
>
>>Compare this class statement with the function def
>>above.
>>
>> class Oval(Circle):
>> """This describe's Oval."""
>> __author__ = 'Paul Morrow'
>> __version__ = '0.1'
>> __metaclass__ = M
>
>
> Yes, I can compare that. It's fine in a class body, which is executed
> when the class definition is executed. When I see it in a function
> definition, I see a strange use of the assignment operator, which is
> normally a 'run-time' operator, in a place where it not being executed
> at run-time.
>
It's also a 'definition-time' operator, as it often appears in class and
module definitions. So why not be consistent and let it appear in
function definitions too? What is so outrageous about this idea? On the
'strange-meter', does it rank as high as docstrings or meaningful
indentation?
> You keep displaying such examples as if they should make your point
> self-evident. Already several people have pointed out that they don't
> see it at self-evident. That might give you a clue that it isn't
> self-evident.
>
Again, I have no idea if I'm alone on this. A few have suggested that
they somewhat agree, but there's not been much support beyond that.
>
>>>It's true that it makes it look more like what __*__ attribute
>>>assignment in class bodies does, but the simple fact is that class
>>>bodies are executed when the class statement is executed, and function
>>>bodies are not executed when the def statement is executed. Now you
>>>want part of the function bodies to be executed at def time, and part at
>>>call time.
>>
>>Maybe we just need to be clear as to which lines under the def statement
>>constitute the function's /body/.
>
>
> That's perfectly clear without your change. Your change is confusing
> the issue, by also using an operator which one (well, at least many of
> us - not you, obviously) expects to be executed in the function body.
>
We change the definition of where the function body starts, to exclude
any __xxx__ assignments immediately following the docstring. Then the
'=' operator has definition-time semantics, not run-time.
>
>>The function's docstring gets
>>assigned during processing of the def statement, therefore the docstring
>>is apparently not part of the function body;
>
>
> The function's doc string doesn't look like a statement, and even if it
> one thinks it is one (which documentation handling somehow picks out
> specially) it would be a no-op inside the function, so there is nothing
> confusing about it.
>
> BTW, I was all for having the function body extend further down - my
> first decorator choice (before J2) would have been the
> def foo(...):
> ...decorators and doc string...
> some_keyword:
> ...body...
>
> syntax. Like you point out about __*__ assigments it stands out, but
> it's even clearer, but it doesn't look like it would normally
> dosomething else, so the uninitiated will have far less reason to wonder
> what kind of voodoo is going on to make it do what it does.
>
All of the suggestions I've seen like that one have the section keyword
at the same level of indentation as the def keyword, which is wrong
(IMO). There is no precedent for that in Python. We are defining
something; every statement related to that definition should be indented
farther to the right than the def statement (IMO).
But this is all moot I guess. Guido has apparently already made up his
mind on the decorator syntax. So I guess I'm done with this subject [*].
Thanks for your feedback.
Paul
[*] for now :-)
Jeff Shannon
> Hallvard B Furuseth wrote:
>
>> Yes, I can compare that. It's fine in a class body, which is executed
>> when the class definition is executed. When I see it in a function
>> definition, I see a strange use of the assignment operator, which is
>> normally a 'run-time' operator, in a place where it not being executed
>> at run-time.
>>
>
> It's also a 'definition-time' operator, as it often appears in class and
> module definitions. So why not be consistent and let it appear in
> function definitions too? What is so outrageous about this idea? On the
> 'strange-meter', does it rank as high as docstrings or meaningful
> indentation?
Yes, it does, because of the way that definition works in Python. Keep
in mind that definitions are generated at *run-time*, with dependent
suites being handled appropriately according to the type (class, def) of
compound statement being executed. Assignment isn't working any
differently in class definitions than it is anywhere else; it's just
that suites are handled slightly differently in class statements than
they are in def statements. (And this different behavior is essential,
as that's what creates the difference between a class and a function.)
You're asking now that what appears to be a single suite is actually two
separate entities, even though there's no visual indication of the
break-point (except for the presence of "magic" variables). One of
those suites is executed when the def statement is; the other is
executed when the function object is called. Wildly different semantics
from very similar-looking code at the same level of indentation, based
on what *looks* to anyone familiar with current Python as if it's merely
an odd naming convention.
(I'm sure that you'll argue that the presence of those "magic" variables
is enough distinction between the two radically-different blocks of
code; I'm also pretty confident that very few people will agree with
you, and that nobody with a reasonable understanding of Python internals
will be among those few people.)
Jeff Shannon
Technician/Programmer
Credit International
Jeff Epler
functions give NameErrors when called, and others succeed.
def f(): # Succeeds
if 1:
__x__ = None
return __x__
def f2(): # Fails (but identical to above, today)
__x__ = None
return __x__
def f3(): # but this one succeeds
global z
__x__ = None
return __x__
def f4(): # and so does this one
for __x__ in (None,): pass
return __x__
def g(): # Succeeds
import os as __x__
return __x__
def g2(): # Fails (but identical to above, today)
__x__ = __import__("os")
return __x__
def h(): # Succeeds
class __x__(object): pass
return __x__
def h2(): # Fails (but identical to above, today)
__x__ = type('__x__', (), {})
def j(): # Succeeds
def __x__(): pass
return __x__
def j2(): # Fails (but identical to above, today)
__x__ = lambda: None
return __x__
You complicate the language when you create a new type of assignment
that is special not because of the syntax, but because of the location
in the suite of "def".
You violate the law that most several special statements are expressible
in terms of assignment (import, class, and def in terms of __import__,
type(), and lambda:) or assignment plus looping (for in terms of while +
except + assignment).
You also add a new constraint not expressible directly in the Grammar
file.
What about these? Are 2/3 of them now to be invalid syntax (and the
remaining gives NameError)? Or will they all compile, and 2/3 of them
give an exception?
def k():
__x__, y = range(2)
return __x__
def k2():
y, __x__ = range(2)
return __x__
def k3():
__y__, __x__ = range(2)
return __x__
Have you defined what happens in this case (I imagine that 'False' is
the proper output, but I also believe that many would be surprised to
get True)?
__x__ = False
def m():
__x__ = True
__y__ = __x__
print m.__y__
Jeff
Paul Morrow
> I think it's crazy to change the language so that some of these
> functions give NameErrors when called, and others succeed.
>
> def f(): # Succeeds
> if 1:
> __x__ = None
> return __x__
>
> def f2(): # Fails (but identical to above, today)
> __x__ = None
> return __x__
>
[snip]
With contrived examples, lots of things can look 'crazy'. You don't
really use __x__ as a local variable name do you? That should be a
compiler warning at least when you do that. In fact, we obviously don't
take magic variables seriously enough (IMO). If we want to invent a new
one, we should probably have to declare it somehow, otherwiwe you get a
'undefined magic name' error. That would make it a little harder to
confuse the reader of your code.
So please try your examples again, but this time choose code that
doesn't misuse magic variable syntax and which might possibly show up in
the real world.
> You complicate the language when you create a new type of assignment
> that is special not because of the syntax, but because of the location
> in the suite of "def".
>
docstrings are the precendent for this.
> You violate the law that most several special statements are expressible
> in terms of assignment (import, class, and def in terms of __import__,
> type(), and lambda:) or assignment plus looping (for in terms of while +
> except + assignment).
>
Yep, only the equal sign ('=') would be allowed there. That's not very
limiting considering the intended purpose (making declarations), and it
helps reinforce the visual separation of declaration and body.
> You also add a new constraint not expressible directly in the Grammar
> file.
>
It would be easy to write a BNF expression that states that the optional
docstring is followed by 0+ assignments to magic variables. So what are
you referring to here?
> What about these? Are 2/3 of them now to be invalid syntax (and the
> remaining gives NameError)? Or will they all compile, and 2/3 of them
> give an exception?
>
> def k():
> __x__, y = range(2)
> return __x__
>
> def k2():
> y, __x__ = range(2)
> return __x__
>
> def k3():
> __y__, __x__ = range(2)
> return __x__
>
As above, I think that the answer to your question becomes obvious once
you try to craft real-life examples, using properly named magic and
local variables.
> Have you defined what happens in this case (I imagine that 'False' is
> the proper output, but I also believe that many would be surprised to
> get True)?
>
> __x__ = False
> def m():
> __x__ = True
> __y__ = __x__
> print m.__y__
>
> Jeff
>
Rewrite and you'll see...
__author__ = 'Morrow' # defines the container's author
def m():
__author__ = 'Smith' # defines m's author
__doc__ = __author__ # defines m's docstring
print m.__doc__
Paul
Jeff Epler
> >You also add a new constraint not expressible directly in the Grammar
> >file.
>
> It would be easy to write a BNF expression that states that the optional
> docstring is followed by 0+ assignments to magic variables. So what are
> you referring to here?
Please show me the Grammar rule you have in mind. Here's the relevant
part of the current Grammar file, for reference:
funcdef: [decorators] 'def' NAME parameters ':' suite
suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
... (simple_stmt and stmt both include expr_stmt as an alternative)
expr_stmt: testlist (augassign testlist | ('=' testlist)*)
As far as thinking of "non-contrived" examples, the only code I've ever
written which explicitly used function attributes used non-underscored
names[1], so your syntax wouldn't help me. You could use __* (class
private; but I wouldn't have wanted name-mangling) or _* (private) but
now you've reached areas where you clearly can't claim you'll affect no
existing code.
On the other hand, looking at examples where the names have no meaning
seems relevant to me, because it shows just how much complication you've
added, and how much currently legal code would have its meaning changed
under your proposal.
> Rewrite and you'll see...
>
> __author__ = 'Morrow' # defines the container's author
> def m():
> __author__ = 'Smith' # defines m's author
> __doc__ = __author__ # defines m's docstring
> print m.__doc__
It sounds like you're suggesting m.__doc__ will be 'Smith'. Now I have to be confused that
y = 3
def n():
__x__ = y
is the same as
def m(): pass
m.__x__ = y
but
y = 3
def n2():
__x__ = __y__
is the same as
m.__x__ = m.__y__
and will raise NameError when the body of the module containing n2
is executed.
But, oh well, you would "never write that", so it doesn't matter that
you can't give it sane semantics.
Jeff
[1] __*__
System-defined names. These names are defined by the interpreter and
it's implementation (including the standard library); applications
should not expect to define additional names using this convention.
The set of names of this class defined by Python may be extended in
future versions. See section 3.3, ``Special method names.''
-- http://docs.python.org/ref/id-classes.html
My function attribute was not system-defined, but
application-defined.