Proposal: reducing self.x=x; self.y=y; self.z=z boilerplate code
Ralf W. Grosse-Kunstleve
This posting is also available in HTML format:
http://cci.lbl.gov/~rwgk/python/adopt_init_args_2005_07_02.html
******************************************************************************
Hi fellow Python coders,
I often find myself writing::
class grouping:
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
# real code, finally
This becomes a serious nuisance in complex applications with long
argument lists, especially if long variable names are essential for
managing the complexity. Therefore I propose that Python includes
built-in support for reducing the ``self.x=x`` clutter. Below are
arguments for the following approach (*please* don't get too agitated
about the syntax right here, it really is a secondary consideration)::
class grouping:
def __init__(self, .x, .y, .z):
# real code right here
Emulation using existing syntax::
def __init__(self, x, y, z):
self.x = x
del x
self.y = y
del y
self.z = z
del z
Is it really that important?
----------------------------
For applications of non-trivial size, yes. Here is a real-world example
(one of many in that source tree):
http://cvs.sourceforge.net/viewcvs.py/cctbx/cctbx/cctbx/geometry_restraints/manager.py?view=markup
Fragment from this file::
class manager:
def __init__(self,
crystal_symmetry=None,
model_indices=None,
conformer_indices=None,
site_symmetry_table=None,
bond_params_table=None,
shell_sym_tables=None,
nonbonded_params=None,
nonbonded_types=None,
nonbonded_function=None,
nonbonded_distance_cutoff=None,
nonbonded_buffer=1,
angle_proxies=None,
dihedral_proxies=None,
chirality_proxies=None,
planarity_proxies=None,
plain_pairs_radius=None):
self.crystal_symmetry = crystal_symmetry
self.model_indices = model_indices
self.conformer_indices = conformer_indices
self.site_symmetry_table = site_symmetry_table
self.bond_params_table = bond_params_table
self.shell_sym_tables = shell_sym_tables
self.nonbonded_params = nonbonded_params
self.nonbonded_types = nonbonded_types
self.nonbonded_function = nonbonded_function
self.nonbonded_distance_cutoff = nonbonded_distance_cutoff
self.nonbonded_buffer = nonbonded_buffer
self.angle_proxies = angle_proxies
self.dihedral_proxies = dihedral_proxies
self.chirality_proxies = chirality_proxies
self.planarity_proxies = planarity_proxies
self.plain_pairs_radius = plain_pairs_radius
# real code, finally
Not exactly what you want to see in a high-level language.
Is there a way out with Python as-is?
-------------------------------------
Yes. If you take the time to look at the file in the CVS you'll find
that I was cheating a bit. To reduce the terrible clutter above, I am
actually using a simple trick::
adopt_init_args(self, locals())
For completeness, the implementation of ``adopt_init_args()`` is here:
http://cvs.sourceforge.net/viewcvs.py/cctbx/scitbx/scitbx/python_utils/misc.py?view=markup
While this obviously goes a long way, it has several disadvantages:
- The solution doesn't come with Python -> everybody has to reinvent.
- People are reluctant to use the trick since scripts become
dependent on a non-standard feature.
- It is difficult to remember which ``import`` to use for
``adopt_init_args`` (since everybody has a local version/variety).
- The ``adopt_init_args(self, locals())`` incantation is hard to
remember and difficult to explain to new-comers.
- Inside the ``__init__()`` method, the same object has two names,
e.g. ``x`` and ``self.x``. This lead to subtle bugs a few times
when I accidentally assigned to ``x`` instead of ``self.x`` or vice
versa in the wrong place (the bugs are typically introduced while
refactoring).
- In some cases the ``adopt_init_args()`` overhead was found to
introduce a significant performance penalty (in particular the
enhanced version discussed below).
- Remember where Python comes from: it goes back to a teaching
language, enabling mere mortals to embrace programming.
``adopt_init_args(self, locals())`` definitely doesn't live up
to this heritage.
Minimal proposal
----------------
My minimal proposal is to add an enhanced version of ``adopt_init_args()``
as a standard Python built-in function (actual name secondary!)::
class grouping:
def __init__(self, x, y, z):
adopt_init_args()
# real code
Here is a reference implementation:
http://cvs.sourceforge.net/viewcvs.py/cctbx/libtbx/libtbx/introspection.py?rev=1.2&view=markup
Implementation of this proposal would remove all the disadvantages
listed above. However, there is another problem not mentioned before:
It is cumbersome to disable adoption of selected variables. E.g.::
class grouping:
def __init__(self, keep_this, and_this, but_not_this, but_this_again):
self.keep_this = keep_this
self.and_this = and_this
self.but_this_again = but_this_again
# real code, finally
would translate into::
class grouping:
def __init__(self, keep_this, and_this, but_not_this, but_this_again):
adopt_init_args(exclusions=["but_not_this"])
# real code
Enhanced syntax proposal
------------------------
The exclusion problem suggests these alternatives::
class grouping:
def __init__(self, self.keep_this, self.and_this, but_not_this,
self.but_this_again):
# real code right here
This is conceptually similar to the existing automatic unpacking of tuples.
A shorter alternative (my personal favorite since minimally redundant)::
class grouping:
def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again):
# real code right here
I guess both versions could be implemented such that users don't incur
a performance penalty compared to the ``self.x=x`` alternative. At the
danger of being overly optimistic: I can imagine that my favorite
alternative will actually be faster (and the fastest).
Enhanced __slot__ semantics proposal
------------------------------------
When ``__slots__`` are used (cool feature!) the boilerplate problem
becomes even worse::
class grouping:
__slots__ = ["keep_this", "and_this", "but_this_again"]
def __init__(self, keep_this, and_this, but_not_this, but_this_again):
self.keep_this = keep_this
self.and_this = and_this
self.but_this_again = but_this_again
# real code, finally
Each variable name appears four times! Imagine yourself having to
do this exercise given the real-world example above. Ouch.
Based on the "Enhanced syntax proposal" above I see this potential
improvement::
class grouping:
__slots__ = True
def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again):
# real code right here
Each variable name appears only once. Phew!
Author: rw...@yahoo.com, July 02, 2005
P.S.: If you reply to this message, please clearly separate
naming/syntax issues from the core issue of providing built-in support
designed to reduce clutter.
____________________________________________________
Yahoo! Sports
Rekindle the Rivalries. Sign up for Fantasy Football
http://football.fantasysports.yahoo.com
Gregory K. Johnson
[...]
> Minimal proposal
> ----------------
>
> My minimal proposal is to add an enhanced version of ``adopt_init_args()``
> as a standard Python built-in function (actual name secondary!)::
>
> class grouping:
>
> def __init__(self, x, y, z):
> adopt_init_args()
> # real code
> Enhanced syntax proposal
> ------------------------
>
> The exclusion problem suggests these alternatives::
>
> class grouping:
>
> def __init__(self, self.keep_this, self.and_this, but_not_this,
> self.but_this_again):
> # real code right here
> A shorter alternative (my personal favorite since minimally redundant)::
>
> class grouping:
>
> def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again):
> # real code right here
> P.S.: If you reply to this message, please clearly separate
> naming/syntax issues from the core issue of providing built-in support
> designed to reduce clutter.
I share many of the qualms of those who responded to your earlier post.
(Is this really a problem? Can't a good editor save you the typing?
Explicit is better than implicit, and all that.)
But it also occurs to me to ask: Is a function-definition syntax
extension really appropriate when it's only likely usefulness is for the
particular function called __init__()? That seems more un-Pythonic to me
than straightforward, idiomatic, but somewhat verbose boilerplate.
An adoptargs([excluded=None]) builtin or similar seems much more viable.
(Although even there I don't feel any pressing need: I'm content with
"self.x = x".)
--
Gregory K. Johnson
Roy Smith
> class grouping:
>
> def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again):
> # real code right here
I'm really torn about this. On the one hand, my first thought was "you
shouldn't be writing constructors with arguments lists so long that this is
a problem", but you showed a reasonable counter-example to that argument.
I'm a big fan of DRY (Don't Repeat Yourself), sometimes expressed as "Once
And Only Once", and your proposal lets you do that.
It also has the nice feature that it doesn't break any existing code; the
suggested syntax is not currently legal Python.
What happens if I do:
def __init__ (self, .x, .y, .z):
x = 0
what does the assignment x do? Does it automatically get promoted to an
assignment to self.x? Does it generate an error?
The big question in my mind is not "Is this useful" (since it clearly is),
but "Does the utility justify the cost?". In other words, will it be used
frequently enough to compensate for the added complexity to the language?
I'm not convinced of that.
There have been some proposals floating around to implement a "with"
keyword which would create implicit namespaces. That's sort of what you're
proposing here. I'm not convinced either is a good idea, but if they were
to be adopted, I'd certainly want to see the them done in a uniform,
logically consistent way.
Andrew Koenig
news:mailman.1236.1120298...@python.org...
> class grouping:
>
> def __init__(self, .x, .y, .z):
> # real code right here
> Emulation using existing syntax::
> def __init__(self, x, y, z):
> self.x = x
> del x
> self.y = y
> del y
> self.z = z
> del z
I think this is a bad idea, for a subtle reason.
In Python, unlike many other languages, the names of formal parameters are
part of a function's interface. For example:
def f(x, y):
return x-y
Now f(3, 4) is -1 and f(y=3,x=4) is 1.
The names of instance variables are generally not part of a class'
interface--they are part of its implementation.
This proposed feature, whenever used, would tie a class' implementation to
the interface of every method that uses the feature. As far as I can see,
it is impossible to use the feature without constraining the implementation
in this way.
For this reason, I would much rather have the mapping between parameter
names and instance variables be explicit.
Roy Smith
> In Python, unlike many other languages, the names of formal parameters are
> part of a function's interface. For example:
>
> def f(x, y):
> return x-y
>
> Now f(3, 4) is -1 and f(y=3,x=4) is 1.
>
> The names of instance variables are generally not part of a class'
> interface--they are part of its implementation.
>
> This proposed feature, whenever used, would tie a class' implementation to
> the interface of every method that uses the feature. As far as I can see,
> it is impossible to use the feature without constraining the implementation
> in this way.
While I suppose that's true from a theoretical point of view, as a
practical matter, I don't see it being much of a big deal. I don't think
I've ever written an __init__ method which saved its parameters and used
different names for the parameter and the corresponding instance variable.
Doing so would just be confusing (at least for the kind of code I write).
Also, it doesn't really tie it in any hard and fast way. Right now, I
would write:
def __init__ (self, x, y, z):
self.x = x
self.y = y
self.z = z
blah
under the new proposal, I would write:
def __init__ (self, .x, .y, .z):
blah
If at some time in the future, if I decided I need to change the name of
the instance variable without changing the exposed interface, it would be
easy enough to do:
def __init__ (self, .x, .y, z):
self.zeta = z
blah
I'm still not convinced we need this, but the exposed interface issue
doesn't worry me much.
Walter Brunswick
class Grouping:
def __init__(self,x,y,z):
self.__dict__.update(locals())
Ron Adam
> class grouping:
>
> def __init__(self, .x, .y, .z):
> # real code right here
The way this would work seems a bit inconsistent to me. Args normally
create local variables that receive references to the objects passed to
them.
In this case, a function/method is *creating* non local names in a scope
outside it's own name space to receive it's arguments. I don't think
that's a good idea.
A better approach is to have a copy_var_to(dest, *var_list) function
that can do it. You should be able to copy only selected arguments, and
not all of them.
copy_var_to(self,x,z)
Not exactly straight forward to do as it runs into the getting an
objects name problem.
> Emulation using existing syntax::
>
> def __init__(self, x, y, z):
> self.x = x
> del x
> self.y = y
> del y
> self.z = z
> del z
The 'del's aren't needed as the references will be unbound as soon as
__init__ is finished. That's one of the reasons you need to do self.x=x
, the other is to share the objects with other methods.
> Is there a way out with Python as-is?
> -------------------------------------
With argument lists that long it might be better to use a single
dictionary to keep them in.
class manager:
def __init__(self, **args):
defaults = {
'crystal_symmetry':None,
'model_indices':None,
'conformer_indices':None,
'site_symmetry_table':None,
'bond_params_table':None,
'shell_sym_tables':None,
'nonbonded_params':None,
'nonbonded_types':None,
'nonbonded_function':None,
'nonbonded_distance_cutoff':None,
'nonbonded_buffer':1,
'angle_proxies':None,
'dihedral_proxies':None,
'chirality_proxies':None,
'planarity_proxies':None,
'plain_pairs_radius':None }
defaults.update(args)
self.data = defaults
# real code
Regards,
Ron
Roy Smith
"Walter Brunswick" <walterb...@sympatico.ca> wrote:
That's pretty clever. The only minor annoyance is that it creates a
self.self. If that bothers you, you can fix it with:
def __init__ (self, x, y, z):
vars = locals()
del vars["self"]
self.__dict__.update(vars)
or, perhaps:
def __init__ (self, x, y, z):
self.__dict__.update(locals())
del self.self
It doesn't give you all the flexibility of the original proposal (i.e.
name-by-name selectivity of what gets imported into self), but it does
solve the OP's OP (Original Poster's Original Problem).
jcar...@uci.edu
> A shorter alternative (my personal favorite since minimally redundant)::
>
> class grouping:
>
> def __init__(self, .keep_this, .and_this, but_not_this, .but_this_again):
> # real code right here
There is also the variant which I proposed on python-dev:
class grouping:
def __init__(self, _keep_this, _and_this, but_not_this, _but this
again):
InitAttrs(self, locals())
#real code goes here
Essentially you replace the '.' with '_', which doesn't require a
syntax change. Unfortunately, both are almost invisible. It does
offer you what you want right now (without that whole waiting a year+
for Python 2.5, PEP process, etc.).
> Enhanced __slot__ semantics proposal
> ------------------------------------
>
> When ``__slots__`` are used (cool feature!) the boilerplate problem
> becomes even worse::
>
> class grouping:
>
> __slots__ = ["keep_this", "and_this", "but_this_again"]
>
> def __init__(self, keep_this, and_this, but_not_this, but_this_again):
> self.keep_this = keep_this
> self.and_this = and_this
> self.but_this_again = but_this_again
> # real code, finally
There is also the AutoSlots metaclass (which I have fixed) that does
this as well.
class grouping(object):
__metaclass__ = AutoSlots
def __init__(self, _keep_this, _and_this, but_not_this,
_but_this_again):
InitAttrs(self, locals())
#real code goes here
Both AutoSlots and InitAttrs use leading underscores on the __init__
method to discover which attributes are to be __slots__, and which
should be automatically assigned.
> P.S.: If you reply to this message, please clearly separate
> naming/syntax issues from the core issue of providing built-in support
> designed to reduce clutter.
Because you don't seem to have listened in python-dev, I'll say it
here. Not everything that reduces clutter should be syntax, and not
every function, class, and module which reduces programming time should
be builtin. Why?
1. Expanding Python syntax bloats the language. It increases what you
need to teach to new Python users. In my opinion, syntax additions
should really only be considered when significant gains in readability
and writability for many users are realized, that a lack of syntax
cannot offer.
2. Expanding the Python standard library bloats the distribution.
Right now, Python is a relatively light download. But if one were to
include the top packages in everything, the distribution would quickly
bloat to 40+ megs. This is not an option. Generally, the requirements
of getting code into the standard library is either a demonstrated need
for the addition, or a known best-of-breed implementation for a
commonly used module, or both.
I believe your syntax change is a non-starter. Why? Because I've
offered code that does essentially everything you want, without a
syntax change. If someone happens to include AutoSlots and InitAttrs
with their code, module, what have you, and it manages to make its way
into standard Python, so be it (I place the code into the public
domain).
The code for the InitAttrs and AutoSlots mechanism are available here:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/435880
- Josiah
Mike C. Fletcher
...
> class grouping:
>
> def __init__(self, x, y, z):
> self.x = x
> self.y = y
> self.z = z
> # real code, finally
>
>This becomes a serious nuisance in complex applications with long
>argument lists, especially if long variable names are essential for
>managing the complexity. Therefore I propose that Python includes
>built-in support for reducing the ``self.x=x`` clutter. Below are
>arguments for the following approach (*please* don't get too agitated
>about the syntax right here, it really is a secondary consideration)::
>
>
...
This can be dealt with quite nicely if you take a step back and consider
what you're trying to do:
* define a set of properties/fields for a class, potentially with
default values for any given property
* define an initialisation function which, given an instance of a
class with defined properties (potentially with defaults) and a
mapping of name:value pairs, assigns the named values to the named
properties...
class Grouping( propertied.Propertied ):
x = FloatProperty(
"x", """Holder for the grouping node's x coordinate""",
defaultValue = 0,
)
y = FloatProperty(
"y", """Holder for the grouping node's y coordinate""",
defaultValue = 0,
)
z = FloatProperty(
"z", """Holder for the grouping node's z coordinate""",
defaultValue = 0,
)
def __init__( self, **args ):
# code here to do whatever you like to args (or whatever)...
super( Grouping, self ).__init__( **args )
# extra code here...
where the propertied.Propertied class defines a simple init that
iterates through the defined properties pulling those which are present
in the **args of the __init__ function and assigning them to the
appropriate property.
The advantage here is that your class is now documented properly, with
the information about a given property all localised in a definition of
the property. Your properties can also implement the observer pattern,
automatic type checking and coercion, and can provide default values
that appear when you delete current values (not just during
initialisation). You can also do useful things like using the defined
properties of a class to auto-generate __repr__ and __str__ values. You
can have your properties delegate to other objects, or have them delay
loading/resolving a property's value until the last possible second
(lazy loading from an object database, for instance).
The fact that it's all doable in Python 2.2+ without any need for new
syntax... well... that's cool too.
The disadvantage is that the properties tend to be unordered (unless you
explicitly order them, and that's awkward looking), and you have to pass
them to the constructor as keyword arguments. But then, if you're
defining a class with 50 properties, you likely aren't really wanting to
pass all 50 as positional arguments anyway.
You'll find patterns like this in my own OpenGLContext (3D scenegraph
viewer) and BasicProperty packages, as well as in Traits (originally
part of a 2D graphics package), and Zope's FieldProperty (and PEAK,
though as always with PEAK, it's somewhat weird in there ;) ). All of
those are largish systems (or used in largish systems), btw.
Just my thoughts,
Mike
--
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com
Terry Hancock
> I often find myself writing::
>
>
> def __init__(self, x, y, z):
> self.x = x
> self.y = y
> self.z = z
> # real code, finally
Fortunately, you don't have to, try this:
#(1)
class grouping:
def __init__(self, *args):
parms = ('x', 'y', 'z')
arg_parms = zip(parms, args[:len(parms)])
for parm, arg in arg_parms:
setattr(self, parm, arg)
In this case, of course, this isn't any shorter, but if you have
a few dozen parameters, it can be.
Note, that it may be stylistically better to do this:
#(2)
class grouping:
def __init__(self, **kw):
for key, val in kw.items():
setattr(self, key, val)
or possibly:
#(3)
class grouping:
def __init__(self, **kw):
parms = ('x', 'y', 'z')
for key, val in kw.items():
if key in parms: setattr(self, key, val)
Because, if you are going to have dozens of arguments, it will be hard
for you to remember their order in the caller, and keyword arguments
will make it possible to set them by name. Example (2) doesn't validate
the arguments -- allowing you to set any arbitrary collection of values
you want (can be a useful way to create a shared namespace), but (3)
does (which is probably important if your class actually does specific
things with the parameters).
--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com
Andrea Griffini
<rw...@yahoo.com> wrote:
>Hi fellow Python coders,
>
>I often find myself writing::
>
> class grouping:
>
> def __init__(self, x, y, z):
> self.x = x
> self.y = y
> self.z = z
> # real code, finally
>
>This becomes a serious nuisance in complex applications with long
>argument lists, especially if long variable names are essential for
>managing the complexity. Therefore I propose that Python includes
>built-in support for reducing the ``self.x=x`` clutter.
With some help from new-style classes you can get more than
just removing the "self.x = x" clutter.
I'm not an expert of these low-level python tricks, but
you can download from http://www.gripho.it/objs.py a
small example that allows you to write
class MyClass(Object):
x = Float(default = 0.0, max = 1E20)
y = Float(min = 1.0)
and you can get in addition of redudancy removal also
parameter checking. You can also have an __init__
method that gets called with attributes already set up.
HTH
Andrea
Bengt Richter
What if parameter name syntax were expanded to allow dotted names as binding
targets in the local scope for the argument or default values? E.g.,
def foometh(self, self.x=0, self.y=0): pass
would have the same effect as
def foometh(self, self.y=0, self.x=0): pass
and there would be a persistent effect in the attributes of self
(whatever object that was), even with a body of pass.
I'm not sure about foo(self, **{'self.x':0, 'self.y':0}), but if
you didn't capture the dict with a **kw formal parameter, IWT you'd
have to be consistent and effect the attribute bindings implied.
(Just a non-thought-out bf here, not too serious ;-)
Regards,
Bengt Richter
Ron Adam
> What if parameter name syntax were expanded to allow dotted names as binding
> targets in the local scope for the argument or default values? E.g.,
>
> def foometh(self, self.x=0, self.y=0): pass
>
> would have the same effect as
>
> def foometh(self, self.y=0, self.x=0): pass
>
> and there would be a persistent effect in the attributes of self
> (whatever object that was), even with a body of pass.
>
> I'm not sure about foo(self, **{'self.x':0, 'self.y':0}), but if
> you didn't capture the dict with a **kw formal parameter, IWT you'd
> have to be consistent and effect the attribute bindings implied.
>
> (Just a non-thought-out bf here, not too serious ;-)
>
> Regards,
> Bengt Richter
Well it works the other way around to some degree.
def foo(self, x=x, y=y):pass
x=x binds the class variables to the arguments without the self. if no
value is given.
Which is kind of strange, since x by it self gives an error if no value
is given. The strange part is x=x is not the same as just x. I
understand why, but it still looks odd.
Why isn't there a dict method to get a sub dict from a key list?
Fromkeys doesn't quite do it.
sub-dict = dict.subdict(key_list)
Or have dict.copy() take a key list. (?)
<Just a thought.>
The following works and doesn't seem too obscure, although the x=x,
etc.. could be annoying if they were a lot of long names.
Seems like mutable default arguments is what's needed to make it work,
not that it's needed IMO. But it's an interesting problem.
def subdict(dict, keys):
d = {}
for k in keys:
d[k] = dict[k]
return d
class foo(object):
x = 1
y = 2
z = 3
def __init__(self,x=x,y=y,z=z):
save_these = subdict(locals(),['x','y'])
self.__dict__.update(save_these)
# rest of code
print self.x, self.y, self.z
f = foo()
f = foo(5,6,7)
Bengt Richter
>Bengt Richter wrote:
>
>> What if parameter name syntax were expanded to allow dotted names as binding
>> targets in the local scope for the argument or default values? E.g.,
>>
>> def foometh(self, self.x=0, self.y=0): pass
>>
>> would have the same effect as
>>
>> def foometh(self, self.y=0, self.x=0): pass
>>
>> and there would be a persistent effect in the attributes of self
>> (whatever object that was), even with a body of pass.
>>
>> I'm not sure about foo(self, **{'self.x':0, 'self.y':0}), but if
>> you didn't capture the dict with a **kw formal parameter, IWT you'd
>> have to be consistent and effect the attribute bindings implied.
>>
>> (Just a non-thought-out bf here, not too serious ;-)
>>
>> Regards,
>> Bengt Richter
>
>Well it works the other way around to some degree.
>
>def foo(self, x=x, y=y):pass
>
>x=x binds the class variables to the arguments without the self. if no
>value is given.
>
>Which is kind of strange, since x by it self gives an error if no value
>is given. The strange part is x=x is not the same as just x. I
>understand why, but it still looks odd.
ISTM you are comparing apples to oranges, execution-wise. The def in the
context of a class definition is executed as soon as you get to the end
of the class suite. The execution happens in a special name space similar
to what happens when you execute a function suite by calling the function,
except the class definition body is executed automatically.
When you write def foo(self, x=x): ... the second x is looked up
starting in the local excution namespace of the class body, so it finds
class variables, if they are defined, otherwise it looks for an outer
scope x for the value.
Note that order counts:
>>> class Foo(object):
... def foo(self, x=x): print x
... x = 123
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in Foo
NameError: name 'x' is not defined
>>> class Foo(object):
... x = 123
... def foo(self, x=x): print x
...
>>> Foo().foo()
123
Note that self.x is a different access mechanism, and would access the
same Foo.x that became a default value, but the value is the only thing
they have in common. You could extract the function from the class,
and the plain function would still have the default value:
>>> bar = Foo.foo.im_func
>>> bar('ignored self param')
123
>>> import dis
>>> dis.dis(bar)
3 0 LOAD_FAST 1 (x)
3 PRINT_ITEM
4 PRINT_NEWLINE
5 LOAD_CONST 0 (None)
8 RETURN_VALUE
>>> import inspect
>>> inspect.getargspec(bar)
(['self', 'x'], None, None, (123,))
There's where the default value is. It's put there when x=x is evaluated
during the def execution for the function, which happens to have happened
courtesy of a class body execution defining the class, but that's
the only connection the 123 has.
>
>
>Why isn't there a dict method to get a sub dict from a key list?
>Fromkeys doesn't quite do it.
>
> sub-dict = dict.subdict(key_list)
>
>Or have dict.copy() take a key list. (?)
>
><Just a thought.>
sub_dict = dict([(key, srcdct[key]) for key in srcdct]) #untested, should work
(if you don't use 'dict' for a variable ;-)
>
>
>
>The following works and doesn't seem too obscure, although the x=x,
>etc.. could be annoying if they were a lot of long names.
>
>Seems like mutable default arguments is what's needed to make it work,
>not that it's needed IMO. But it's an interesting problem.
>
>
>def subdict(dict, keys):
> d = {}
> for k in keys:
> d[k] = dict[k]
> return d
>
>class foo(object):
> x = 1
> y = 2
> z = 3
> def __init__(self,x=x,y=y,z=z):
> save_these = subdict(locals(),['x','y'])
> self.__dict__.update(save_these)
self.__dict__.update({'x':x, 'y':y, 'z':z}) # should work without save_these
>
> # rest of code
> print self.x, self.y, self.z
>
>f = foo()
>f = foo(5,6,7)
>
>
Regards,
Bengt Richter
Bengt Richter
oops, I see you only wanted x and y, so that should be
self.__dict__.update({'x':x, 'y':y}) # should work without save_these
mch...@gmail.com
> I often find myself writing::
>
> class grouping:
>
> def __init__(self, x, y, z):
> self.x = x
> self.y = y
> self.z = z
> # real code, finally
>
> This becomes a serious nuisance in complex applications with long
> argument lists
Yes... indeed it does. This is so common that there is a standard
idiom for handling it:
def __init__(self, x, y, z):
self.__dict__.update(locals())
sometimes with modifications to avoid setting self.self.
> Therefore I propose that Python includes
> built-in support for reducing the ``self.x=x`` clutter.
If all you were proposing was a built-in function to make this
particular
idiom clearer and more reliable, then I think I'd back such a feature
because the need is SO common. However, the suggestion you actually
make:
> def __init__(self, .x, .y, .z):
> # real code right here
is far too broad and introduces new syntax unnecessarily.
You yourself are using a helper function (although I belive it could
be done more easily than you did it):
> I am actually using a simple trick::
>
> adopt_init_args(self, locals())
To which you raise the following objections:
> - The solution doesn't come with Python -> everybody has to reinvent.
Good point. Particularly since people won't think of all the
special cases (eg: classes with __slots__ defined).
> - People are reluctant to use the trick since scripts become
> dependent on a non-standard feature.
> - It is difficult to remember which ``import`` to use for
> ``adopt_init_args`` (since everybody has a local version/variety).
If the implementation is only 3-4 lines long (and a simpler
implementation
can be), then is can simply be included inline with every script that
needs
to use it.
> - The ``adopt_init_args(self, locals())`` incantation is hard to
> remember and difficult to explain to new-comers.
A better name would help with this. The need for locals() is
unavoidable.
But for REAL beginners, I wouldn't even bother... writing out "self.x =
x"
is useful for beginners since it helps make it very clear and concrete
to
them just what is happening.
> - Inside the ``__init__()`` method, the same object has two names,
> e.g. ``x`` and ``self.x``. This lead to subtle bugs a few times
> when I accidentally assigned to ``x`` instead of ``self.x`` or vice
> versa in the wrong place (the bugs are typically introduced while
> refactoring).
Hmm... I've never had that problem, myself.
> - In some cases the ``adopt_init_args()`` overhead was found to
> introduce a significant performance penalty (in particular the
> enhanced version discussed below).
Again... a different code will help here. And if execution speed is
REALLY a concern, then you can just write it out the long way!
> - Remember where Python comes from: it goes back to a teaching
> language, enabling mere mortals to embrace programming.
> ``adopt_init_args(self, locals())`` definitely doesn't live up
> to this heritage.
No, but "self.x = x" does. It's only when you have lots of variables
or very long names that this approach becomes unwieldy.
> My minimal proposal is to add an enhanced version of ``adopt_init_args()``
> as a standard Python built-in function (actual name secondary!)::
I'd alter the name and the implementation, but the basic idea seems
sound to me.
> However, there is another problem not mentioned before:
> It is cumbersome to disable adoption of selected variables.
The VERY simple, VERY straightforward, VERY common behavior of
"store all the arguments as like-named attributes of self" is
worth having a standard idiom (and *perhaps* a built-in). But
odd special cases like skipping some arguments... that calls
for writing the whole thing out. I'm firmly -1 on any proposal
to support skipping arguments.
> When ``__slots__`` are used (cool feature!) the boilerplate problem
> becomes even worse::
>
> class grouping:
>
> __slots__ = ["keep_this", "and_this", "but_this_again"]
>
> def __init__(self, keep_this, and_this, but_not_this, but_this_again):
> self.keep_this = keep_this
> self.and_this = and_this
> self.but_this_again = but_this_again
> # real code, finally
>
> Each variable name appears four times!
** NO! **
__slots__ is *NOT* to be used except for those times when you NEED
the performance advantages (mostly memory use). The simple rule is
that you should *NEVER* use __slots__ (if you are in a situation
where you *do* need it, then you'll know enough to understand why
this advice doesn't apply to you). There should NOT be any support
for auto-setting __slots__ *anywhere* in the standard library,
because it would make it FAR too tempting for people to mis-use
__slots__.
Besides, a metaclass would be a better solution, and it can be done
today with no modifications to Python.
. . .
All in all, I think there's SOME merit to this idea, in that this
is a common enough practice that it might be nice to make it easy
to type (and read). But your proposal entangles the good idea with
several ideas I rather dislike, and on the whole I think it sounds
rather dangerous.
-- Michael Chermside
Thomas Heller
> Ralf W. Grosse-Kunstleve wrote:
>> I often find myself writing::
>>
>> class grouping:
>>
>> def __init__(self, x, y, z):
>> self.x = x
>> self.y = y
>> self.z = z
>> # real code, finally
>>
>> This becomes a serious nuisance in complex applications with long
>> argument lists
>
> Yes... indeed it does. This is so common that there is a standard
> idiom for handling it:
>
> def __init__(self, x, y, z):
> self.__dict__.update(locals())
>
> sometimes with modifications to avoid setting self.self.
>> I am actually using a simple trick::
>>
>> adopt_init_args(self, locals())
>
> If the implementation is only 3-4 lines long (and a simpler
> implementation can be), then is can simply be included inline with
> every script that needs to use it.
>
>> - The ``adopt_init_args(self, locals())`` incantation is hard to
>> remember and difficult to explain to new-comers.
>
> A better name would help with this. The need for locals() is
> unavoidable.
Ahem - sys._getframe()
Thomas
Ralf W. Grosse-Kunstleve
Thanks for taking a careful look, and thanks for the balanced analysis!
Michael Chermside wrote:
> A better name would help with this.
OK. After getting so many beatings I'd agree to anything. :)
> The need for locals() is unavoidable.
It is in fact not unavoidable. I pointed to my prototype before,
but sorry, it is probably confusing that my two prototypes have the
same name. They just live in different modules. For easier access I
inline the second version that works without having to write "locals()"
(from libtbx.introspection):
def frame_object(frames_back=0):
try: raise Exception
except:
t = sys.exc_info()[2]
f = t.tb_frame
for i in xrange(frames_back+1):
if (f.f_back is None): break
f = f.f_back
return f
def adopt_init_args(exclusions=[], prefix="", frames_back=0):
frame = frame_object(frames_back=frames_back+1)
varnames = frame.f_code.co_varnames
exclusions.append(varnames[0]) # self
init_locals = frame.f_locals
self = init_locals[varnames[0]]
for varname in varnames:
if (varname not in exclusions):
setattr(self, prefix+varname, init_locals[varname])
if ("__init__varnames__" not in exclusions):
setattr(self, "__init__varnames__", varnames)
I am sure this can be improved and stripped down to the essentials.
E.g. I wasn't aware of sys._getframe() as just pointed out by Thomas
Heller (thanks!). To me this just shows that finding the optimal
solution is not entirely trivial and should therefore be done "right"
once and forever in a community effort. Ultimately I hope it will be
done in C for speed. That's the main goal of my proposal.
> But for REAL beginners, I wouldn't even bother... writing out "self.x = x"
> is useful for beginners since it helps make it very clear and concrete to
> them just what is happening.
In my experience real Python beginners grow very skeptical when I
make them type highly redundant code. Many are just waiting for
excuses not to learn Python and to continue with whatever they know
(in my field FORTRAN!). Even
class point:
def __init__(self, x, y):
self.x = x
self.y = y
makes for wrinkled noses. If I didn't use my home-grown
adopt_init_args() function in the real-world code pointed out earlier
the FORTRAN faction probably wouldn't stop making fun of me ("Why
doesn't he use COMMON blocks?").
> > - In some cases the ``adopt_init_args()`` overhead was found to
> > introduce a significant performance penalty (in particular the
> > enhanced version discussed below).
>
> Again... a different code will help here. And if execution speed is
> REALLY a concern, then you can just write it out the long way!
Why should anyone have to think about technicalities like this?
If we had an efficient C version included with Python nobody would
have to. One solution works for everything.
> The VERY simple, VERY straightforward, VERY common behavior of
> "store all the arguments as like-named attributes of self" is
> worth having a standard idiom (and *perhaps* a built-in). But
> odd special cases like skipping some arguments... that calls
> for writing the whole thing out. I'm firmly -1 on any proposal
> to support skipping arguments.
OK. "del x" is almost as good. I was motivated by speed concerns
(first adding x to self.__dict__, then removing it again), but it is
most likely a very minor issue, and my prototype with the repeated
list lookup may even backfire anyway.
> > When ``__slots__`` are used (cool feature!) the boilerplate problem
> > becomes even worse::
> >
> > class grouping:
> >
> > __slots__ = ["keep_this", "and_this", "but_this_again"]
> >
> > def __init__(self, keep_this, and_this, but_not_this,
but_this_again):
> > self.keep_this = keep_this
> > self.and_this = and_this
> > self.but_this_again = but_this_again
> > # real code, finally
> >
> > Each variable name appears four times!
>
> ** NO! **
>
> __slots__ is *NOT* to be used except for those times when you NEED
> the performance advantages (mostly memory use). The simple rule is
> that you should *NEVER* use __slots__ (if you are in a situation
> where you *do* need it, then you'll know enough to understand why
> this advice doesn't apply to you). There should NOT be any support
> for auto-setting __slots__ *anywhere* in the standard library,
> because it would make it FAR too tempting for people to mis-use
> __slots__.
>
> Besides, a metaclass would be a better solution, and it can be done
> today with no modifications to Python.
I am very puzzled to see the huge ** NO! **. If __slots__ improve
performance and reduce memory, what is wrong with using this feature?
I clearly was in situations where I needed __slots__, but I still
don't know what the dangers could be. Could you please explain
a little more?
> All in all, I think there's SOME merit to this idea, in that this
> is a common enough practice that it might be nice to make it easy
> to type (and read). But your proposal entangles the good idea with
> several ideas I rather dislike, and on the whole I think it sounds
> rather dangerous.
I am OK to strip down the proposal to concentrate solely on an
optimal, fast, C-coded built-in function. That alone would be a major
step forward, with a life-changing potential similar to that of zip,
enumerate, or inplace operators (+=, *=, etc.). (I guess I am showing
my age.)
I was also considering working in Josiah Carlson's metaclass solution
(http://mail.python.org/pipermail/python-list/2005-July/288349.html)
to the __slots__ problem, but after reading your message I am not so
sure anymore...
Cheers,
Ralf
NickC
I'd be very interested to hear your opinion on the 'namespace' module,
which looks at addressing some of these issues (the Record object, in
particular). The URL is http://namespace.python-hosting.com, and any
comments should be directed to the name...@googlegroups.com
discussion list.
Regards,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.blogspot.com
Ralf W. Grosse-Kunstleve
> I'd be very interested to hear your opinion on the 'namespace' module,
> which looks at addressing some of these issues (the Record object, in
> particular). The URL is http://namespace.python-hosting.com, and any
> comments should be directed to the name...@googlegroups.com
> discussion list.
Hi Nick,
The namespace module looks interesting, thanks for the pointer! (I saw your
other message but didn't have a chance to reply immediately.)
I tried out the namespace.Record class. The resulting user code looks nice, but
I have two concerns:
- It requires a different coding style; until it is well established it will
surprise people.
- The runtime penalty is severe.
Attached is a simple adopt_timings.py script. If I run it with Python 2.4.1
under RH WS3, 2.8GHz Xeon, I get:
overhead: 0.01
plain_grouping: 0.27
update_grouping: 0.44
plain_adopt_grouping: 0.68
record_grouping: 10.85
I.e. record_grouping (using namespace.Record) is about 40 times slower than the
manual self.x=x etc. implementation.
My conclusion is that namespace.Record may have merits for specific purposes,
but is impractical as a general-purpose utility like I have in mind.
Note that the attached code includes a new, highly simplified "plain_adopt()"
function, based on the information I got through other messages in this thread.
Thanks to everybody for suggestions!
Cheers,
Ralf
____________________________________________________
Sell on Yahoo! Auctions – no fees. Bid on great items.
http://auctions.yahoo.com/
Ralf W. Grosse-Kunstleve
> My apologies for having to resort to email but for some reason I can't post
> this message to comp.lang.python. I've tried about 4 times including
> starting a
> new thread, but for some reason it doesn't turn up, though I've followed
> up on
> another thread without problem.
Very strange. I hope it is not a conspiracy! :)
You can email to the list directly (that's what I am doing):
> > Is there a way out with Python as-is?
> > -------------------------------------
> >
> > Yes. If you take the time to look at the file in the CVS you'll find
> > that I was cheating a bit. To reduce the terrible clutter above, I am
> > actually using a simple trick::
> >
> > adopt_init_args(self, locals())
> >
>
> Also there is a decorator solution:
>
> def init_self( init ):
> vn = init.func_code.co_varnames[ 1 : init.func_code.co_argcount ]
>
> def decorated_init(self, *args, **kw):
> off = 0
> for name in vn:
> if not name.startswith('_'):
> if name in kw:
> value = kw[name]
> else:
> value = args[off]
> off += 1
>
> setattr( self, name, value )
>
> init( self, *args, **kw )
> return decorated_init
>
>
> class MyClass(object):
> __slots__ = ['x', 'z']
>
> @init_self
> def __init__( self, x, _y, z ):
> pass
>
> def show( self ):
> for i in self.__slots__:
> print 'self.%s = %d' %(i,eval('self.%s' % (i,)))
>
> MyClass( 1, 2, 3 ).show()
>
> MyClass( z = 1, x = 2, _y = 3 ).show()
>
> The __slots__ is in the test just because you mentioned you like it,
> the decorator doesn't need a "slots" class.
>
> AFAICT this differs from your proposal in that in your proposal you
> want to use '.' as an include prefix the above uses '_' as an exclude
> prefix.
>
> Rob.
I like the looks of the decorator approach a lot. Could this somehow be
enhanced such that the user doesn't have to know about the underscore? Thinking
about runtime efficiency, could the decorator approach somehow achieve that the
attribute-to-be arguments are never added to locals()?
Lonnie Princehouse
But you can make a decorator to achieve the same result. There's not
really a convincing case for extending python syntax.
def attribute_decorator(f):
import inspect
argnames = inspect.getargspec(f)[0]
def decorator(*args, **keywords):
bound_instance = args[0]
for name, value in zip(argnames[1:], args[1:]):
setattr(bound_instance, name, value)
return f(*args, **keywords)
return decorator
#--------- example use:
class foo(object):
@attribute_decorator
def __init__(self, thing):
print "init: self.thing is", repr(self.thing)
f = foo('hello world')
--ljp
Rob Williscroft
news:mailman.1616.1121111...@python.org in
comp.lang.python:
> --- Robert Williscroft <r...@freenet.co.uk> wrote:
>
>> My apologies for having to resort to email but for some reason I
>> can't post this message to comp.lang.python. I've tried about 4 times
>> including starting a
>> new thread, but for some reason it doesn't turn up, though I've
>> followed up on
>> another thread without problem.
>
> Very strange. I hope it is not a conspiracy! :)
> You can email to the list directly (that's what I am doing):
>
> pytho...@python.org
>
I'll follow up and post this, and then see what happens :).
The underscore is just there so there is a way of having arguments
that aren't added as attributes to the first object, so if that isn't
required it could just be omited.
Also AIUI (the above is my first attempt to write a decorator),
decorators can have arguments so the decorator could take an
"omit_prefix" argument, so the user could specify there own prefix.
Or maybe have an include prefix, say "self_", if thats what you prefer.
Personaly I'm happy with the underscore, 1 leading underscore at
module scope means hide the name, 2 at class scope means hide the
name, so I kinda like that an undersore is used to omit an argument.
> Thinking about runtime efficiency, could the decorator
> approach somehow achieve that the attribute-to-be arguments are never
> added to locals()?
>
Again AIUI, locals() is an inbuild that constructs a dict of the local
arguments and variables, IOW the dict doesn't exist *until* you call
locals().
Wether or not the fact the decorator doesn't call locals() makes any
difference though I don't know, the real arguments will get passed
to decorated_init() in some way and then setattr()'d to the self object.
But then again there is a loop, 2 if's and a lookup of the keyword
dictionary in there, using this as it stands is going to hurt compared
to doing the asignments manually inside __init__(), though
as I haven't timed it, I could be wrong ;-).
Ralf W. Grosse-Kunstleve
> IIRC, the self.__dict__.update(locals()) trick confuses psyco.
>
> But you can make a decorator to achieve the same result. There's not
> really a convincing case for extending python syntax.
Not if you have (enough memory for) psyco. :)
I am doing C++ extensions by hand; did quite a lot of them. Anything that helps
in pushing back the point where I have to move from Python to C++ is highly
appreciated. That's probably the strongest argument for the (self, self.x, ...)
approach. I believe it can be made more efficient than any other solution. But
see also the two other arguments:
http://mail.python.org/pipermail/python-list/2005-July/289721.html
> def attribute_decorator(f):
> import inspect
> argnames = inspect.getargspec(f)[0]
> def decorator(*args, **keywords):
> bound_instance = args[0]
> for name, value in zip(argnames[1:], args[1:]):
> setattr(bound_instance, name, value)
> return f(*args, **keywords)
> return decorator
>
> #--------- example use:
>
> class foo(object):
> @attribute_decorator
> def __init__(self, thing):
> print "init: self.thing is", repr(self.thing)
>
> f = foo('hello world')
Thanks! Rob Williscroft had a similar suggestion:
http://mail.python.org/pipermail/python-list/2005-July/289734.html
Does anyone know if there is a way to hide the _ or self_ from the user of the
class, i.e. given:
class foo(object):
@attribute_decorator
def __init__(self, x, _y, z):
pass
can we make it such that the user can still write
foo(x=1,y=2,z=3)
without the underscore?
Cheers,
Ralf
__________________________________
Discover Yahoo!
Use Yahoo! to plan a weekend, have fun online and more. Check it out!
http://discover.yahoo.com/
Rob Williscroft
comp.lang.python:
> Does anyone know if there is a way to hide the _ or self_ from the
> user of the class, i.e. given:
>
> class foo(object):
> @attribute_decorator
> def __init__(self, x, _y, z):
> pass
>
> can we make it such that the user can still write
>
> foo(x=1,y=2,z=3)
>
> without the underscore?
>
<light-goes-on/>
Sorry I didn't understand what you ment before:
def init_self( init ):
class KeywordArgumentError(Exception):
pass
vn = init.func_code.co_varnames[ 1 : init.func_code.co_argcount ]
def decorated_init(self, *args, **kw):
off = 0
for name in vn:
if not name.startswith('_'):
if name in kw:
value = kw[name]
else:
value = args[off]
off += 1
setattr( self, name, value )
else:
off += 1 #was missing (a bug) in last version.
if name in kw:
raise KeywordArgumentError(
"Use %s not %s" % (name[1:],name)
)
if name[1:] in kw:
kw[name] = kw[name[1:]]
del kw[name[1:]]
init( self, *args, **kw )
return decorated_init
class MyClass(object):
@init_self
def __init__( self, x, _y, z ):
print "in __init__() _y =", _y
def show( self ):
for i in self.__dict__:
print 'self.%s = %d' %(i,eval('self.%s' % (i,)))
MyClass( 1, 2, 3 ).show()
MyClass( z = 1, x = 2, y = 3 ).show()
MyClass( z = 1, x = 2, _y = 3 ).show()
Ralf W. Grosse-Kunstleve
Wow!
Here is the output, so everyone can see it immediately:
in __init__() _y = 2
self.x = 1
self.z = 3
in __init__() _y = 3
self.x = 2
self.z = 1
Traceback (most recent call last):
File "/net/cci/rwgk/decorated_init.py", line 45, in ?
MyClass( z = 1, x = 2, _y = 3 ).show()
File "/net/cci/rwgk/decorated_init.py", line 21, in decorated_init
raise KeywordArgumentError(
__main__.KeywordArgumentError: Use y not _y
I am impressed. I'll do some timings to see where we stand...
Ralf W. Grosse-Kunstleve
> class MyClass(object):
> @init_self
> def __init__( self, x, _y, z ):
> print "in __init__() _y =", _y
Here are the timings (script attached):
overhead: 0.01
plain_grouping: 0.26
update_grouping: 0.45
plain_adopt_grouping: 0.69
autoinit_grouping: 1.14
autoinit_setattr_grouping: 1.07
autoattr_grouping: 1.01
decorated_init_grouping: 0.94
I think your decorator solution looks nice and has a good potential time-wise.