getattr and subclassing
Mark Lutz
book :-), and I came across __getattr__ behavior that seems
strange. Apologees in advance if this has been fixed: I tested
this on 1.1.1 on Linux (don't have 1.2 access at the moment).
I'm trying to wrap a C module in a class, so I can specialize
some of its methods. If I define a class with 1 method per
module function, things work as advertised:
envsub.py:
import os, environ # get C module
class Environ:
def getenv(self, name): # C module wrapper class
return environ.getenv(name) # delegate to C module
def putenv(self, name, value):
return environ.putenv(name, value)
class EnvSync(Environ): # extend by subclassing
def putenv(self, name, value):
os.environ[name] = value
Environ.putenv(self, name, value) # do superclass putenv
def getenv(self, name):
value = Environ.getenv(self, name) # do superclass getenv
os.environ[name] = value # integrity check
return value
Env = EnvSync() # make one instance
> python
>>> import os
>>> from envsub import Env
>>> os.environ['USER']
'mark'
>>> Env.getenv('USER')
'mark'
>>> Env.putenv('USER', 'spam')
>>> os.environ['USER']
'spam'
>>> Env.getenv('USER')
'spam'
But if I use __getattr__ in the superclass, instead of real
methods:
class Environ:
def __getattr__(self, name): # proxy class
return getattr(environ, name) # pass to C module
it doesn't work (I get an Attribute error in the subclass). Here's
the only version that seemed to work with __getattr__ in the Environ
superclass; I had to use different names for the subclass methods:
class EnvSync(Environ): # can't call it "putenv"
def put(self, name, value):
os.environ[name] = value
self.putenv(name, value)
def get(self, name): # can't call it "getenv"
value = self.getenv(name)
os.environ[name] = value
return value
I could always create a Environ instance, and store it away inside
the EnvSync instance (i.e., EnvSync contains a Environ, but isn't
a subclass of one). It's also probably better to extend the
functions with a module, instead of a class (there's never more
than 1 instance). But I thought this would have worked as coded.
A bug? A feature? A generally silly question?
Mark L.
Guido van Rossum
> book :-), and I came across __getattr__ behavior that seems
> strange. Apologees in advance if this has been fixed: I tested
> this on 1.1.1 on Linux (don't have 1.2 access at the moment).
>
> I'm trying to wrap a C module in a class, so I can specialize
> some of its methods. If I define a class with 1 method per
> module function, things work as advertised:
[working example deleted]
> But if I use __getattr__ in the superclass, instead of real
> methods:
>
> class Environ:
> def __getattr__(self, name): # proxy class
> return getattr(environ, name) # pass to C module
Hmm, taken literally, this would look for something in module environ
called name; that's probably not what you want. However I suspect
this has nothing to do with the problem you are looking at.
> it doesn't work (I get an Attribute error in the subclass). Here's
> the only version that seemed to work with __getattr__ in the Environ
> superclass; I had to use different names for the subclass methods:
>
> class EnvSync(Environ): # can't call it "putenv"
> def put(self, name, value):
> os.environ[name] = value
> self.putenv(name, value)
>
> def get(self, name): # can't call it "getenv"
> value = self.getenv(name)
> os.environ[name] = value
> return value
If you give a method in the derived class the same name as a method in
the base class, you are overriding the base class method. All
references to "self.putenv" will get the derived class's putenv --
since self is a member of the derived class. What you want to do in
the derived class if you want to call the method that you are
overriding is this:
class EnvSync(Environ):
def putenv(self, name, value):
os.environ[name] = value
Environ.putenv(self, name, value)
Note that you have to name the class explicitly ("Environ.putenv") and
you have to pass "self" explicitly ("(self, name, value)").
This is one of the pains we have to endure for not having a "super"
keyword... Note that C++ doesn't either; it uses a similar solution
but doesn't require you to pass an explicit self since in C++ self
("this") is always implicit.
--Guido van Rossum <gu...@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
Mark Lutz
Well, I did some digging; at least in 1.1.1, __getattr__ dispatch
logic is only implemented for instance objects, not class objects.
I.e., Python only tries __getattr__ as a last resort when an instance
is qualified, not a class [instance_getattr() in Objects/classobject.c].
Should this work for classes too? Not sure what this could break,
but it looks like a simple change to class_getattr() (?)
Mark L.
Mark Lutz
problem well enough:
> > class Environ:
> > def __getattr__(self, name): # proxy class
> > return getattr(environ, name) # pass to C module
>
> Hmm, taken literally, this would look for something in module environ
> called name; that's probably not what you want. However I suspect
> this has nothing to do with the problem you are looking at.
It's what I want (at least in this example). When I do something like:
Env.getenv("USER")
'name' should be the string "getenv" if it worked as planned-- I wanted
to map an instance member to a module function called "getenv" too.
> If you give a method in the derived class the same name as a method in
> the base class, you are overriding the base class method. All
> references to "self.putenv" will get the derived class's putenv --
> since self is a member of the derived class. What you want to do in
> the derived class if you want to call the method that you are
> overriding is this:
>
> class EnvSync(Environ):
> def putenv(self, name, value):
> os.environ[name] = value
> Environ.putenv(self, name, value)
>
> Note that you have to name the class explicitly ("Environ.putenv") and
> you have to pass "self" explicitly ("(self, name, value)").
But this is exactly what I did to discover the problem (if it is one).
Calling the superclass method explicitly like this trigerred the
AttributeError.
What I was trying (and failed!) to explain, was that replacing the
'Environ' superclass that has explicit methods, with the version that
uses __getattr__ didn't work. The second version of the subclass:
class EnvSync(Environ): # can't call it "putenv"
def put(self, name, value):
os.environ[name] = value
self.putenv(name, value)
was the only version that worked, because it didn't rely on calling
a superclass method through the superclass name directly (say that
3 times fast!... :-). But it's not really specialization. Here's
the complete failing module:
import os, environ # get C module
class Environ:
def __getattr__(self, name): # proxy class
return getattr(environ, name) # pass to C module
class EnvSync(Environ): # extend by subclassing
def putenv(self, var, value):
os.environ[var] = value
Environ.putenv(self, var, value) # do superclass putenv
def getenv(self, var):
value = Environ.getenv(self, var) # do superclass getenv
os.environ[var] = value # integrity check
return value
Env = EnvSync() # make one instance
And changing the superclass to this makes it work:
class Environ:
def getenv(self, var): # C module wrapper class
return environ.getenv(var) # delegate to C module
def putenv(self, var, value):
return environ.putenv(var, value)
Hope this is a bit clearer. The problem (see my second mail) is that
__getattr__ isn't tried when qualifying a class object (only an instance).
I thought that asking a superclass for a member that doesn't exist at the
superclass and above (but does in 'self') would trigger the superclass's
__getattr__.
Whew; guess this example is a bit too complex for the introduction! :-)
Mark L.
Guido van Rossum
class from a module. Realizing that a base class must be a class
itself, he creates a a fake base class that has no ordinary methods
but has a __getattr__ discipline that returns the corresponding
function from the module. However he now gets in trouble when he
wants to extend one of the fake methods in a derived class, since the
__getattr__ only works for instances, not for classes, and you need to
get the unbound method from the base class in order to call it in the
method of the derived class that overrides it. (If you think I should
explain this using examples, consider this -- Mark showed examples and
I completely misunderstood what he was trying to do. So this is just
me taking revenge :-)
The solution, of course, is to create a base class that contains
actual methods. One way would be to give up on automating, and just
write:
import environ
class Env:
def getenv(self, name): return environ.getenv(name)
def putenv(self, name, value): environ.outenv(name, value)
However, with a little help from the parser we can do better. How
about the following:
import environ
prototype = "def %s(self, *args): return apply(environ.%s, args)"
class Env:
for name in dir(environ):
object = getattr(environ, name)
if callable(object):
exec prototype % (name, name)
del name, object
(The only reason I made the prototype string into a separate variable
is that it didn't fit on the line :-)
Note the use of callable() to avoid creating functions from data that
may be present in the environ module. You may want to do something
else with data other than ignore it, e.g. just stick it in the Env
module, or create a __getattr__ hook to retrieve it (which may be
better if it's not a constant).
Note the use of *args and apply() in the prototype to overcome the
problem that you don't know the argument count of the called function.
If you need to do this for many modules, you can make a subroutine out
of this, but since it should affect the class definition's local
variables you would have to pass the dictionary of local variables
into it -- use the vars() built-in function.
I don't think it is possible to do this without using "exec", as you
have to create a real function for each function in the environ
module.