extending methods ?
Arnd Baecker
the summary of my question is:
is there a way to append commands to a method inherited from
another class?
More verbose: Assume the following situation that I have
a class class_A with several methods.
In each of these methods quite a few computations take place
and several variables are defined.
Now I would like to extend this class to a class_B.
However, instead of overriding a method completely,
I would like to extend the method by basically adding
further commands at the end.
Eg., the code could look like:
class class_A:
def method1(self,x):
y=5*x+1 # create a variable
class class_B(class_A):
appendto method1(self,x):
print y # use variable defined in class_A
I.e. Effictively class_B should "look" like:
class class_B(class_A):
def method1(self,x):
y=5*x+1 # create a variable
print y # use the variable
Instead of this, one could in principle
store all variables defined in
method1 of class_A, eg. with self.y=y
and call in class_B the method1 of class_A.
However, with several variables and several methods
the code will not look nice.
(also efficiency is a bit of concern - presumably it shouldn't -
to me here as in my application the methods
are used to plot dots, circles, triangles etc.
so that speed is really important)
So is there a way to extend methods in the above sense
or is there a better (more pythonic ? ;-) approach to what I want?
Many thanks in advance.
Arnd
P.S.: Sorry if I messed up things on the OO side
(I am a real new-comer to this ...;-).
Alex Martelli
> Hi,
>
> the summary of my question is:
> is there a way to append commands to a method inherited from
> another class?
The canonical way to achieve that is to override the method,
and, within the override, call up to the superclass's
implementation.
> More verbose: Assume the following situation that I have
> a class class_A with several methods.
> In each of these methods quite a few computations take place
> and several variables are defined.
Ah, no way, then -- local variables of a method (just like
any other function) are totally internal to that method, and
there is absolutely no reasonable way to access them from
the outside.
> However, with several variables and several methods
> the code will not look nice.
A matter of opinion. I find it much nicer to know that a
local variable is local, and an instance variable is not.
> (also efficiency is a bit of concern - presumably it shouldn't -
Right, it shouldn't. Local variables are fast BECAUSE they
are local. If they were non-local -- as you require, in order
to work with them from outside the defining method -- then
they wouldn't be fast.
> to me here as in my application the methods
> are used to plot dots, circles, triangles etc.
> so that speed is really important)
I strongly doubt that the loss of speed due to making some
needed variable non-local is going to be measurable. Taking
your example...:
[alex@lancelot python2.3]$ python timeit.py \
> -s 'class A:' \
> -s ' def method1(self, x): y = 5 * x + 1' \
> -s ' def method2(self, x): self.y = 5 * x + 1' \
> -s 'a=A()' \
> 'a.method1(23)'
1000000 loops, best of 3: 1.47 usec per loop
and the same with method2 instead: 1.66 usec per loop.
What makes you think that a "slow-down" of 190 nanoseconds,
i.e. about 13% on this method call, is gonna be SO crucial?
And if you ARE working in an environment where every single
nanosecond matter, then what about...:
[alex@lancelot python2.3]$ python timeit.py \
> -s 'class A:' \
> -s ' def method1(self, x): y = 5 * x + 1' \
> -s ' def method2(self, x): self.y = 5 * x + 1' \
> -s 'a=A()' \
- -s 'meth=a.method2' \
> 'meth(23)'
1000000 loops, best of 3: 1.25 usec per loop
See? Extracting the bound method once and for all saves
you a whopping *410* nanoseconds per call, compensating
the 190 "lost" by making instance variables where you need
them *and then some*. In other words, there are ways and
means to squeeze nanoseconds out of some Python bottleneck
that do not necessarily require using local variables where
you need instance or other non-local ones!
> So is there a way to extend methods in the above sense
> or is there a better (more pythonic ? ;-) approach to what I want?
There is no pythonic way to treat local variables as non-local.
(There MAY be horrible bytecode hacks exploiting hooks that are
meant for debugging only -- but THOSE *WILL* slow you down
SERIOUSLY...).
Alex
Arnd Baecker
On Wed, 1 Oct 2003, Alex Martelli wrote:
> > More verbose: Assume the following situation that I have
> > a class class_A with several methods.
> > In each of these methods quite a few computations take place
> > and several variables are defined.
>
> Ah, no way, then -- local variables of a method (just like
> any other function) are totally internal to that method, and
> there is absolutely no reasonable way to access them from
> the outside.
>
> > However, with several variables and several methods
> > the code will not look nice.
>
> A matter of opinion. I find it much nicer to know that a
> local variable is local, and an instance variable is not.
I absolutely agree. And that's why I wrote the (non-existing)
"appendto" in class_B,
class class_B(class_A):
appendto method1(self,x):
print y # use the variable defined in class_A
The idea was that
everything from method1 in class_A is put into method1 in class_B
before the "print y", like a copy-and-paste operation.
Now, why could something like this be useful:
I have a situtation of a class with several methods
and now I would like to extend this class by
adding a few lines to each of the methods.
These further lines would make use of the variables defined
in the original methods.
An extension in this way would not require to
change anything in method1 of class_A
and any changes made in class_A carry over directly to
class_B.
(In a sense one could see this "copy" similar to a
"pre-processor" step ...).
Instead of the above (which does not exist, and I'd guess never will ;-)
one could
a) Invoke the superclass' method in which all the
relevant variables needed for the method in class_B are defined
as self.<variable_name>. And then they can be
accessed in class_B.
b) Take a verbatim copy of class_A
and just add the additional lines at the end
of the corresponding methods.
The drawback here is that any code changes
in the common parts of the methods in class_A and class_B
have to be applied in both parts.
So for b) it is the code duplication which I don't like
and for a) I don't like too much that one has to make
all the variables accessible self.<variable_name>=....
This adds a bunch of code which is only used when class_A
is subclassed.
(Of course there is still option c) to write a small
python programm which takes class_A and does a "copy-and-paste"
operation at the corresponding places in class_B.)
Personally I tend to a) or c) ;-)
But maybe there is a better way of doing the above?
[ ... speed part snipped - many thanks I learned a lot!]
> > So is there a way to extend methods in the above sense
> > or is there a better (more pythonic ? ;-) approach to what I want?
>
> There is no pythonic way to treat local variables as non-local.
I think that this is not really what I want/meant,
as I thought more of the above internal "copy"
to avoid code duplication.
> (There MAY be horrible bytecode hacks exploiting hooks that are
> meant for debugging only -- but THOSE *WILL* slow you down
> SERIOUSLY...).
Oh no I am definitively not going along that route
- that's a different league ;-)
Many thanks,
Arnd
David C. Fox
If you really want to do this, you are probably better off doing it
explicitly. For example
class A:
def store_locals(self, local_variables, keep_locals):
if keep_locals is None:
return
keep_locals.update(local_variables)
def m(self, x, keep_locals = None):
"""If you want to keep the local variables of this method,
specify a dictionary for keep_locals
"""
y = 5*x+1
self.store_locals(locals(), keep_locals)
class C(A):
def m(self, x, keep_locals = None):
if keep_locals is None:
keep_locals = {}
v = A.m(self, x, keep_locals = keep_locals)
print 'y was ', keep_locals['y']
return v
> P.S.: Sorry if I messed up things on the OO side
> (I am a real new-comer to this ...;-).
Perhaps if you tell us more about why you want to do this, we could come
up with a better OO solution.
David
Alex Martelli
...
>> A matter of opinion. I find it much nicer to know that a
>> local variable is local, and an instance variable is not.
>
> I absolutely agree. And that's why I wrote the (non-existing)
Let me put it another way: I find it much nicer that LOCAL
variables are a LOCAL issue of the function using them, and
NO other function can possibly ever know anything about them
(excepting debuggers & the like). For your idea to work,
local variables would have to NOT be LOCAL issues -- they
would have to be somehow known OUTSIDE the function.
> Now, why could something like this be useful:
> I have a situtation of a class with several methods
> and now I would like to extend this class by
> adding a few lines to each of the methods.
Think of it another way: a function is a black box. Make
believe that you DO NOT know the exact source code with
which the function is currently implemented, e.g. such
details as what names are internally, LOCALLY used for its
LOCAL variables. Is your desired approach even CONCEIVABLE
in such a worldview? It appears to me that it isn't; and
thus, that you desire a worldview where such boundaries as
"inside" vs "outside" a function cannot really exist. I
strongly believe that removing such boundaries would be an
utter disaster, leading to totally unmaintainable code; i.e.,
that such boundaries are STRONGLY desirable.
> These further lines would make use of the variables defined
> in the original methods.
> An extension in this way would not require to
> change anything in method1 of class_A
> and any changes made in class_A carry over directly to
> class_B.
> (In a sense one could see this "copy" similar to a
> "pre-processor" step ...).
Good analogy. The preprocessor of C is a good contender for
its single most error-prone feature; C++ has been fighting
long and hard to wipe it out (though for backwards compatibility
reasons it will never entirely succeed). Reason #1 for the
preprocessor being such a health hazard is that it's no
respecter of SCOPE -- it doesn't know nor care what is LOCAL
to WHERE.
You are claiming that "any changes made in class_A carry over
directly" -- but you seem to miss that, in your proposal,
changes that are normally totally innocuous, because they
are guaranteed to be isolated from the outside, would instead
"carry over directly" *AND DESTRUCTIVELY* to subclasses that
made use of your innovation. I.e., the base class would
suddenly become constrained in such intimate implementation
details as *THE NAME OF LOCAL VARIABLES* (not to mention
many other CURRENTLY-LOCAL implementation details of methods).
***The base class would become totally dependent, down to
minute details of its implementation, on which of those
details happen to be exploited by its subclasses***. Good
bye, therefore, to the "open/closed principle" (google it up)
and with it to just about every substantial advantage of
object-oriented programming. *SHUDDER*.
> Instead of the above (which does not exist, and I'd guess never will ;-)
I earnestly hope and pray and trust it never will.
> one could
> a) Invoke the superclass' method in which all the
> relevant variables needed for the method in class_B are defined
> as self.<variable_name>. And then they can be
> accessed in class_B.
Of course! If the superclass CHOOSES, as part of its interface, to
expose certain variables as instance attributes, then it's PERFECTLY
SENSIBLE to access them in subclasses.
> b) Take a verbatim copy of class_A
> and just add the additional lines at the end
> of the corresponding methods.
Such "copy and paste reuse" is generally a disaster waiting to
happen -- albeit in a longer time horizon than your proposal would
be -- because modifications to the original do not "propagate" once
a copy is made, so functionality may "drift" among copies (and it
generally does). Such a "solution" should only be considered if
the base class was badly designed for the subclass's needs AND if
it was a worse choice to modify (redesign) the base class -- for
example, the base class comes from somebody else's package, and
does not offer sufficient granularity of customizability for your
current needs. Then, you do need to entirely override certain
methods, reimplementing them from scratch -- one strategy for such
reimplementation is, in fact, to start with A's current choices
of implementations, and edit them as appropriate. But once you
are having to reimplement anyway you might be better advised to
consider, as long as you're at it, implementing strategy [A] --
redo the base class ONCE by supplying it with the amount of
flexibility (and granularity within the flexibility) that you do
now in fact know you need; "actual" subclasses can then all be
coded sensibly.
> The drawback here is that any code changes
> in the common parts of the methods in class_A and class_B
> have to be applied in both parts.
Yes, that is indeed the downside of "copy and paste reuse".
> So for b) it is the code duplication which I don't like
> and for a) I don't like too much that one has to make
> all the variables accessible self.<variable_name>=....
> This adds a bunch of code which is only used when class_A
> is subclassed.
What "bunch of code"?! Accessing, e.g., self.x is exactly
as much code as accessing a bare x would be -- one expression
or statement, direct and immediate 1:1 correspondence.
> (Of course there is still option c) to write a small
> python programm which takes class_A and does a "copy-and-paste"
> operation at the corresponding places in class_B.)
>
> Personally I tend to a) or c) ;-)
> But maybe there is a better way of doing the above?
Designing your classes AS CLASSES rather than as bunches of
vaguely related snippets of source would undoubtedly be a
far better way to proceed. But if you keep presenting a
problem as you do, I can't give any suggestions on how the
design should be. Try this thoughtexperiment: imagine your
fairy godmother put a spell on you making it IMPOSSIBLE for
you to SEE the *sources* of the base class. You can ONLY see
the INTERFACE of the base class -- you have complete knowledge
of WHAT each method does (in particular, what instance variables
it uses and/or affects), but NO knowledge whatsoever of HOW it
does it, internally. Would your ACTUAL problem then become
totally insoluble? I very much doubt it. So, CAN you perhaps
present your ACTUAL problem -- WITHOUT any reference to the
INTERNAL source details of the existing baseclass...???
That you CAN in fact see how the sources "happen" to be may
be blinding you to the actual problem. "Information hiding"
was invented for a good reason -- sometimes, seeing more info
than you actually SHOULD see makes your problem HARDER to
express and solve effectively. So, learning to selectively
"UN-SEE" some of the information you "happen" to have is an
important trick to learn.
> [ ... speed part snipped - many thanks I learned a lot!]
You're welcome. But speed is hardly ever the overriding issue
in programming (Knuth guesstimated that we should ignore
micro-optimizations "maybe 97% of the time", if I recall
correctly -- and in today's application programming it may
be even more often...:-).
Alex
David Eppstein
"David C. Fox" <davi...@post.harvard.edu> wrote:
> Perhaps if you tell us more about why you want to do this, we could come
> up with a better OO solution.
Actually this seems exactly the sort of thing for which aspect oriented
programming was developed.
http://www.google.com/search?q=aspect-oriented+python
seems to find a few relevant pages...
--
David Eppstein http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science
Robert Brewer
> a) Invoke the superclass' method in which all the
> relevant variables needed for the method in class_B are defined
> as self.<variable_name>. And then they can be
> accessed in class_B.
There is a fourth option:
class A:
def __init__(self, x):
y = x + 42
self.postinit(y)
def postinit(self, y):
pass
class B(A):
def postinit(self, y):
dosomethingspecialwith(y)
class C(A):
def postinit(self, y):
dosomethingelsewith(y)
This is similar to a) but doesn't "invoke a bunch of code" if you don't
want it to. You don't even have to define postinit() in class A if you
don't want to; you could just skip the call if your (sub)class doesn't
have that method/attribute. It breaks the isolation of A a bit, but if
that's an integral part of your design, I won't take issue with it. ;)
Robert Brewer
MIS
Amor Ministries
fuma...@amor.org
Stephen Horne
<arnd.b...@web.de> wrote:
>the summary of my question is:
>is there a way to append commands to a method inherited from
>another class?
If you can call...
instance.method (arg1, ...)
then you can equivalently call...
classname.method (instance, arg1, ...)
You can use this in your subclasses method to call the method from a
specified base class (with multiple inheritance there may be more than
one base). For example...
>>> class A (object) :
... def dostuff (self) :
... print "running A.dostuff"
...
>>> class B (A) :
... def dostuff (self) :
... print "running B.dostuff"
... A.dostuff (self)
...
>>> b=B()
>>> b.dostuff()
running B.dostuff
running A.dostuff
--
Steve Horne
steve at ninereeds dot fsnet dot co dot uk
Arnd Baecker
sorry for causing you to
> *SHUDDER*
and many thanks for your detailed and clear remarks
(as I wrote I am a newbie wrt OO-stuff).
Now I better lean back and re-think
the overall design with all the input/suggestions
from you and the other posters in mind.
Many thanks,
Arnd
Carlo v. Dango
news:eppstein-A4BD3C...@news.service.uci.edu:
> In article <HyBeb.645796$YN5.495743@sccrnsc01>,
> "David C. Fox" <davi...@post.harvard.edu> wrote:
>
>> Perhaps if you tell us more about why you want to do this, we could
>> come up with a better OO solution.
>
> Actually this seems exactly the sort of thing for which aspect
> oriented programming was developed.
excuse me, but that is utterly and completely rubbish! :-) Specializing
methods is actually very useful, but I've only seen it in the language
Beta (well I'm certain it could be achieved in Simula 67 as well). In
simula classes and methods has been unified into the notion of a
"pattern". So since you can specialize classes, you now also can
specialize methods using this pattern construct.
AOP was designed to be able to define CROSS-CUTTING entities crosscutting
at least 2 inheritance hierachies.
You should all at least try to read up on the Beta language and grasp its
powerful idea of the pattern construct!
-carlo van dango
Carlo v. Dango
news:mailman.106508196...@python.org:
> Now I better lean back and re-think
> the overall design with all the input/suggestions
> from you and the other posters in mind.
As I have just posted to this thread, specializing a method is a natural
thing... at least in other languages than the mainstream (java, c++, python
etc etc..).. this is important to understand. python has only method
overwritting, not method specialization. Look into the Beta language.. its
mind buggling (and has method specialization :).. Another peculiar thing in
Beta is that it has no "super" construct, but an "inner" which works the
other way around! yes! In Beta it's the LEAST specific method (pattern
actually) which is called FIRST.. it then calls "inner" to activate more
specialized methods. If the "inner" construct is forgotten its equivalent
with final methods in e.g. Java.