aspect-oriented demo using metaclasses
Mark McEahern
morning, I trudged up the learning curve a little further than I have before
and I wanted to share what I discovered. I've been intrigued by the idea of
using metaclasses as a way to implement aspect-oriented programming (AOP) in
Python. To me, the promise of AOP is you should ideally write your classes
in complete ignorance of the aspects that you want to plug in later.
Tracing is probably the simplest example of an aspect I can think of.
Persistence is a more complicated aspect (to me, anyway). So I figured I'd
start with tracing.
I owe much to previous threads on AOP, in particular Pedro Rodriguez's
non-metaclass implementation:
http://groups.google.com/groups?selm=pan.2002.01.13.11.22.54.248401.9857%40c
lub-internet.fr
This is written in the style of a tutorial, but admittedly, it's not very
polished. I would appreciate any and all comments, criticism, feedback,
suggestions, amplifications, discussion, etc.
**
A simple AOP framework in Python using metaclasses
Let's start with a simple class:
<code>
class foo:
def bar(self, *args, **kwargs):
pass
f = foo()
f.bar()
</code>
As expected, this doesn't do anything. Suppose I want to be notified
before and after the bar method is called. (The wording of that is
purposefully vague.) And I don't want to have to modify the foo class
at all in order to do this. I want to intervene in the creation of
this class (not just instances of the class) and insert pre and post
methods for each function.
Let's start with a simple metaclass definition that doesn't do
anything:
<code>
class aspect(type):
def __init__(cls, name, bases, dict):
super(aspect, cls).__init__(name, bases, dict)
__metaclass__ = aspect
class foo:
def bar(self, *args, **kwargs):
pass
f = foo()
f.bar()
</code>
Again, this doesn't have any noticeable effect. Let's insert some
print statements to see what's happening though:
<code>
class aspect(type):
def __init__(cls, name, bases, dict):
super(aspect, cls).__init__(name, bases, dict)
for k,v in dict.items():
print "type(%s) --> %s" % (k, type(v))
__metaclass__ = aspect
print "before class foo:"
class foo:
def bar(self, *args, **kwargs):
pass
print "before f = foo()"
f = foo()
f.bar()
</code>
<output>
before class foo:
type(__module__) --> <type 'str'>
type(bar) --> <type 'function'>
before f = foo()
</output>
This shows us that the aspect metaclass' __init__ gets called as a
result of declaring our foo class, before any instance's of foo are
created.
We want to iterate through the class's dict and do something for each
method (I'm not worried about distinguishing staticmethod and
classmethod type methods for now):
<code>
class Aspect(type):
def __init__(cls, name, bases, dict):
super(Aspect, cls).__init__(name, bases, dict)
for k,v in dict.items():
if type(v) == type(lambda x:x):
print "%s is a function." % k
__metaclass__ = Aspect
class foo:
def bar(self, *args, **kwargs):
print "real bar(%s, %s)" % (args, kwargs)
f = foo()
f.bar("a", "b", foo="c")
</code>
<output>
bar is a function.
real bar(('a', 'b'), {'foo': 'c'})
</output>
Rather than importing types and using types.FunctionType, I just
compare the type of the item to the type of an anonymous function (via
lambda). Is that cool or what? ;-)
Also, notice that I replaced pass in the body of bar() with a simple
print statement that tells me how the real bar() is being called.
That will be useful later.
For each method (or function), I want to wrap that method so that I
get notified when it's called. So I created a wrapped_method class
that is somewhat (exactly?) like a field descriptor (e.g., property):
<code>
class wrapped_method(object):
def __init__(self, cls, method):
self.class_name = cls.__name__
self.method = method
self.method_name = method.func_name
self.observers = []
def __call__(self, *args, **kwargs):
self.before(*args, **kwargs)
self.method(self, *args, **kwargs)
self.after(*args, **kwargs)
def before(self, *args, **kwargs):
print "before %s.%s(%s, %s)" % (self.class_name, self.method_name,
args, kwargs)
def after(self, *args, **kwargs):
print "after %s.%s(%s, %s)" % (self.class_name, self.method_name,
args, kwargs)
class Aspect(type):
def __init__(cls, name, bases, dict):
super(Aspect, cls).__init__(name, bases, dict)
for k,v in dict.items():
if type(v) == type(lambda x:x):
setattr(cls, k, wrapped_method(cls, v))
__metaclass__ = Aspect
class foo:
def bar(self, *args, **kwargs):
print "real bar(%s, %s)" % (args, kwargs)
f = foo()
f.bar("a", "b", foo="c")
</code>
<output>
before foo.bar(('a', 'b'), {'foo': 'c'})
real bar(('a', 'b'), {'foo': 'c'})
after foo.bar(('a', 'b'), {'foo': 'c'})
</output>
Now we're starting to get somewhere! The next step is to create a
framework where multiple observers can plug into the before and after
events for any given method call. Rather than taking more baby steps
to get there, this is the complete implementation I have so far:
<code>
import sys
class wrapped_method(object):
def __init__(self, cls, method):
self.class_name = cls.__name__
self.method = method
self.method_name = method.func_name
self.observers = []
def __call__(self, *args, **kwargs):
self.before(*args, **kwargs)
self.method(self, *args, **kwargs)
self.after(*args, **kwargs)
def before(self, *args, **kwargs):
for o in self.observers:
o.before(self.class_name, self.method_name, *args, **kwargs)
def after(self, *args, **kwargs):
for o in self.observers:
o.after(self.class_name, self.method_name, *args, **kwargs)
def notify_me(self, observer):
self.observers.append(observer)
class VirtualClassError(Exception):pass
class Observer:
def __init__(self, *args, **kwargs):
if self.__class__ is Observer:
raise VirtualClassError(self.__class__.__name__)
def before(self, class_name, method_name, *args, **kwargs):
pass
def after(self, class_name, method_name, *args, **kwargs):
pass
class Trace(Observer):
def __init__(self, filename=None):
self.filename = filename
def write(self, prefix, class_name, method_name, *args, **kwargs):
cls = class_name
s = "%s: %s.%s(%s, %s)" % (prefix, cls, method_name, args, kwargs)
if not self.filename:
f = sys.stdout
else:
f = file(self.filename)
f.write(s)
f.write("\n")
if self.filename:
f.close()
def before(self, class_name, method_name, *args, **kwargs):
self.write("before", class_name, method_name, *args, **kwargs)
def after(self, class_name, method_name, *args, **kwargs):
self.write("after", class_name, method_name, *args, **kwargs)
class Aspect(type):
def __init__(cls, name, bases, dict):
super(Aspect, cls).__init__(name, bases, dict)
for k,v in dict.items():
if type(v) == type(lambda x:x):
setattr(cls, k, wrapped_method(cls, v))
__metaclass__ = Aspect
class foo:
def bar(self, *args, **kwargs):
print "real bar(%s, %s)" % (args, kwargs)
def main():
foo.bar.notify_me(Trace())
f = foo()
f.bar("a", "b", foo="c")
if __name__ == "__main__":
main()
</code>
<output>
before: foo.bar(('a', 'b'), {'foo': 'c'})
real bar(('a', 'b'), {'foo': 'c'})
after: foo.bar(('a', 'b'), {'foo': 'c'})
</output>
The output is the same, but it's actually being generated by an
instance of the Trace class.
************
Observations
************
Obviously, Trace is designed so that I can specify a filename when
creating an instance of it and that would use that file rather than
sys.stdout.
Observers should probably be able to register for particular events
(i.e., just before). They should be able to unregister. It should
also be possible to register for some classes, some methods, and not
others. What is a good way to write the rules for enrolling
in notifications? (And what's the word for this that AspectJ uses?)
I explicitly pass self from the wrapped_method instance to the actual
wrapped method. However, this is not really the instance of the
class, it's the instance of the wrapped_method object. How do I pass
the actual instance to the wrapped method--whether explicitly or
implicitly?
I should probably explicitly set the metaclass for wrapped_method,
Trace, Observer, and even VirtualClassError to "type" (the default
metaclass) to avoid weird infinite loops where the members of the
aspect framework are themselves being aspected.
I need to think more about how this would be used. My vague
impression of AspectJ is that you use it from an IDE, specifying
different conditions for filtering classes and methods (what's the
word for that again?). When you compile with aspect support turned
on, the generated code has the aspects weaved in. What are the
equivalent developer use cases for weaving aspects into Python code?
The whole point it seems to me is that when I author the classes to be
aspected, I shouldn't have to do a thing--no subclassing, no method
definition. I just write my classes, omitting entirely any
consideration of the future aspects to be weaved in. I mean, that's
the whole point, right? To simplify the code.
In the example, foo doesn't know diddly about the fact that it's being
traced. And we could turn that tracing on or off simply by changing
the module-level __metaclass__ variable.
I should probably and try:...except:pass to the framework so that it
doesn't generate errors. During development of the framework itself,
of course, I want to know about errors.
Observer is an interface-type base class. It doesn't provide
implementation. It could easily be omitted, but it servers to
document what Trace and other potential observers need to look like.
It would be easy to add aspects for error handling. This is a very
quick and dirty example for error notification:
In wrapped_method, I would change __call__ like this:
def __call__(self, *args, **kwargs):
self.before(*args, **kwargs)
try:
self.method(self, *args, **kwargs)
except Exception, e:
self.on_error(e, *args, **kwargs)
raise
self.after(*args, **kwargs) Rather than using sys.stdout, I could
have specified a file.
and add on on_error() method to wrapped_method:
def on_error(self, error, *args, **kwargs):
for o in self.observers:
o.on_error(error, self.class_name, self.method_name,
*args, **kwargs)
Observer:
def on_error(self, error, class_name, method_name, *args, **kwargs):
pass
and Trace:
def on_error(self, error, class_name, method_name, *args, **kwargs):
self.write("error %s" % error, class_name, method_name, *args,
**kwargs)
-
Mark McEahern
> What is a good way to write the rules for enrolling
> in notifications? (And what's the word for this that AspectJ uses?)
The phrase I was look for was join points.
// mark
-
Pedro RODRIGUEZ
<snip>
> I owe much to previous threads on AOP, in particular Pedro Rodriguez's
> non-metaclass implementation:
>
Mark,
If you really want something more consistent than my posting, I will
recommend you to also check:
- Pythius by Juergen Hermann
where Frank J. Tobin has introduced an aop module
http://pythius.sourceforge.net/ by Juergen Hermann
- Transwarp by Phillip J. Eby
this a more complex framework using in some way concepts of aop
http://telecommunity.com/TransWarp/
http://www.zope.org/Members/pje/Wikis/TransWarp/HomePage
<snipping introduction on metaclass>
>
> We want to iterate through the class's dict and do something for each
> method (I'm not worried about distinguishing staticmethod and
> classmethod type methods for now):
>
You should <wink>. At least for the purpose of the exercise ;)
(Honestly I didn't try either, it may end up with a two liner...
but I wonder if they will take two minutes or two hours to be written ;)
<sniping>
> Rather than importing types and using types.FunctionType, I just compare
> the type of the item to the type of an anonymous function (via lambda).
> Is that cool or what? ;-)
>
Even if deprecation of the types module have been discussed on the devel
list, I think it is preferable.
<sniping>
> Now we're starting to get somewhere! The next step is to create a
> framework where multiple observers can plug into the before and after
> events for any given method call. Rather than taking more baby steps to
> get there, this is the complete implementation I have so far:
>
You went even further than that. In the next step, you start separating
(at least ;) the aspect and the 'aspectified' object in a less intrusive
way.
> <code>
> import sys
>
> class wrapped_method(object):
>
> def __init__(self, cls, method):
> self.class_name = cls.__name__
> self.method = method
> self.method_name = method.func_name
> self.observers = []
>
> def __call__(self, *args, **kwargs):
> self.before(*args, **kwargs)
> self.method(self, *args, **kwargs)
> self.after(*args, **kwargs)
>
Oops... Didn't I make the same mistake...
... forgetting about the returned value ;)
Something like could be better:
def __call__(self, *args, **kwargs):
self.before(*args, **kwargs)
ret = self.method(self, *args, **kwargs)
self.after(*args, **kwargs)
return ret
<sniping>
> class Trace(Observer):
>
> def __init__(self, filename=None):
> self.filename = filename
>
> def write(self, prefix, class_name, method_name, *args, **kwargs):
> cls = class_name
> s = "%s: %s.%s(%s, %s)" % (prefix, cls, method_name, args,
> kwargs) if not self.filename:
> f = sys.stdout
> else:
> f = file(self.filename)
> f.write(s)
> f.write("\n")
> if self.filename:
> f.close()
>
> def before(self, class_name, method_name, *args, **kwargs):
> self.write("before", class_name, method_name, *args, **kwargs)
>
> def after(self, class_name, method_name, *args, **kwargs):
> self.write("after", class_name, method_name, *args, **kwargs)
>
This Trace class is an aspect.
> class Aspect(type):
>
> def __init__(cls, name, bases, dict):
> super(Aspect, cls).__init__(name, bases, dict) for k,v in
> dict.items():
> if type(v) == type(lambda x:x):
> setattr(cls, k, wrapped_method(cls, v))
>
This is not an Aspect. This is your way to implement method call interception.
<sniping>
> ************
> Observations
> ************
>
> Obviously, Trace is designed so that I can specify a filename when
> creating an instance of it and that would use that file rather than
> sys.stdout.
>
> Observers should probably be able to register for particular events
> (i.e., just before). They should be able to unregister. It should also
> be possible to register for some classes, some methods, and not others.
What about module functions ? Bounded methods ?
Using metaclass for call interception is too restrictive I believe.
> What is a good way to write the rules for enrolling in notifications?
> (And what's the word for this that AspectJ uses?)
Joinpoint... but you already found it ;)
>
> I explicitly pass self from the wrapped_method instance to the actual
> wrapped method. However, this is not really the instance of the class,
> it's the instance of the wrapped_method object. How do I pass the
> actual instance to the wrapped method--whether explicitly or implicitly?
>
This is the trickier part. When you do :
setattr(cls, k, wrapped_method(cls, v))
you substitute a function (actually an unbounded method) by a callable
object. Unfortunately when you'll invoke this object with a classical
method call, Python will not pass the instance as the first argument.
I only know two ways to substitute an unbounded method by an object
that will allow to retrieve the instance argument :
- providing a function, and take benifit of nested_scopes to retrieve
all the information from your context
- create an object through the 'new' package (new.instancemethod)
that was broken for new type objects in 2.2, and fixed in 2.2.1
So your wrapped_method class is not sufficient per se to achieve this
goal.
> I should probably explicitly set the metaclass for wrapped_method,
> Trace, Observer, and even VirtualClassError to "type" (the default
> metaclass) to avoid weird infinite loops where the members of the aspect
> framework are themselves being aspected.
>
I don't think that metaclass'ing is the good thing to do, but that's
just me.
> I need to think more about how this would be used. My vague impression
> of AspectJ is that you use it from an IDE, specifying different
> conditions for filtering classes and methods (what's the word for that
> again?). When you compile with aspect support turned on, the generated
> code has the aspects weaved in. What are the equivalent developer use
> cases for weaving aspects into Python code? The whole point it seems to
> me is that when I author the classes to be aspected, I shouldn't have to
> do a thing--no subclassing, no method definition. I just write my
> classes, omitting entirely any consideration of the future aspects to be
> weaved in. I mean, that's the whole point, right? To simplify the
> code.
>
Yep. And this is why I consider the metaclass thing for being too intrusive.
> In the example, foo doesn't know diddly about the fact that it's being
> traced. And we could turn that tracing on or off simply by changing the
> module-level __metaclass__ variable.
>
Too intrusive. I don't believe that you can do it dynamically, at least
not for classes defined at module level. They will be created with the
metaclass defined at compilation time.
<sniping>
> It would be easy to add aspects for error handling. This is a very
> quick and dirty example for error notification:
>
<snip>
Yes. Interception of raised exception is a good (and easy ;) feature.
Just try to go a step further with 'around' methods.
Thanks, for this posting Mark. Reminds me that aop is my task list for
the moment I will have some spare time (not so far I hope ;)
Just to give some hints on aop, this is what I wrote as a reminder
6 months ago along with a more complete implementation of aop :
Vocabulary
pointcut
an object identifying a set of joinpoints, and aggregating a set of
advices
joinpoint
a callable function in Python (unbound method, bound method, function)
advice
a function that will be called when a joinpoint is reached. It may be
triggered before the joinpoint is called, after, or when an exception
occurs.
An advice may also be set around a joinpoint, meaning that it will be
able to do some job before and after the joinpoint runs, but it will
have to call the joinpoint himself.
aspect
a class that contains poincuts and their related advices.
And a UML'ish model :
+-------------+ n +-------------+ 1 +-------------+
| PointCut | - - - -> | JoinPoint |-------->| Function +
+-------------+ +-------------+ +-------------+
| |
| |
V 1 |
+-------------+ n |
| AdviceStack |<-----------+
+-------------+
|
|
V n
+-------------+
| Advice |
+-------------+
- A pointcut uses several joinpoints
- Since a joinpoint may participate in several pointcuts, it will have
to trigger several set of advices (AdviceStack) related to each of the
pointcut it belongs to.
- A joinpoint is related to a Python function. It will intercept calls
to it.
+-------------+ 1 +-------------+
| JoinPoint |---------->| Function |
+-------------+ +-------------+
^ n
|
|
+-------------+
| CallableSet |
+-------------+
A
|
+-----------------+-------+---------+----------------+
| | | |
+--------------+ +-------------+ +-------------+ +-------------+
|ClassicalClass| | TypeClass | | Module | | Instance |
+--------------+ +-------------+ +-------------+ +-------------+
- A Python function related to a joinpoint belongs to a CallableSet.
- A CallableSet can be a classical python class, in which case the
function is an unbound method.
- A CallableSet can be a new python class, in which case the function
is an unbound method.
- A CallableSet can be a python instance, in which case the function
is a bound method.
- A CallableSet can be a python module, in which case the function
is a standard method.
Best regards,
Pedro
Mark McEahern
metaclasses:
This version fixes the problem where the wrapped method's self variable was
pointing to the field descriptor instance rather than the class instance.
I've also separated the functionality into three files:
aspect.py - the framework and the Observer interface definition
trace.py - the simple trace Observer
test.py - not yet a PyUnit test, just a demo for now
***
# aspect.py
#! /usr/bin/env python
__doc__ = \
"""
TODO:
[x] Separate into appropriate modules.
[ ] Should the framework raise errors?
[ ] Add unit tests.
[ ] Define scenarios for join points, point cutting, etc.:
Register for all events, all classes, all methods.
Register for some events, some classes, some methods. Etc.
What's most common?
[ ] Explicitly specify default metaclass for framework classes. Fixed by
separating aspect into separate module?
[ ] The data in the trace file is not valid xml because there is no
top-level
document. That may be ok, but document it.
"""
class VirtualClassError(Exception):pass
class Observer:
"""Interface for observers of Aspected classes."""
def __init__(self, *args, **kwargs):
if self.__class__ is Observer:
raise VirtualClassError(self.__class__.__name__)
def on_before(self, class_name, method_name, *args, **kwargs):
pass
def on_after(self, class_name, method_name, *args, **kwargs):
pass
def on_error(self, class_name, method_name, *args, **kwargs):
pass
def make_event(event_name, method_name):
def event(self, *args, **kwargs):
class_name = self.__class__.__name__
for o in self.observers[event_name]:
e = getattr(o, event_name)
e(class_name, method_name, *args, **kwargs)
return event
def make_wrapped_method(cls, original_name, method):
# TODO: Refactor.
on_before_event = make_event("on_before", original_name)
on_after_event = make_event("on_after", original_name)
on_error_event = make_event("on_error", original_name)
on_before_event_name = "on_before_%s" % original_name
on_error_event_name = "on_error_%s" % original_name
on_after_event_name = "on_after_%s" % original_name
setattr(cls, on_before_event_name, on_before_event)
setattr(cls, on_error_event_name, on_error_event)
setattr(cls, on_after_event_name, on_after_event)
on_before_event = getattr(cls, on_before_event_name)
on_error_event = getattr(cls, on_error_event_name)
on_after_event = getattr(cls, on_after_event_name)
def wrapped(self, *args, **kwargs):
on_before_event(self, *args, **kwargs)
try:
ret = method(self, *args, **kwargs)
except:
on_error_event(self, *args, **kwargs)
raise
on_after_event(self, *args, **kwargs)
return ret
return wrapped
class Aspected(type):
def __init__(cls, name, bases, dict):
super(Aspected, cls).__init__(name, bases, dict)
# Skip "private" methods for now.
skip_prefix = "__"
observers_dict = {'on_before': [],
'on_error': [],
'on_after': []}
setattr(cls, "observers", observers_dict)
for k, v in dict.items():
print type(v)
if not k.startswith(skip_prefix):
# TODO: This test will skip staticmethods and classmethods.
if type(v) == type(lambda x: x):
new_name = "__%s" % k
setattr(cls, new_name, v)
new_method = getattr(cls, new_name)
setattr(cls, k, make_wrapped_method(cls, k, new_method))
***
***
# trace.py
# --------------------------------------------------------------------------
---
# Standard library imports
# --------------------------------------------------------------------------
---
import sys
# --------------------------------------------------------------------------
---
# Third party imports
# --------------------------------------------------------------------------
---
import mx.DateTime
# --------------------------------------------------------------------------
---
# Sibling imports
# --------------------------------------------------------------------------
---
import aspect
# --------------------------------------------------------------------------
---
# Classes
# --------------------------------------------------------------------------
---
class Trace(aspect.Observer):
def __init__(self, filename=None):
self.filename = filename
def write(self, prefix, data, class_name, method_name, *args, **kwargs):
# TODO: Wrap import mx in try:except ImportError and use time?
now = mx.DateTime.now()
if not data:
data = ""
s = """<event name='%(prefix)s' datetime='%(now)s'>
<call>%(class_name)s.%(method_name)s(%(args)s, %(kwargs)s)</call>
<data>%(data)s</data>
</event>
""" % locals()
if not self.filename:
f = sys.stdout
else:
f = file(self.filename, "a")
f.write(s)
if self.filename:
f.close()
def on_before(self, class_name, method_name, *args, **kwargs):
prefix = "on_before"
data = None
self.write(prefix, data, class_name, method_name, *args, **kwargs)
def on_after(self, class_name, method_name, *args, **kwargs):
prefix = "on_after"
data = None
self.write(prefix, data, class_name, method_name, *args, **kwargs)
def on_error(self, class_name, method_name, *args, **kwargs):
prefix = "on_error"
data = get_traceback()
self.write(prefix, data, class_name, method_name, *args, **kwargs)
# --------------------------------------------------------------------------
---
# Helper methods
# --------------------------------------------------------------------------
---
def get_traceback():
"""get_traceback() --> string
Return the traceback exception information as a string.
"""
import traceback
from StringIO import StringIO
f = StringIO()
traceback.print_exc(limit=None, file=f)
s = f.getvalue()
return s
***
***
# test.py
#! /usr/bin/env python
# --------------------------------------------------------------------------
---
# Sibling imports
# --------------------------------------------------------------------------
---
import aspect
import trace
__metaclass__ = aspect.Aspected
# --------------------------------------------------------------------------
---
# Classes
# --------------------------------------------------------------------------
---
class MyClass:
def bar(self, *args, **kwargs):
print "real bar(%s, %s)" % (args, kwargs)
def raise_error(self, *args, **kwargs):
print "real raise_error(%s, %s)" % (args, kwargs)
1/0
def return_something(self, *args, **kwargs):
print "real return_something(%s, %s)" % (args, kwargs)
return self.something
def static_method(*args, **kwargs):
print "this is a staticmethod"
def class_method(cls, *args, **kwargs):
print "this is a classmethod"
print cls.__name__
static_method = staticmethod(static_method)
class_method = classmethod(class_method)
# --------------------------------------------------------------------------
---
# Demo
# --------------------------------------------------------------------------
---
def main():
filename = "junk.txt"
file_trace = trace.Trace(filename)
stdout_trace = trace.Trace()
MyClass.observers["on_before"] += [file_trace, stdout_trace]
MyClass.observers["on_after"] += [file_trace, stdout_trace]
MyClass.observers["on_error"] += [file_trace, stdout_trace]
# Exercise the class to test the Aspected framework.
f = MyClass()
f.bar("a", "b", foo="c")
try:
f.raise_error("a", "b", foo="c")
except Exception, e:
print "An error occurred in main(): %s" % e
f.something = "foo"
x = f.return_something(1, 2, 3)
print x
MyClass.static_method()
MyClass.class_method()
if __name__ == "__main__":
main()
***
-
Mark McEahern
comments will refer to the latest version of code that I just posted
separately.
[Pedro]
> If you really want something more consistent than my posting, I will
> recommend you to also check:
[snip]
> - Pythius by Juergen Hermann
> - Transwarp by Phillip J. Eby
[snip]
I've looked at Transwarp and failed to get it. Last time I looked there was
no simple demo that showed what problem it was trying to solve and how TW
solves it. But I'm dense and I should probably give it another try. I look
forward to checking out Pythius. Someone else mentioned that
recently--probably in the recent thread on metaclasses.
I wrote:
> (I'm not worried about distinguishing staticmethod and classmethod
> type methods for now):
[Pedro]
> You should <wink>. At least for the purpose of the exercise ;)
> (Honestly I didn't try either, it may end up with a two liner...
> but I wonder if they will take two minutes or two hours to be written ;)
The latest version I posted continues to punt on staticmethod and
classmethod. They're currently not aspected because my type comparison to
type(lambda x:x) will effectively filter out staticmethod and classmethod.
I would have to change the observer interface slightly since staticmethods
don't have have a self parameter and the first parameter of classmethods is
a reference to the class not the instance. I don't see this being too
hard--I plan to fix it in the next iteration.
> Even if deprecation of the types module have been discussed on the devel
> list, I think it is preferable.
Yeah, using type(lambda x:x) seems more portable or something. I just did a
little interactive session to verify that I can do this too:
type(staticmethod(lambda x:x))
type(classmethod(lambda x:x))
I can see the outlines of some sort of method wrapper factory even now:
anon = lambda x:x
# f, g, and h are make_wrapped_method variants for each type of method.
wrapper_factory = {type(anon): f,
type(staticmethod(anon)): g,
type(classmethod(anon)): h}
for k, v in dict.items():
wrapper = wrapper_factory[type(v)]
...
> Oops... Didn't I make the same mistake...
> ... forgetting about the returned value ;)
>
> Something like could be better:
> def __call__(self, *args, **kwargs):
> self.before(*args, **kwargs)
> ret = self.method(self, *args, **kwargs)
> self.after(*args, **kwargs)
> return ret
Great catch! Thanks. I fixed that in the latest version.
> This Trace class is an aspect.
Ah, I see that I was calling the metaclass Aspect whereas Trace is really an
Aspect. I definitely need help with naming and terminology, as you can
tell. I renamed the metaclass to Aspected. Does that seem better or am I
still missing the correct term?
> This is not an Aspect. This is your way to implement method call
> interception.
Thank you.
> What about module functions ? Bounded methods ?
> Using metaclass for call interception is too restrictive I believe.
Hmm, you're right, my approach will do NOTHING for methods that aren't
associated with a class. BTW, I thought a normal method associated with a
class whether new type or old was a bound method? I'm not too worried about
old style classes--although maybe I should be? I figure if a requirement of
using this is that it only works with new-style classes, that's fine with
me. Until I run into a problem, of course. ;-)
I often use methods that aren't associated with a class and I'd like to
aspect those as well, although, when I think about something like
Persistence, that does seem initimately bound up with classes. So maybe it
depends on the aspect?
I should say that my motivation for pursuing a metaclass implementation is
partly just to learn metaclasses, but also partly because I want something
that requires no modification to the aspected class and minimal effort,
maximum flexibility for defining join points. What I like about the
metaclass approach is that I just wire up every single class that's aspected
for all events. It's then up to the observer to hook into them. Of course,
I need to try to use this for something real and I'm sure that will expose
its weaknesses/my conceptual gaps quickly.
> This is the trickier part. When you do :
> setattr(cls, k, wrapped_method(cls, v))
> you substitute a function (actually an unbounded method) by a callable
> object. Unfortunately when you'll invoke this object with a classical
> method call, Python will not pass the instance as the first argument.
Yup, I fixed this.
> - providing a function, and take benifit of nested_scopes to retrieve
> all the information from your context
That's the approach I took. And to think, 6 months ago I had no freaking
idea what the point of nested scopes was. ;-)
> Too intrusive. I don't believe that you can do it dynamically, at least
> not for classes defined at module level. They will be created with the
> metaclass defined at compilation time.
This is a good point. Here's the sort of dynamism I'm aiming for:
The modules containing classes to be aspected have no reference to the
framework inside them. Not even a __metaclass__ statement.
Oops, a little testing shows that may simply not work. Hmm. I don't want
to have to edit the modules that contain the classes to be aspected, even to
add a __metaclass__ declaration. Since I can't seem to change that at
runtime, this approach probably won't work.
> Yes. Interception of raised exception is a good (and easy ;) feature.
> Just try to go a step further with 'around' methods.
I avoided adding around because it doesn't seem primitive to me. In other
words, isn't around just before + after notification? So I could add
support for around simply by making it so that observers wanting around
notification just got before and after notification? Not sure what to do if
the method raises an error--skip the after?
> Thanks, for this posting Mark. Reminds me that aop is my task list for
> the moment I will have some spare time (not so far I hope ;)
You're very welcome. Your comments have been most helpful.
> Just to give some hints on aop, this is what I wrote as a reminder
> 6 months ago along with a more complete implementation of aop :
[snip]
Wow, that's eye opening. I look forward to seeing more. In the meantime,
I'm going to reflect on whether to abandon the metaclass approach. It does
seem, as you say, restrictive. That is, it doesn't allow sufficient runtime
dynamism and it seems to require modifying the code-to-be-aspected.
Cheers,
// mark
-
Pedro Rodriguez
> The latest version I posted continues to punt on staticmethod and
> classmethod. They're currently not aspected because my type comparison
> to type(lambda x:x) will effectively filter out staticmethod and
> classmethod. I would have to change the observer interface slightly
> since staticmethods don't have have a self parameter and the first
> parameter of classmethods is a reference to the class not the instance.
> I don't see this being too hard--I plan to fix it in the next iteration.
>
>> Even if deprecation of the types module have been discussed on the
>> devel list, I think it is preferable.
>
> Yeah, using type(lambda x:x) seems more portable or something.
>
Sorry, but I wasn't clear : I meant that using the types module should
be better. But honnestly I am not that sure.
[Mark]
> Hmm, you're right, my approach will do NOTHING for methods that aren't
> associated with a class. BTW, I thought a normal method associated with
> a class whether new type or old was a bound method? I'm not too worried
> about old style classes--although maybe I should be? I figure if a
> requirement of using this is that it only works with new-style classes,
> that's fine with me. Until I run into a problem, of course. ;-)
>
Just to be sure, let me clarify some points about functions and methods :
- a def statement introduces a function
- when a function is declare in a class we should talk of an unbound
method
- when we make a reference to a function from an instance we talk of a
bounded method
>>> def f(x): pass
...
>>> class A:
... def f(self): pass
...
>>> a = A()
>>>
>>> print f
<function f at 0x8150854>
>>> print A.f
<unbound method A.f>
>>> print a.f
<bound method A.f of <__main__.A instance at 0x8150824>>
>>>
The difference between an unbound method and a bound method is that the
latest won't need to be passed a 'self' argument because it is already
bounded to an object.
For class and static methods, I will need some time to learn how to
identify them and catalog them in my 'bestiaire'.
[Mark]
> I often use methods that aren't associated with a class and I'd like to
> aspect those as well, although, when I think about something like
> Persistence, that does seem initimately bound up with classes. So maybe
> it depends on the aspect?
I see your points here.
Could/Should an aspect be used for Persistence ? I don't know, but it is
a good remark.
I recall that I discovered AOP after having to add some notification in
Models in an Model/View/Presenter framework and it seemed quite intrusive.
With your remark I wonder if the saving of my models (call it
Persistence/Serialization) should also be considered as a good candidate
for being aspectified. By doing so that I could keep my model simple and
externalize the way I want to implement save/load.
>
> I should say that my motivation for pursuing a metaclass implementation
> is partly just to learn metaclasses,
Understanding metaclasses is quite a challenge, putting them to good use
is, IMO, even more challenging (Alex Martelli recommended the reading of
the book mentionned in the 'descintro' document :
Putting Metaclasses to Work: A New Dimension in Object-Oriented Programming,
by Ira R. Forman and Scott H. Danforth)
> ... but also partly because I want
> something that requires no modification to the aspected class and
> minimal effort, maximum flexibility for defining join points. What I
> like about the metaclass approach is that I just wire up every single
> class that's aspected for all events. It's then up to the observer to
> hook into them. Of course, I need to try to use this for something real
> and I'm sure that will expose its weaknesses/my conceptual gaps quickly.
>
I think you should clearly separate :
1. the place where you do your 'method call interception'
2. the way you stack (and call) advices
Actually you do 1. at class definition level for all methods defined in
the class by using a metaclass, and for 2. you use an observer-like
object.
For 1. I decided to it in a different way, closer to Java implementations
(and I think that aop.py in Pythius already did it this way) :
- I created a Joinpoint class
- the constructor received an object called a callableSet and a string
pattern
- a callableSet was anything that could contain a 'function' I wanted to
intercept. This could be a module, a class, an instance
- I scaned the callableSet in search for attributes that were functions
that matched the pattern
- I intercepted the matching functions with what I defined for 2.
[Pedro]
>> Too intrusive. I don't believe that you can do it dynamically, at least
>> not for classes defined at module level. They will be created with the
>> metaclass defined at compilation time.
[Mark]
>
> This is a good point. Here's the sort of dynamism I'm aiming for:
>
> The modules containing classes to be aspected have no reference to the
> framework inside them. Not even a __metaclass__ statement.
>
> Oops, a little testing shows that may simply not work. Hmm. I don't
> want to have to edit the modules that contain the classes to be
> aspected, even to add a __metaclass__ declaration. Since I can't seem
> to change that at runtime, this approach probably won't work.
>
This is what I feared with the usage of metaclasses (at least by solely
using __metaclass__ in this way)
[Pedro]
>> Yes. Interception of raised exception is a good (and easy ;) feature.
>> Just try to go a step further with 'around' methods.
[Mark]
>
> I avoided adding around because it doesn't seem primitive to me. In
> other words, isn't around just before + after notification? So I could
> add support for around simply by making it so that observers wanting
> around notification just got before and after notification? Not sure
> what to do if the method raises an error--skip the after?
>
No 'around' is not that simple I fear. I think you could make some
comparison with 'generators'. One of the purpose of generators is to
simplify design by not having to keep a context between invocations.
The same thing occurs with 'around' in a less trivial way.
Consider this : you want to track some log information on a method :
- when it started
- how long it lasted
So that you have a trace like :
Method XXX called for object YYY at HH:MM for XX seconds
Question : how will you track the start time so that you'll be able to
compute the duration and produce your log line ?
[Take into account that method XXX may be called recursively, or
concurently in a multi-threaded environment]
An 'around' advice will simplify your life as a user of aop (but not as
the designer of the aspect framework ;) this is were I ended with an
implementation that seemed quite complicate and that will need some rework
- and also why I didn't dare posting its UML description ;)
Bon courage,
Pedro