macro FAQ
Andrew Dalke
threads. Appropriate comments appreciated
X.Y: Why doesn't Python have macros like in Lisp or Scheme?
Before answering that, a clarification on what 'macro' means.
A Lisp macro is a way of modifying code when that code is first
defined. It can rearrange the structure of the code, and add and
remove parts of it. Unlike C's #define macro language, Lisp
macros understand the structure of the expression and the
context (or "closure") in which it is found.
Here's a simple example of what a macro might do for Python.
A complaint about Python is that constructors for simple data
types require typing the parameter names three times, as in
class Country:
def __init__(self, name, capitol):
self.name = name
self.capitol = capitol
Suppose you could create a new way to create functions, which
looked at the argument list and automatically set the corresponding
member. This would let you replace the above with something
more like
class Country:
def __init__(self, name, capitol) [arg_assign]:
pass
where the 'arg_assign' would use the function definition
and code to make a new function used in place of the
existing one and with the 'self.name = name' and
'self.capitol = capitol' code automatically added to
the front of the code.
This is a very powerful technique and especially nice for
high-performance code where the macro can optimize
code before it is actually evaluated. Macros work well in
Lisp or Scheme, where the code is the data, all written
pretty much as a parse tree.
The first problem with implementing a macro system in Python
is coming up with a readable syntax; that is, how it's "spelled."
The example shown above, using the [] after the function
definition, was taken from a proposed syntax for adding
function modifiers like 'staticmethod' and 'classmethod' to
Python.
The second problem is that Python's code blocks do not
store the original parse tree, so there's nothing to manipulate.
If you use 'dir()' or the inspect module you'll see some fields
like the list of argument names and the default value to use
for the fields. Nowhere will you find the original parse tree.
(The C implementation of Python does give access to the
Python byte code, but this is not part of the language
specification and Java implementation doesn't support it.)
These problems can be resolved, and Dylan shows that
an infix-based/Algol-like language can support macros
similar to those used in Lisp.
The deeper question is, should Python include macros?
People with a Lisp background might have a problem
understanding this viewpoint. Macros add flexibility, and
more flexibilty lets programmers be more productive; or
so the argument goes.
Those against including macros in Python point out that
Python has a different focus. It has a broad target audience.
Beginners can pick it up pretty easily yet advanced
developers also enjoy working in the language. New features,
like new builtin functions, additional core types, expanding
the standard libraries, list comprensions, metaclasses and
deriving from object, and macros, all make it harder to
learn how to use Python well and shifts the balance more
towards advanced programmers.
More importantly, the detractors -- including those
with plenty of experience using macros in Lisp -- argue that
macros cause dialects to form. Macros can modify other code to
make it fit the problem better, while functions only use other
code but make no modifications. This makes them very
powerful but means that understanding a section of code
requires also knowing about any macros which might use the
code. In an extreme case which wouldn't be used in real
projects, every * could be replaced with a +.
(As an aside, some proponents argue that macros and
functions are essentially the same. Alex Martelli made
an interesting observation about one difference: it's often
worthwhile to turn a set of code into a function even if it
is only called from one place, because it helps simplify
the code into smaller chucks which are easier to understand.
Macros, on the other hand, should almost never be used
unless they are used many times.)
With only one or a small group of people working together
on a project there is little problem. Macros help in developing
idioms specific to the problem and group. When groups
share code, they also share idioms, and anyone who has had
to deal with UK vs. US English knows the effect idioms have
in mututal understanding.
Those against macros say their benefits do not outweigh
the likelihood that the Python community will be broken up
into distinct subsets, where people from one group find it
hard to use code from another.
The proponents for macros say this has not been a problem
in Lisp. Macros are misused, but so are functions and classes.
The same community feedback which encourages people to
use proper coding styles also encourages people to use macros
correctly. Those against reassert that not only can it be a
problem but it's happened to them personally.
At this point the discussion usually breaks down, with the
Lispers pulling out the old canard about Python (and every
other language) being a half-complete, poor reimplementation
of Lisp, while Pythonistas trot out the expression "Lisp is like
a ball of mud. You can add any amount of mud to it and it
still looks like a ball of mud."
Those interested in actual language improvements then ask for
concrete examples of how macros might be used for Python.
So far, the examples given are ones which could equally well
be done in Python using metaclasses, properties, and other
existing mechanisms. For example, the arg_assign example
given above could, I think, be done with a modified__new__.
The problem at this point seems to be that the two languages
have different enough object models and accepted best
practices that it's hard for an intuition based on Lisp to
know immediately how to apply that intuition to Python.
If you want to encourage Python to have macros, you must
therefore propose a syntax for it and give solid examples of
what the result would look like when applied to real Python
code. Simply stating that macros are needed is not good
enough nor are examples showing how macros are useful
for Lisp code. (Besides, believe it or not but some of us
don't know enough Lisp to follow your examples ;)
In the meanwhile, if you want to experiment with manipulating
code trees, you can use decompyle to turn byte code back
into Python source and the compiler module to turn source into
an AST and thence into byte code.
Andrew Dalke
da...@dalkescientific.com
Michele Simionato
> Here's a proposed Q&A for the FAQ based on a couple recent
> threads. Appropriate comments appreciated
>
> X.Y: Why doesn't Python have macros like in Lisp or Scheme?
This is all very interesting and I do agree that there shouuld be a
FAQ about macros, nevertheless I don't see your message as suitable
for a FAQ. It is ways too verbose. I mean, if one asks for a macro,
s/he already knows what a macro is. The FAQ should be much
shorter. What you wrote is more useful and interesting for
Pythonistas than from Lispers.
I think your document should be made available on the Web somewhere,
but not in the FAQ. Of course the FAQ could and should give a pointer
to it. Maybe lispers would be helped from a page describing how
some of the things made in macros in Lisp can be made in Python.
This page should be referred in the FAQ too. It is clear, however,
that many things involving macros cannot just be done in Python
(I mean, unless implementing a custom macro interpreter in Python,
which actually I wrote few months ago as a proof of concept (google
on c.l.py for simionato macros, first hit) and which I DO NOT use ;).
Michele Simionato, Ph. D.
MicheleS...@libero.it
http://www.phyast.pitt.edu/~micheles
--- Currently looking for a job ---
John J. Lee
> "Andrew Dalke" <ada...@mindspring.com> wrote in message news:<Dsz1b.3409$Ej6...@newsread4.news.pas.earthlink.net>...
> > Here's a proposed Q&A for the FAQ based on a couple recent
> > threads. Appropriate comments appreciated
> >
> > X.Y: Why doesn't Python have macros like in Lisp or Scheme?
[...]
> I think your document should be made available on the Web somewhere,
> but not in the FAQ. Of course the FAQ could and should give a pointer
> to it.
[...]
+1, FWIW
John
John J. Lee
> Here's a proposed Q&A for the FAQ based on a couple recent
> threads. Appropriate comments appreciated
[...]
> Those against including macros in Python point out that
> Python has a different focus. It has a broad target audience.
> Beginners can pick it up pretty easily yet advanced
> developers also enjoy working in the language. New features,
> like new builtin functions, additional core types, expanding
> the standard libraries, list comprensions, metaclasses and
> deriving from object, and macros, all make it harder to
> learn how to use Python well and shifts the balance more
> towards advanced programmers.
>
> More importantly, the detractors -- including those
> with plenty of experience using macros in Lisp -- argue that
> macros cause dialects to form. Macros can modify other code to
> make it fit the problem better, while functions only use other
> code but make no modifications. This makes them very
> powerful but means that understanding a section of code
> requires also knowing about any macros which might use the
> code. In an extreme case which wouldn't be used in real
> projects, every * could be replaced with a +.
[...]
About the first paragraph here: I don't recall anyone arguing that,
but maybe I forget. Anyway, my problem with it is that it reinforces
the "Python is a scripting language for amateurs, not suitable for
large scale / systems programming" attitude. I think Python
demonstrates that there isn't much tension in reality between the
'advanced programmers' and the beginners -- both seem to want a simple
language. And as you say, the argument about dialects is more
important anyway.
Other than that, excellent stuff.
John
Jacek Generowicz
> Here's a proposed Q&A for the FAQ based on a couple recent threads.
> Appropriate comments appreciated
Such an FAQ item is probably a good idea.
However, I think that we should distil out much of the opinion
(particularly the opinions about opinions of others :-) and try to
base it on fact:
- what are macros (C-like and Lisp-like),
- what are the technical difficulties with introducing them into
Python
> X.Y: Why doesn't Python have macros like in Lisp or Scheme?
>
> Before answering that, a clarification on what 'macro' means.
> A Lisp macro is a way of modifying code when that code is first
> defined. It can rearrange the structure of the code, and add and
> remove parts of it. Unlike C's #define macro language, Lisp
> macros understand the structure of the expression and the
> context (or "closure") in which it is found.
A closure is a function which "remembers" variables from an enclosing
lexical scope (also available in Python, since 2.1). Lisp macros
merely replace themselves with some source code (their expansion)
before the compiler sees them. I'm not sure what you mean by "macros
... understand the context".
> Here's a simple example of what a macro might do for Python.
> A complaint about Python is that constructors for simple data
> types require typing the parameter names three times, as in
>
> class Country:
> def __init__(self, name, capitol):
> self.name = name
> self.capitol = capitol
[snip]
Probably not the most exciting or edifying example, particularly
given that metaclasses should be able to do it.
I'm sure we'll be able to come up with a better one. I'll put some
ideas forward at the end, to act as a starting point.
> Macros work well in Lisp or Scheme, where the code is the data, all
> written pretty much as a parse tree.
This is very important, and is central to the problem of "how do we do
it in Python?"
> The second problem is that Python's code blocks do not
> store the original parse tree, so there's nothing to manipulate.
[...]
> Nowhere will you find the original parse tree.
This is typically true in Lisp as well, particularly after you have
compiled your code. (Common Lisp _must_ have a compiler, but may _in
addition_ have an interpreter; the standard defines a function which
returns the source code of a function, but it is allowed to say
"sorry, you can't have it".)
The point in Lisp is that you _write_ your code in the form of a parse
tree, in Lisp lists, and you can pass your parse tree to a macro
without having to re-format it, and have the macro easily manipulate
it. It's not clear how you would do this in Python.
But you seem to be implying that macros go off and find the parse
trees of code that has already been compiled, and then change it?
(Hint: they don't.)
> The deeper question is, should Python include macros?
>
> People with a Lisp background might have a problem understanding
> this viewpoint.
For the record, I am not advocating the inclusion of macros in
Python. (I do object to the suggestion that macros are responsible for
some supposed "fragmentation" of Lisp.)
> More importantly, the detractors -- including those with plenty of
> experience using macros in Lisp -- argue that macros cause dialects
> to form.
Could you please give me a reference to someone "with plenty of
experience using macros in Lisp" arguing this ?
I just don't believe it. (That's not to say that it's not true.)
> Macros can modify other code to make it fit the problem better,
What do you mean by "other" code? Macros modify the code that is
passed to them as an argument, transforming it before the compiler
gets to see it. I get the impression that you believe that macros can
somehow modify code from other parts of the program. They can't.
Ultimately, macros just save you a lot of typing of source code. (And
thereby save you a lot of bugs.) If you can't type it as soure code,
then a macro can't do it.
> while functions only use other code but make no modifications.
This only strengthens the above suspicion, but I'm not really sure
what you mean, here.
> This makes them very powerful but means that understanding a section
> of code requires also knowing about any macros which might use the
> code.
What do you mean by "macros using code" ?
The macros are part of the code, just like functions are. To
understand the code, you must understand what the macros and functions
do.
> In an extreme case which wouldn't be used in real projects, every *
> could be replaced with a +.
This almost completely convinces me that you are very confused about
what macros can achieve.
Either, you are suggesting that one might write a macro which replaces
every * operator with a + operator, and then pass _the entire source
code_ of a project to it (possible, but patently absurd); or you are
suggesting that it is possible to write a macro "over here" which,
somehow, surreptitiously modifies existing source code "over there".
a) Macros cannot do the latter.
b) You can achieve something very similar in Python, by re-binding
attributes of __builtins__.
Think of the point of macros in another way. The point is _not_ to
take existing source code, and to change it. The point is to take
something that is not (necessarily) valid source code, and turn it
into valid source code, before the compiler gets to see it. Why?
Because this allows you to create a shorthand notation, and use macros
to expand it to real code. Think in terms of encoding design patterns.
> (As an aside, some proponents argue that macros and
> functions are essentially the same. Alex Martelli made
> an interesting observation about one difference: it's often
> worthwhile to turn a set of code into a function even if it
> is only called from one place, because it helps simplify
> the code into smaller chucks which are easier to understand.
> Macros, on the other hand, should almost never be used
> unless they are used many times.)
I broadly agree. (I think :-)
A function written for a single-location-call, does not really
abstract anything, it just moves some code elsewhere to make the
original location look less hairy. Rarely would you need to use a
macro for this purpose, but in such cases there would be no objection
to writing a single-invocation macro.
Macros which truly abstract something, can be difficult to write and
difficult to read; in such cases it is important that the cost of
creating a robust macro and the cost incurred by others in trying to
understanding it, be offset by the savings made by its _repeated_
usage.
But then, functions which provide an abstraction are also more
difficult to write and understand than ones which merely "move code
out of the way", and you'd have to think twice whether the abstraction
is really useful, before deciding to pay the price for writing it, and
making readers understand it.
Even in this respect, there is no clear-cut distinction between
functions and macros.
> With only one or a small group of people working together
> on a project there is little problem. Macros help in developing
> idioms specific to the problem and group. When groups
> share code, they also share idioms, and anyone who has had
> to deal with UK vs. US English knows the effect idioms have
> in mututal understanding.
>
> Those against macros say their benefits do not outweigh
> the likelihood that the Python community will be broken up
> into distinct subsets, where people from one group find it
> hard to use code from another.
I believe that anyone reaching such a conclusion can only do so
on the basis of a misunderstanding of what macros can do.
People in the music software "group" will find it hard to use code
from people writing software for bioinformatics ... with or without
macros. This has nothing to do with macros.
OK, I promised some examples of macros. Note that I have not shown a
single line of Lisp code in these threads, because I agree that it is
likely to be meaningless to most readers of this group. I'll try
continue without having to resort to Lisp examples.
==== Example 1: a debugging aid ================================
One litte macro I have for debugging purposes (let's call it "show"),
allows me to print out the source code of a number of expressions,
along with their values. At this point, note that Python's horribly
irregular syntax <0.5 wink> already starts causing problems: should
the macro look like a block of code, or should it look like a function
call; in Lisp both look identical.
The "block" version of the show macro invocation:
show:
sin(x)
a
a + b + c
The "function" version of the show macro invocation:
show(sin(x), a, a + b + c)
In both cases, the action of the macro should be to replace itself
with its expansion, _before_ the compiler gets to see the source code.
The expansion should look like this:
print "sin(x) =>", sin(x)
print "a =>", a
print "a + b + c =>", a + b + c
Note the key points:
1) The macro receives some data, and transforms it into valid Python
source code
2) This happens before compile time
3) Nothing "outside" the macro call gets affected by the macro's
action.
Can this be implemented using metaclasses, or any other existing
Python mechanism ?
=== Example 2: Alexander Schmolck's updating classes ===============
Alexander recently expressed the desire to have all existing instances
of a class be updated, when he changes the source of his class, and
re-evaluates the definition.
This might be achieved by doing something like the following:
temp = MyClass
class MyClass:
blah
temp.__dict__ = MyClass.__dict__
I'm not so much interested in the fine Python details of exactly what
needs to be modified (__dict__, __bases__, whatever); what is
important is that at least 3 distinct steps need to be taken, and
trying to bundle these three steps (this pattern) into a function
is impossible, because you can't pass the text of a class definition
body to a function. (You could pass it as a string, but then your IDE
would treat it as a string and refuse to indent it for you, and
writing classes this way would be very unnatural.)
A macro might help as follows. Write a macro called updating_class,
which you would use instead of the built-in class statement. This
macro might work according to an algorithm like this:
class_name = <the first token of what was passed in>
the_body = <everything that was passed in except, the first token>
expansion = "temp = %s" % class_name
expansion += "class %s%s" % (class_name, the_body)
expansion += "temp.__dict__ = %s.__dict__" % class_name
return expansion
Here I'm assuming that the return value of the macro replaces the
macro call in the source code, before the compiler gets to see the
source code (which is pretty much what happens in Lisp).
So,
updating_class foo:
blah
would turn into
temp = foo
class foo:
blah
temp.__dict__ = foo.__dict__
Again, note that any code not appearing "inside" the macro call, is
not affected by the macro in any way ... almost ...
... This second example also serves the purpose of demonstrating free
variable capture (the other type of variable capture is called macro
argument capture). Variable capture is what is (according to the
Schemers) unhygenic about CL macros. What is it? Well, note that, if a
variable called "temp" had already existed in our enclosing scope, the
updating_class macro expansion would have clobbered it.
Scheme gets around this by ensuring that "temp" lives inside a local
scope of the macro. The CL philosophy respects the fact that sometimes
you deliberately _want_ to capture some variables. CL allows you to
protect yourself against variable capture by using the "gensym"
function; a function which creates a symbol (think of it as an
identifier), which is guaranteed not to have existed before.
This has turned out rather long, but I hope that it demystifies these
scary exotic macros for at least one or two people on the list.
Doug Tolton
I think Jacek has made an extremely good point here, and I want to
follow up on it to determine if everyone has a clear understanding of
what macros are and are not. Following I have outlined a series of
questions an answers to attempt to illuminate the issue. Please
suggest any clarifications or distinctions that need to be made. Also
note that the examples presented are purposefully trivial and could in
many instances be accomplished other ways, however I am presenting
these examples in order to provide a meaningful context to proceed
from.
Question: What are Macros?
Answer: Macros are an advanced templating system that allow dynamic
evaluation of code during run time. Essentially Macros allow you to
pass in some variables that are then expanded into valid Python syntax
which is then executed.
Question: Can't I accomplish this same functionality via string
parsing and eval?
Answer: Eval will accept any valid Python expression and execute it.
This is however regarded by many as a security hole. Macros allow
this type of expressiveness while limiting many of the security risks.
With a Macro you can more specifically control what types of actions
are permissible.
Question: I have some existing modules, if I import a macro from
someone else, could the macro make my existing code perform in
unexpected ways?
Answer: No. A macro doesn't modify existing code, in order to invoke
a macro it must be called just like a function call. So a Macro
couldn't alter the functionality of existing code any more than a
function could. The only way to accomplish such a feat would be to
manually open each file and parse through them and re-evaluate them.
This type of functionality is not what macros are about any more than
a function call would be.
Question: How would Macros work in practice?
Answer: Here is a trivial example macro to start with:
def HelloFunc(message):
print "%s\n" % (message)
defmacro uselessmacro(message):
macroexpansion = "HelloFunc(%s)" % (message)
defmacro callfunc(funcname, message):
macroexpansion = """
calllog = open("/var/log/functions.log", "a")
calllog.write("%(funcname)s was called.\n")
calllog.flush()
calllog.close()
%(funcname)s(%(message)s)
""" % locals()
if __name__ == "__main__":
uselessmacro("Hello World")
callfunc("HelloFunc", "Hello World")
# Note, Jacek's method of using indentation is an interesting
alternative to calling them as a
# function.
# Second note, this textarea box does not have a compiler, so please
forgive any syntax errors :)
uselessmacro simply passes it's arguments through to HelloFunc
undisturbed
callfunc logs all calls to a function before evaluating the function
You could clearly accomplish this functionality using other means,
however I have used such a trivial example in order to illustrate
several points:
1) Macros once they are defined would most likely be invoked the same
way any other function call would be or represented as a block of
code.
2) Macros taken in isolation would not allow any strange changes to
the language to be made (eg changing all *'s to +'s, or changing the
behavior of existing classes - metaclasses, builtins and magic methods
are required for this)
3) Understanding a macro's behavior is typically easy for someone
familiar with Python's syntax and constructs. In otherwords a Macro
allows you to encapsulate commonly used idioms into a cleaner syntax,
but would not allow you to alter the behavior of the language itself.
My examples are barely touching the surface of what is possible with
macros, but I wanted to start with some simple examples in order to
give everyone some solid examples of what Macros are, and what Macros
are not.
Terry Reedy
these
> scary exotic macros for at least one or two people on the list.
Definitely. The sort of concrete example I asked for. Thank you.
Terry
Miki Tebeka
Great work!
For a good reading about macros and why/how to use them you can point
to Paul Graham's "On Lisp" which is freely available online at
http://www.paulgraham.com/onlisptext.html
He has great examples over there. My personal favorites are the
anaphoric macros (aif ...)
HTH.
Miki
Michele Simionato
Yes.
At the end it uses metaclasses+exec+a simple argument parser based on
tokenize. EXPERIMENTAL code, which I would not trust except as proof
of concept.
> === Example 2: Alexander Schmolck's updating classes ===============
I think this also can be done with metaclasses (somebody already
posted a recipe in some other thread, Diederich?)
Jacek Generowicz
> Also note that the examples presented are purposefully trivial and
> could in many instances be accomplished other ways,
I hoped that the examples that I presented satisfied the requirements
that
a) They be simple enough to understand
b) not be (at least easily) implementable using existing Python
features
c) Not be completely useless
Did I fail ? :-)
> Essentially Macros allow you to pass in some variables that are then
> expanded into valid Python syntax which is then executed.
Well, not just variables, data in general, some of which might be
variables, some of which might be variable names (identifiers) and
some of which might be (peudo-) code.
> Question: Can't I accomplish this same functionality via string
> parsing and eval?
>
> Answer: Eval will accept any valid Python expression and execute it.
> This is however regarded by many as a security hole. Macros allow
> this type of expressiveness while limiting many of the security risks.
> With a Macro you can more specifically control what types of actions
> are permissible.
I don't think that this is relevant (or true, for that matter). Lisp
macros do not expand arbitrary data that a user passes in, they appear
as literals in your source code. The security issues are the same as
those surrounding "os.system('rm -rf /')" appearing literally in your
source code.
What is relevant, is the pain involved in doing it via strings. The
(pseudo) code that you pass to a Lisp macro, not only looks exactly
like Lisp code, but it is automatically tokenized and parsed for you,
and very easily manipulatable.
> # Note, Jacek's method of using indentation is an interesting
> alternative to calling them as a
> # function.
It's probably worth pointing out, again, how important Lisp's
uniformity of syntax is, for its macro system. Generally speaking, a
macro accepts as arguments some mixture of data, identifiers and
expressions (let's forget about the existence of Python statements for
now, they complicate the situation even more). How do you pass in a
mixture of data and code?
Data is usually presented in a tuple:
foo(datum1, datum2, datum3)
Expressions are usually presented on separate lines:
def foo(...):
expression1
expression2
expression3
Let's try to think of what a Python with-open-file macro call would
look like.
In CL, with-open-file takes
1) a list containing an identifier (a symbol), a pathname (let's call
it a string) with some optional keywords.
2) arbitrary code
It then binds the identifer to a stream associated with the file, and
expands the source code inside an exception handling environment.
So, the expansion of a (simplistic) Python with_open_file might look
like this:
outfile = file("foo/bar", "w")
try:
print >> outfile, foo(...)
for i in range(10):
print >> outfile, bar(...)
except:
perform the necessary closing and cleaning up
But what should the corresponding call to with_open_file look like?
We have to pass in the identifier "outfile" (this is where it is very
handy to have symbols as a first-class data type, as in Lisp), and the
file name "foo/bar" ... but then we also have to pass in the chunk of
code
print >> outfile, foo(...)
for i in range(10):
print >> outfile, bar(i)
In Lisp it would look something like this:
(with-open-file (outfile "foo/bar" :direction :output)
(format outfile "~a" (foo ...))
(dotimes (i 10)
(format outfile "~&~a" (bar i))))
Now, I refuse to contemplate passing the source code in a string, in
the Python version.
Maybe we have to accept the distinction between data-like data, and
code-like data, having the macro accept both a tuple of arguments, and
blocks of code, so the call might look thus:
with_open_file("outfile", "foo/bar", "w"):
code:
print >> outfile, foo(...)
for i in range(10):
print >> outfile, bar(...)
(If you need to pass in more than one code block, each one would go in
a seperate "code:" block.)
What could the definition look like?
defmacro with_open_file(stream, filename, mode)(block1)
expand(stream) = file(filename, mode)
try:
expand(block1)
except:
perform the necessary closing and clearing up
(expand would be a function which gets replaced by its argument, at
macro expansion time.)
Hmm, just about bearable, I guess, but I doubt that it generalizes
very well.
Note that we didn't transform the source code argument, just pasted it
in directly. If we wanted to manipulate it first, we'd be in for
serious suffering.
> You could clearly accomplish this functionality using other means,
I think it is best to steer clear of such examples, as experience
shows that some people will not be able to resist the temptation to
conclude (and suggest) that there is no point to macros, and the
discussion gets derailed. We've seen enough of "you could do this
quite easily without macros, but ...", and not enough of "This would
be soooo painful without macros ..." (I won't say "impossible", as
there is always Turing completeness.)
What was wrong with my original examples? I'd like to know, so that I
can replace them with better ones.
Jacek Generowicz
> My personal favorites are the anaphoric macros (aif ...)
Though, you do realize that these are buggy, in terms of their
interaction with the package system :-)
Jacek Generowicz
> > Can this be implemented using metaclasses, or any other existing
> > Python mechanism ?
>
> Yes.
>
> http://groups.google.it/groups?hl=it&lr=&ie=UTF-8&threadm=2259b0e2.0306040808.1d737a54%40posting.google.com&rnum=2&prev=/groups%3Fhl%3Dit%26lr%3D%26ie%3DISO-8859-1%26q%3Dsimionato%2Bmacros%26meta%3Dgroup%253Dcomp.lang.python.*
Unfortunately I can't easily access this at the moment. I'll try to
have a look later.
> At the end it uses metaclasses+exec+a simple argument parser based on
> tokenize. EXPERIMENTAL code, which I would not trust except as proof
> of concept.
If it uses exec, then it is faking up macros.
Lisp macros are little else besides offering language support for the
use of exec for this kind of thing.
If people start resorting to this sort of trick themselves, and
re-inventing it over and over again, then you might as well design it
properly, once and for all, and call it a macro system.
I submit that Python already has macros ... they just need a bit of
streamlining :-)
John J. Lee
> "Andrew Dalke" <ada...@mindspring.com> writes:
>
> > Here's a proposed Q&A for the FAQ based on a couple recent threads.
> > Appropriate comments appreciated
>
> Such an FAQ item is probably a good idea.
>
> However, I think that we should distil out much of the opinion
> (particularly the opinions about opinions of others :-) and try to
> base it on fact:
I disagree. People reading a FAQ do want to know the opinion of
language users, and more importantly, of Guido, because that's what
explains why the language is as it is, and determines what will happen
in the future. That's fact too, just fact about people's opinions.
> - what are macros (C-like and Lisp-like),
>
> - what are the technical difficulties with introducing them into
> Python
Arguably, these are both secondary issues given the fact that both
Guido (I presume?) and most Python users don't want macros. Obviously
some background should be in there explaining what macros are and how
they might apply to Python, but Andrew did just that.
[...snip useful technical criticism of Andrew's FAQ...]
> Could you please give me a reference to someone "with plenty of
> experience using macros in Lisp" arguing this ?
Maybe he was referring to Laura Creighton? He posted a quote and a
link to another of her posts.
[...snip more technical points...]
> But then, functions which provide an abstraction are also more
> difficult to write and understand than ones which merely "move code
> out of the way", and you'd have to think twice whether the abstraction
> is really useful, before deciding to pay the price for writing it, and
> making readers understand it.
I'm amazed that you say this. I think most programmers would disagree
with you here (certainly I do). Sure, abstract functions require some
work to understand the concept involved, but the important way in
which one function is easier or harder to understand than another has
to do with how much detail you need to know, not how hard the concepts
are. Needing to know details causes bugs. So does needing to know
concepts (though less often), but you can't get rid of the concepts,
whereas you *can* hide the details. And that's what abstraction is:
hiding details.
(and there's no need to answer here that you think macros are a good
way of hiding detail: we've gathered that ;-)
[...]
> People in the music software "group" will find it hard to use code
> from people writing software for bioinformatics ... with or without
> macros. This has nothing to do with macros.
But people writing code using one bioinformatics library will not find
it hard to read code using another. That might not be the case if
macros were built-in.
> OK, I promised some examples of macros. Note that I have not shown a
> single line of Lisp code in these threads, because I agree that it is
> likely to be meaningless to most readers of this group. I'll try
> continue without having to resort to Lisp examples.
>
> ==== Example 1: a debugging aid ================================
>
> One litte macro I have for debugging purposes (let's call it "show"),
> allows me to print out the source code of a number of expressions,
> along with their values. At this point, note that Python's horribly
[...]
> The "function" version of the show macro invocation:
>
> show(sin(x), a, a + b + c)
[...is equivalent to...]
> print "sin(x) =>", sin(x)
> print "a =>", a
> print "a + b + c =>", a + b + c
[...]
> Can this be implemented using metaclasses, or any other existing
> Python mechanism ?
Yes, using Lisp ;-P -- you can do this in Python with an emacs macro.
> === Example 2: Alexander Schmolck's updating classes ===============
>
> Alexander recently expressed the desire to have all existing instances
> of a class be updated, when he changes the source of his class, and
> re-evaluates the definition.
[...]
There was a thread recently about how the problem behind this might be
solved in Python. I started it (a search for "DOM" or "4DOM" or
"hierarchy" and my email address should turn it up).
No, wait a minute -- instances, not classes?? Sounds like an abuse of
the language to me. What *is* the underlying problem to be solved
here?
John
John J. Lee
> dto...@yahoo.com (Doug Tolton) writes:
[...]
> I hoped that the examples that I presented satisfied the requirements
> that
>
> a) They be simple enough to understand
>
> b) not be (at least easily) implementable using existing Python
> features
>
> c) Not be completely useless
>
> Did I fail ? :-)
Yes. :-)
John
John J. Lee
> mi...@pitt.edu (Michele Simionato) writes:
>
> > > Can this be implemented using metaclasses, or any other existing
> > > Python mechanism ?
> >
> > Yes.
[...]
> > At the end it uses metaclasses+exec+a simple argument parser based on
> > tokenize. EXPERIMENTAL code, which I would not trust except as proof
> > of concept.
>
> If it uses exec, then it is faking up macros.
>
> Lisp macros are little else besides offering language support for the
> use of exec for this kind of thing.
>
> If people start resorting to this sort of trick themselves, and
> re-inventing it over and over again, then you might as well design it
> properly, once and for all, and call it a macro system.
[...]
Well, as has been repeatedly argued, that's just the point: people
*don't* do that over and over again. Who uses these 'faked-up
macros'? A few people who like messing around with metaclasses for
fun (note Michele's warning about 'EXPERIMENTAL' and 'proof of
concept'). If it's made easier, relatively sane people might do it
too, and then we'll all suffer ;-)
I'm sure you're capable of understanding this position, but the
argument has started going round in circles.
John
Jacek Generowicz
> Jacek Generowicz <jacek.ge...@cern.ch> writes:
>
> > However, I think that we should distil out much of the opinion
> > (particularly the opinions about opinions of others :-) and try to
> > base it on fact:
>
> I disagree. People reading a FAQ do want to know the opinion of
> language users, and more importantly, of Guido, because that's what
> explains why the language is as it is, and determines what will happen
> in the future. That's fact too, just fact about people's opinions.
I agree. However, my feeling was that the original contained too much
uneducated opinion. I was trying to suggest that we first establish
the facts, and then include opinions based on those facts, rather than
ones based on FUD. Particularly, we should banish the myth that macros
fragment languages, or make code less unedrstandable. (Quite the
reverse is true.)
> [...] the fact that both Guido (I presume?) and most Python users
> don't want macros.
<uk7cpj...@boost-consulting.com>
> > Could you please give me a reference to someone "with plenty of
> > experience using macros in Lisp" arguing this ?
>
> Maybe he was referring to Laura Creighton? He posted a quote and a
> link to another of her posts.
The impression I get is that Laura was writing macros for kicks, not
as part of a serious development effort. Of course you could use
macros to write your own language much more easily than if you had to
do it from scratch, but what you would be doing would still be
wrtiting your own, separate, language.
My point is that macros could be immensely useful in making
constructive domain-specific extensions to the language, while _at the
same time_ not fragmenting the Python community.
> [...snip more technical points...]
> > But then, functions which provide an abstraction are also more
> > difficult to write and understand than ones which merely "move code
> > out of the way", and you'd have to think twice whether the abstraction
> > is really useful, before deciding to pay the price for writing it, and
> > making readers understand it.
>
> I'm amazed that you say this. I think most programmers would disagree
> with you here (certainly I do). Sure, abstract functions require some
> work to understand the concept involved, but the important way in
> which one function is easier or harder to understand than another has
> to do with how much detail you need to know, not how hard the concepts
> are. Needing to know details causes bugs. So does needing to know
> concepts (though less often), but you can't get rid of the concepts,
> whereas you *can* hide the details. And that's what abstraction is:
> hiding details.
I agree that abstractions usually hide details. I disagree with the
claim that hiding details is the essence of abstraction. To me, the
essence of abstraction is noticing a similarity between certain
"things", understandig how they differ, and encoding the whole set of
them in one implementation, with an obvious degree of freedom which
selects a specific "thing" from the whole family.
If I have a function which is 400 lines long, and I choose to make it
clearer by cutting and pasting bits of its body into smaller
functions, and calling those functions from the original, I haven't
abstracted anything; I have merely split it into more digestible
chunks.
> (and there's no need to answer here that you think macros are a good
> way of hiding detail: we've gathered that ;-)
But remember that I don't necessarily think they are a good idea in
Python.
> [...]
> > People in the music software "group" will find it hard to use code
> > from people writing software for bioinformatics ... with or without
> > macros. This has nothing to do with macros.
>
> But people writing code using one bioinformatics library will not find
> it hard to read code using another. That might not be the case if
> macros were built-in.
It might also be the case even in the absence of macros.
Why do so many people here think that macros can perform some
obfuscating magic, of which other language constructs such as
functions or classes are not capable ?
> > Can this be implemented using metaclasses, or any other existing
> > Python mechanism ?
>
> Yes, using Lisp ;-P -- you can do this in Python with an emacs macro.
:-)
> > === Example 2: Alexander Schmolck's updating classes ===============
> >
> > Alexander recently expressed the desire to have all existing instances
> > of a class be updated, when he changes the source of his class, and
> > re-evaluates the definition.
> [...]
>
> There was a thread recently about how the problem behind this might be
> solved in Python. I started it (a search for "DOM" or "4DOM" or
> "hierarchy" and my email address should turn it up).
>
> No, wait a minute -- instances, not classes?? Sounds like an abuse of
> the language to me. What *is* the underlying problem to be solved
> here?
It started in the Ruby thread: in Ruby classes are "open".
Say you're developing a class (foo), make a gazillion instances of it,
and play around with them as you are continuing to develop your
program. Suddenly you realize "Darn, I really need a fubar method!"
In Ruby you do something like:
class foo
def fubar ...
end
end
All your existing gazillion instances of foo have just acquired a
fubar method, and you can happily continue your exploration with all
the state you built up still intact.
The first thing you might try to do in Python is to add the fubar
method to the class source code, and re-evaluate it ... only to find
that your existing instances know nothing about the new method ... so
you have to go through the hassle of re-creating a gazillion of them,
before you can continue with what you were doing.
(Note, this is advocated as an interactive development aid, _not_ as a
production code technique.)
Jacek Generowicz
> I'm sure you're capable of understanding this position, but the
> argument has started going round in circles.
As I've said repeatedly:
- I do _not_ necessarily advocate the inclusion of macros in Python.
- I merely wish to dispel the myth that macros make code less
readable[*]: they _ease_ understanding of the code.
[*] And they don't fragment languages either.
Lulu of the Lotus-Eaters
|Could you please give me a reference to someone "with plenty of
|experience using macros in Lisp" arguing this ?
At least Laura Creighton and Alex Martelli stand out as examples. I'm
not sure either of them wants to be named specifically in this FAQ, just
to attract a lot of argumentative email.
Yours, Lulu...
--
Keeping medicines from the bloodstreams of the sick; food from the bellies
of the hungry; books from the hands of the uneducated; technology from the
underdeveloped; and putting advocates of freedom in prisons. Intellectual
property is to the 21st century what the slave trade was to the 16th.
John J. Lee
> j...@pobox.com (John J. Lee) writes:
[...]
> I agree. However, my feeling was that the original contained too much
> uneducated opinion. I was trying to suggest that we first establish
> the facts, and then include opinions based on those facts, rather than
> ones based on FUD. Particularly, we should banish the myth that macros
> fragment languages, or make code less unedrstandable. (Quite the
> reverse is true.)
It would be nice to make a definitive ruling :-), but it seems clear
most people here disagree with that. If Andrew's FAQ answer is going
to be linked to, no reason why other people can't supply an
alternative answer and have that linked to from the Python FAQ too.
[...]
> I agree that abstractions usually hide details. I disagree with the
> claim that hiding details is the essence of abstraction. To me, the
> essence of abstraction is noticing a similarity between certain
> "things", understandig how they differ, and encoding the whole set of
> them in one implementation, with an obvious degree of freedom which
> selects a specific "thing" from the whole family.
Or, abstracting a bit: hiding details <wink>.
> If I have a function which is 400 lines long, and I choose to make it
> clearer by cutting and pasting bits of its body into smaller
> functions, and calling those functions from the original, I haven't
> abstracted anything; I have merely split it into more digestible
> chunks.
We're probably in violent agreement.
John
Doug Tolton
> dto...@yahoo.com (Doug Tolton) writes:
>
> > Also note that the examples presented are purposefully trivial and
> > could in many instances be accomplished other ways,
>
> I hoped that the examples that I presented satisfied the requirements
> that
>
> a) They be simple enough to understand
>
> b) not be (at least easily) implementable using existing Python
> features
>
> c) Not be completely useless
>
> Did I fail ? :-)
No I don't think you failed, I just wanted to nail down that specific
point. The reason I purposefully used trivial examples was to
illustrate the point the Macros don't somehow magically alter existing
source code, rather they operate on source code that is passed into
them. I actually liked your examples, I was just hoping to come at
them from a slightly different example.
>
> > Essentially Macros allow you to pass in some variables that are then
> > expanded into valid Python syntax which is then executed.
>
> Well, not just variables, data in general, some of which might be
> variables, some of which might be variable names (identifiers) and
> some of which might be (peudo-) code.
>
> > Question: Can't I accomplish this same functionality via string
> > parsing and eval?
> >
> > Answer: Eval will accept any valid Python expression and execute it.
> > This is however regarded by many as a security hole. Macros allow
> > this type of expressiveness while limiting many of the security risks.
> > With a Macro you can more specifically control what types of actions
> > are permissible.
>
> I don't think that this is relevant (or true, for that matter). Lisp
> macros do not expand arbitrary data that a user passes in, they appear
> as literals in your source code. The security issues are the same as
> those surrounding "os.system('rm -rf /')" appearing literally in your
> source code.
>
> What is relevant, is the pain involved in doing it via strings. The
> (pseudo) code that you pass to a Lisp macro, not only looks exactly
> like Lisp code, but it is automatically tokenized and parsed for you,
> and very easily manipulatable.
>
Again, I wanted to focus the discussion for a moment to help alleviate
people's fear of Macro's, as they seem to be based on
mis-understanding what they are about. Please forgive my over
simplification.
I agree some people have been derailed by that concept, but I'd hoped
that making it painfully clear that this wasn't the only way to do it,
nor was it necessarily a representative sample of Macros, that maybe
some people would see conceptually what Macros are in an extremely
simple way. The Hello World of Macros.
Again, I thought your examples were fine, I just wanted to simplify
them to help the discussion along. If you hate them that much, we can
forget I ever posted them. :-p
Doug Tolton
eltr...@juno.com
writes:
> As I've said repeatedly:
> - I do _not_ necessarily advocate the inclusion of macros in
> Python.
grain of salt?
> - I merely wish to dispel the myth that macros make code less
> readable[*]: they _ease_ understanding of the code.
maybe with 'a few' simple macros. the fear, if any, is that imported
files depending on more macros will eventually be too great to fit your
brain. its a decision many agree with and has been amply fleshed out with
all sides represented.
we would all use them, debug them, curse them often.
macros containing macros will be possible.
with latent bugs and security holes now multiplied.
they can solve nothing useful speedwise since they wont replace code
inline. with lazy evaluation of code this is less of a concern.
>
> [*] And they don't fragment languages either.
I'm betting you usually get the last word.
macros [will/might/beyond all doubt can] fragment the language into
harder to read pieces. not the origional argument, but a concern.
if everyone shoots for 100% of their personal truth the FAQ will be
unreadable.
keep it to at most few short paragraphs please.
e
please forward all spam to "not my real email"<he...@ftc.gov>
________________________________________________________________
The best thing to hit the internet in years - Juno SpeedBand!
Surf the web up to FIVE TIMES FASTER!
Only $14.95/ month - visit www.juno.com to sign up today!
Jacek Generowicz
> Jacek Generowicz <jacek.ge...@cern.ch> writes:
>
> > j...@pobox.com (John J. Lee) writes:
> [...]
> > Particularly, we should banish the myth that macros fragment
> > languages, or make code less unedrstandable. (Quite the reverse is
> > true.)
>
> it seems clear most people here disagree with that.
It seems clear to me, that most of the people who disagree with it,
disagree with it on the basis of a misunderstanding of what it is that
macros actually do.
I am keen to avoid a whole FAQ item, or a whole web page to be built
around a bogus starting point.
Michele Simionato
> Jacek Generowicz <jacek.ge...@cern.ch> writes:
> Well, as has been repeatedly argued, that's just the point: people
> *don't* do that over and over again. Who uses these 'faked-up
> macros'? A few people who like messing around with metaclasses for
> fun (note Michele's warning about 'EXPERIMENTAL' and 'proof of
> concept'). If it's made easier, relatively sane people might do it
> too, and then we'll all suffer ;-)
I wholeheartly agree. I DO NOT use macros, I wrote that module as
a reply to a thread asking about macros. I wanted to see by myself
how difficult it was to implement them. The answer is: EXTREMELY
difficult, if you want a RELIABLE implementation. My implementation
is very poor man with no error checking at all, working only in simple
cases, but still it is not trivial. I am not convinced of the advantages
of having macros in Python, whereas I am convinced of the disadvantages.
So, I am for the status quo.
Christos TZOTZIOY
<jacek.ge...@cern.ch> might have written:
[See at the end for an interactive session transcript -- a simple copy -
paste]
>> No, wait a minute -- instances, not classes?? Sounds like an abuse of
>> the language to me. What *is* the underlying problem to be solved
>> here?
>
>It started in the Ruby thread: in Ruby classes are "open".
>
>Say you're developing a class (foo), make a gazillion instances of it,
>and play around with them as you are continuing to develop your
>program. Suddenly you realize "Darn, I really need a fubar method!"
>
>In Ruby you do something like:
>
>class foo
> def fubar ...
>
> end
>end
>
>All your existing gazillion instances of foo have just acquired a
>fubar method, and you can happily continue your exploration with all
>the state you built up still intact.
>
>The first thing you might try to do in Python is to add the fubar
>method to the class source code, and re-evaluate it ... only to find
>that your existing instances know nothing about the new method ... so
>you have to go through the hassle of re-creating a gazillion of them,
>before you can continue with what you were doing.
##### code starts here #####
>>> class A(object):
pass
>>> a= A()
>>> a.test()
Traceback (most recent call last):
File "<pyshell#21>", line 1, in ?
a.test()
AttributeError: 'A' object has no attribute 'test'
>>> def test(self):
print id(self)
>>> A.test = test
>>> a.test()
13111376
##### code ends here #####
--
TZOTZIOY, I speak England very best,
Microsoft Security Alert: the Matrix began as open source.
Jacek Generowicz
> Again, I wanted to focus the discussion for a moment to help alleviate
> people's fear of Macro's, as they seem to be based on
> mis-understanding what they are about. Please forgive my over
> simplification.
I was merely pointing out that I think that the security issue is
completely orthogonal.
> If you hate them that much, we can forget I ever posted them. :-p
But Google won't :-)
Jacek Generowicz
> On 24 Aug 2003 16:34:08 +0200, rumours say that Jacek Generowicz
> <jacek.ge...@cern.ch> might have written:
>
> >The first thing you might try to do in Python is to add the fubar
> >method to the class source code, and re-evaluate it ... only to find
> >that your existing instances know nothing about the new method ... so
> >you have to go through the hassle of re-creating a gazillion of them,
> >before you can continue with what you were doing.
>
> ##### code starts here #####
>
> >>> class A(object):
> pass
>
> >>> a= A()
> >>> a.test()
> Traceback (most recent call last):
> File "<pyshell#21>", line 1, in ?
> a.test()
> AttributeError: 'A' object has no attribute 'test'
> >>> def test(self):
> print id(self)
>
>
> >>> A.test = test
> >>> a.test()
> 13111376
>
> ##### code ends here #####
No, you are missing the point. The point is that you want your source
code to be in sync with your program state, and you want your source
code to look natural. By "natural" I mean that you want your source
code to look like this:
class A(object):
def test(self):
print id(self)
rather than like this:
class A(object):
pass
def test(self):
print id(self)
A.test = test
If you still don't understand the motivation, please read the original
thread; there's no point in repeating the arguments all over again in
this thread, which is about something completely different.
John J. Lee
> j...@pobox.com (John J. Lee) writes:
>
> > I'm sure you're capable of understanding this position, but the
> > argument has started going round in circles.
>
> As I've said repeatedly:
>
> - I do _not_ necessarily advocate the inclusion of macros in Python.
But you do advocate them in other languages, right? And that was a
significant part of the argument.
> - I merely wish to dispel the myth that macros make code less
> readable[*]: they _ease_ understanding of the code.
>
>
> [*] And they don't fragment languages either.
John
Jacek Generowicz
> Jacek Generowicz <jacek.ge...@cern.ch> writes:
>
> > - I do _not_ necessarily advocate the inclusion of macros in Python.
>
> But you do advocate them in other languages, right? And that was a
> significant part of the argument.
You've lost me.
What I advocate is that, in the process of constructing an FAQ or
article about macros in Python, one should try to exclude material
which sounds like this:
Q: What is static typing, and why isn't it included in Python?
A: Static typing is the requirement that the programmer write his
code while standing up. Some people believe that this causes
infertility.
Or, to make a closer analogy:
Q: What are functions, and why aren't they included in Assembler ?
A: Functions are a very powerful means of creating
abstarctions. Some people believe that functions make a program
more difficult to understand. Functions also cause languages to
split, as evidenced by the existence of many different languages
which support functions. If any programmer on your project writes
a function, random parts of the code _you_ wrote will stop working;
shortly thereafter frogs will rain from the sky for forty days.
Put another way, I advocate avoiding FUD.
To this end, I have tried to provide information which would help
readers of this list, who are unfamiliar with Lisp-style macros, to
understand what Lisp-style macros are.
If I am involved in an argument, then it is an argument about what
Lisp-style macros _are_ and what they can or can't do ... completely
independently of any language in which they may or may not be
available, now or in the future.
I offer no opinion on whether Lisp-style macros are a good idea. I
would like people to form their own opinions on the matter, but I
would like those opinions to be based on fact rather than on FUD.
Now, what is it that _you_ are arguing ?
Hannu Kankaanpää
> ==== Example 1: a debugging aid ================================
Here's another example of a real life situation
where I thought it'd be good to have macros in Python.
I was writing a GUI system and I had to define lots of
code twice, both for the x-axis and y-axis. Such
as
if y > self.y - bs and y < self.y + bs + self.height:
return (abs(x - self.x) < bs,
abs(x - (self.x + self.width)) < bs)
else:
return False, False
and then
if x > self.x - bs and x < self.x + bs + self.width:
return (abs(y - self.y) < bs,
abs(y - (self.y + self.height)) < bs)
else:
return False, False
Obviously this was quite unsatisfactory. I ended up
putting the axis code in a separate class so I could
use them interchangeably. I.e. If I passed
func(self.y, self.x)
and then
func(self.x, self.y)
I would get the same effect on both axises. But this
would've been an excellent place for macros IMO (unless
there's a more clever solution as a whole). Using macros
that combine both function call and "code block" syntax,
I could've written a simple function like this:
defBoth getSize(self):
return self.size
And it would've been expanded to
def getWidth(self):
return self.width
def getHeight(self):
return self.height
The macro would've had to change all "size" to either
"width" or "height", and also change "pos" to either "x"
or "y" and so on.
This way, I could've got no duplicated code but
also a more intuitive interface than I currently have
(to get width, one needs to type obj.x.getSize() instead
of obj.getWidth()). And it's obvious this kind of "defBoth"
wouldn't be added as a language level construct -- Thus
macros are the only good solution.
Jacek Generowicz
> if y > self.y - bs and y < self.y + bs + self.height:
> return (abs(x - self.x) < bs,
> abs(x - (self.x + self.width)) < bs)
> else:
> return False, False
>
>
> and then
>
>
> if x > self.x - bs and x < self.x + bs + self.width:
> return (abs(y - self.y) < bs,
> abs(y - (self.y + self.height)) < bs)
> else:
> return False, False
>
> Obviously this was quite unsatisfactory. I ended up
> putting the axis code in a separate class
Do you mean "function" rather than "class" ?
> so I could
> use them interchangeably. I.e. If I passed
> func(self.y, self.x)
> and then
> func(self.x, self.y)
>
> I would get the same effect on both axises. But this
> would've been an excellent place for macros IMO
I don't see what you gain by using a macro, wrt to using a function in
_this_ case. As no Python macro system exists, I cannot form an
opinion of what the two approaches would look like in Python, but in
Lisp the calls to the macro would look just like the calls to the
function, I think. The only difference would be in the definition of
the macro/function, and the macro would be no simpler, so you wouldn't
really gain anything.
> Using macros
> that combine both function call and "code block" syntax,
> I could've written a simple function like this:
>
> defBoth getSize(self):
> return self.size
>
> And it would've been expanded to
>
> def getWidth(self):
> return self.width
>
> def getHeight(self):
> return self.height
>
> The macro would've had to change all "size" to either
> "width" or "height", and also change "pos" to either "x"
> or "y" and so on.
So you would not only replace whole symbols, but even fragments of
symbols (getSize -> getHeight), and thus macically/implicitly create
new symbols. Many people consider this bad practice. The objection
being that readers of the code come across a symbol in the source, go
off and search for its definition, and tricks like this mean that they
end up wasting their time.
The defstruct macro does this sort of thing; (defstruct foo ... )
would "magically" define a constructor called "make-foo" (and other
stuff). IIRC, the experience with defstruct led the designers of CLOS
to explicitly avoid doing this in CLOS' defclass macro (but I'm far
from the world's leading authority on Lisp history).
Incidentally, in this case, having a string based code representation
would make your job much easier than the structured representation
which Lisp uses. In a string you'd merely do a search and replace; in
an s-expression you would have to recursively search for all symbols
in all sub-expressions, and then do the search and replace within the
name of each symbol you find.
> This way, I could've got no duplicated code but
> also a more intuitive interface than I currently have
> (to get width, one needs to type obj.x.getSize() instead
> of obj.getWidth()). And it's obvious this kind of "defBoth"
> wouldn't be added as a language level construct -- Thus
> macros are the only good solution.
Cue metaclass solution ... :-)
(I suspect that even with metaclasses, you wouldn't be able to avoid
eval ... and that amounts to "informally" writing a macro)
Hannu Kankaanpää
> hanz...@yahoo.com.au (Hannu Kankaanpää) writes:
> > Obviously this was quite unsatisfactory. I ended up
> > putting the axis code in a separate class
>
> Do you mean "function" rather than "class" ?
Actually, I did mean class. Normally I'd have
class Widget:
def __init__(self):
self.x = 0
self.y = 0
self.width = 0
self.height = 0
def getWidth(self):
return self.width # normally they wouldn't be this empty!
def getHeight(self):
return self.height
But by wrapping it inside a new class, I could get rid of
the duplication (partly, at least):
class Widget:
def __init__(self):
self.x = Widget.Axis(0, 0)
self.y = Widget.Axis(0, 0)
class Axis:
def __init__(self, pos, size)
self.pos = pos
self.size = size
def getSize(self): # this is only once here now
return self.size
While this tiny example doesn't show decrease in code
size, it shows that I have a common definition for all
"Axis"-specific code inside the appropriate Axis class.
Rest of the Widget methods would be in the widget class.
Thus self.x.getSize() instead of self.getWidth().
> > so I could
> > use them interchangeably. I.e. If I passed
> > func(self.y, self.x)
> > and then
> > func(self.x, self.y)
> >
> > I would get the same effect on both axises. But this
> > would've been an excellent place for macros IMO
>
> I don't see what you gain by using a macro, wrt to using a function in
> _this_ case.
Ok, it was a bad example. I hope the code above shows
a bit more clearly what I wanted. Anyway, without the code
in the axis-class, I would've had to often say
self.x = func(self.x, self.y, self.width, self.height)
self.y = func(self.y, self.x, self.height, self.width)
Instead of
func(self.x, self.y)
func(self.y, self.x)
Which could modify the axis-specific stuff within the
func()tion. (self.x is no longer a non-modifiable number,
but a modifiable class)
> So you would not only replace whole symbols, but even fragments of
> symbols (getSize -> getHeight), and thus macically/implicitly create
> new symbols. Many people consider this bad practice.
Well, I don't, really. Like any macro that could do something
weird, it just needs to be properly understood by anyone who
wishes to read the code.
> Incidentally, in this case, having a string based code representation
> would make your job much easier than the structured representation
> which Lisp uses. In a string you'd merely do a search and replace; in
> an s-expression you would have to recursively search for all symbols
> in all sub-expressions, and then do the search and replace within the
> name of each symbol you find.
Well, such a recursive search isn't a problem - With help from
a macro ;)
> > This way, I could've got no duplicated code but
> > also a more intuitive interface than I currently have
> > (to get width, one needs to type obj.x.getSize() instead
> > of obj.getWidth()). And it's obvious this kind of "defBoth"
> > wouldn't be added as a language level construct -- Thus
> > macros are the only good solution.
>
> Cue metaclass solution ... :-)
How could metaclasses help? I'm quite inexperienced with them.
Anyway, if I take the eval route, I might as well do
defBoth('''getSize():
return size''')
,retreive appropriate locals() from the stack and modify it
to include the new functions. I'd rather not, though :-)