Getting in to metaprogramming
Rafe
In the name of self-education can anyone share some pointers, links,
modules, etc that I might use to begin learning how to do some
"metaprogramming". That is, using code to write code (right?)
Cheers,
- Rafe
Michele Simionato
The word "metaprogramming" has many meanings. It covers code
generation, as you say,
but also a lots of other techniques; for instance you do
metaprogramming with decorators
when write functions operating over functions, or with metaclasses
when you write classes
operating over classes. You should clarify what you are interested in.
For instance, are
you interested in metaprogramming in Python only or you want a more
general introduction?
Have you read the Wikipedia entry?
http://en.wikipedia.org/wiki/Metaprogramming
Aaron Brady
Python programs can generate code for themselves.
>>> for i in range( 10 ):
... d= { 'cls': i }
... s="""
... class Cls%(cls)s:
... def meth%(cls)s( self, arg ):
... print 'in meth%(cls)s, arg:', arg
... """% d
... exec( s )
... s= """
... inst%(cls)s= Cls%(cls)s()
... """% d
... exec( s )
...
>>> inst0.meth0( "arg" )
in meth0, arg: arg
>>> inst1.meth1( "arg" )
in meth1, arg: arg
>>> inst2.meth2( "arg" )
in meth2, arg: arg
The 'Cls0', 'Cls1', 'Cls2' repetitiveness is taken care of with a for-
loop.
Rafe
Michele, I am thinking about python which writes python. Which makes
Aaron's post accurate to my needs. More specifically, I am considering
what it might be like to use python to build a script file which can
be executed later. Until now I planned to store info to XML and then
parse it to run later. There are good reasons that generating a script
file would be more useful for me.
Aaron, Is it really as simple as gathering strings of code? Sort of
like generating HTML or XML directly? Is there any other framework or
pattern set that is worth looking in to?
Thanks for helping me explore this.
- Rafe
Michele Simionato
> is it really as simple as gathering strings of code?
Yes.
> Sort of like generating HTML or XML directly? Is there any other framework or
> pattern set that is worth looking in to?
Yes, the compiler module and the ast module in the standard library.
You may also
look at how templating languages that compile to Python code work (I
mean mako
or PTL/qpy). You may find interesting how the import hooks work too.
Stef Mientki
this:
GUI = """
self.NB ,wx.Notebook ,style = wx.NO_BORDER
self.Grid ,Base_Table_Grid ,None, data_values, data_types, data_defs
Panel2 ,PanelVer, 11 ,name = "Page2"
list1 ,wx.ListCtrl ,style = wx.LC_REPORT
"""
cheers,
Stef
> Aaron, Is it really as simple as gathering strings of code? Sort of
> like generating HTML or XML directly? Is there any other framework or
> pattern set that is worth looking in to?
>
> Thanks for helping me explore this.
>
> - Rafe
> http://mail.python.org/mailman/listinfo/python-list
>
Aaron Brady
wrote:
Yes.
You could have Python generate HTML, Python generate COBOL, Python
generate Python, Perl generate Python, sure. Not that I know COBOL.
If you're generating Python with Python, you only need one file to do
it, not two separate programs, as you would for the other
combinations; it was my point. It could be really helpful or
unnecessary, depending on your application.
There are some templating modules, possibly for Python code. If
anyone knows them, they can point them out.
I don't know a clean, reliable way to structure a metaprogram though.
Mine always turn out messy.
Arnaud Delobelle
> I don't know a clean, reliable way to structure a metaprogram though.
> Mine always turn out messy.
I don't think Python is designed for large scale metaprogramming. Lisp
is the only naturally suited language that I know.
--
Arnaud
Hendrik van Rooyen
>I don't know a clean, reliable way to structure a metaprogram though.
>Mine always turn out messy.
Yes.
Then another thing - it strikes me that any problem that can be solved
by metaprogramming, can be solved by putting similar code into a class
and instanciating an instance.
Does anybody know if this is true?
If it is, it limits the usefulness of metaprogramming to the creation
of "stored procedures" for "later execution".
- Hendrik
Michele Simionato
wrote:
> Then another thing - it strikes me that any problem that can be solved
> by metaprogramming, can be solved by putting similar code into a class
> and instanciating an instance.
>
> Does anybody know if this is true?
>
> If it is, it limits the usefulness of metaprogramming to the creation
> of "stored procedures" for "later execution".
>
> - Hendrik
Many times (probably most times) you can avoid code generation and use
classes or
higher order functions instead. Actually I don't like code generation
techniques
in Python and other languages. I would make an exception for Lisp-like
languages,
however, since there you a pretty powerful macro mechanism that in
some cases can be
better than using higher order functions, especially if performance is
important.
I say something more in the latest episode of my "Adventures of a
Pythonista in Schemeland"
which some of you may find interesting: http://www.artima.com/weblogs/viewpost.jsp?thread=240836
M. Simionato
Steven D'Aprano
> "Aaron Brady" <casti...@gmail.com> wrote:
>
>
>>I don't know a clean, reliable way to structure a metaprogram though.
>>Mine always turn out messy.
>
> Yes.
>
> Then another thing - it strikes me that any problem that can be solved
> by metaprogramming, can be solved by putting similar code into a class
> and instanciating an instance.
>
> Does anybody know if this is true?
Well, I don't know about "any problem". And it's not so much about
whether metaprograms can solve problems that can't be solved by anything
else, as whether metaprograms can solve problems more effectively than
other techniques.
If you include factory functions, class factories, the builder design
pattern, metaclasses, etc. as "metaprogramming", then I use it all the
time, and find it an excellent technique to use.
But if you mean using a Python program to generate Python source code,
then I can't think of any time I used it. Which doesn't mean that others
don't find it helpful, only that I haven't yet.
Thinking further back, when I was young and programming in Apple's
Hypercard 4GL, I used to frequently use Hypercard scripts to generate new
Hypercard scripts. That was to work around the limitations of the
scripting language.
I don't think metaprogramming in the limited sense (programs to output
source code) is a bad idea, but I do think that there are probably better
alternatives in a language like Python.
--
Steven
Kay Schluehr
Hendrik van Rooyen
"Steven D'Aprano" <steau> wrote:
>
> Well, I don't know about "any problem". And it's not so much about
> whether metaprograms can solve problems that can't be solved by anything
> else, as whether metaprograms can solve problems more effectively than
> other techniques.
>
> If you include factory functions, class factories, the builder design
> pattern, metaclasses, etc. as "metaprogramming", then I use it all the
> time, and find it an excellent technique to use.
>
> But if you mean using a Python program to generate Python source code,
> then I can't think of any time I used it. Which doesn't mean that others
> don't find it helpful, only that I haven't yet.
I am using the term in the restricted sense of Python writing Python source.
Given that, can anybody think of an example that you could not do with
a class? (excepting the "stored procedure" aspect)
Or can I claim a new a new meta - rule - I would call it van Rooyen's folly...
>
> Thinking further back, when I was young and programming in Apple's
> Hypercard 4GL, I used to frequently use Hypercard scripts to generate new
> Hypercard scripts. That was to work around the limitations of the
> scripting language.
What sort of stuff did you do, and would having had simple OO available
have rendered it unnecessary?
>
> I don't think metaprogramming in the limited sense (programs to output
> source code) is a bad idea, but I do think that there are probably better
> alternatives in a language like Python.
>
>
True. No argument here - I was just wondering if the relationship holds.
- Hendrik
Kay Schluehr
> Given that, can anybody think of an example that you could not do with
> a class? (excepting the "stored procedure" aspect)
I just noticed that corepy 1.0 [1] has been released. Corepy is an
embedded DSL for synthesizing machine code from chaining Python
commands. This means it provides objects and exploits control
structures used to create machine code that can finally be executed
interactively.
Let's say you have an ordinary Python function that computes a CRC 32.
Now you could attempt to translate the function into other Python code
that expresses a corepy routine. You could create a decorator that
works as follows
1) reads the source of the decorated function
2) transforms the source into corepy source and compiles it or
3) if 2) fails it just returns the passed code object.
Kay
Terry Reedy
> I am using the term in the restricted sense of Python writing Python source.
>
> Given that, can anybody think of an example that you could not do with
> a class? (excepting the "stored procedure" aspect)
I am not sure I understand your question.
def iterize(recursive_function_text):
<code to parse input and fill a template>
return equivalent_iterative_function_text
where input and output are both Python code. If one were to implement
this by compiling the input to AST form and then walking the tree, the
AST node classes would be involved, but I would scarely say the
translation was done by the classes, as opposed to functions which might
or might not be attacked to a class as methods.
tjr
Aaron Brady
wrote:
> "Steven D'Aprano" <steau> wrote:
>
>
>
> > Well, I don't know about "any problem". And it's not so much about
> > whether metaprograms can solve problems that can't be solved by anything
> > else, as whether metaprograms can solve problems more effectively than
> > other techniques.
>
> > If you include factory functions, class factories, the builder design
> > pattern, metaclasses, etc. as "metaprogramming", then I use it all the
> > time, and find it an excellent technique to use.
>
> > But if you mean using a Python program to generate Python source code,
> > then I can't think of any time I used it. Which doesn't mean that others
> > don't find it helpful, only that I haven't yet.
>
> I am using the term in the restricted sense of Python writing Python source.
>
> Given that, can anybody think of an example that you could not do with
> a class? (excepting the "stored procedure" aspect)
>
> Or can I claim a new a new meta - rule - I would call it van Rooyen's folly...
The example I think of is the Visitor Pattern of Gamma and all. One
class's method calls another's method with its own class's name in the
name.
class Visitor:
def visit_A( self, arg ):...
def visit_B( self, arg ):...
class A:
def visit( self, vis ):
vis.visit_A( self )
class B:
def visit( self, vis ):
vis.visit_B( self )
As you can see, the 'visit' method is mechanical for classes A and B.
One might want to autogenerate those in some languages, but Python has
introspection:
class BaseAB:
def visit( self, vis ):
getattr( vis, 'visit_%s'% self.__class__.__name__ )( self )
And it's easier to modify the default behavior this way than in
autogenerated code too.
Steven D'Aprano
> I am using the term in the restricted sense of Python writing Python
> source.
>
> Given that, can anybody think of an example that you could not do with a
> class? (excepting the "stored procedure" aspect)
GUI designer. You write a program to let the user create code by clicking
buttons, dragging objects, drawing lines, etc. The GUI designer may use
classes, but the purpose of those classes is to generate source code.
Testing code speed... you might have some functions with a loop, and you
want to unroll the loop as an optimization. If you have one function, you
can unroll it yourself. If you have a hundred such functions, you might
want to write a program to do it. (Yes, I'm stretching...)
Don't like that Python doesn't optimize tail-recursion? Then write a
source-code analyzer that detects tail-recursion and re-writes the
function using a while loop.
>> Thinking further back, when I was young and programming in Apple's
>> Hypercard 4GL, I used to frequently use Hypercard scripts to generate
>> new Hypercard scripts. That was to work around the limitations of the
>> scripting language.
>
> What sort of stuff did you do, and would having had simple OO available
> have rendered it unnecessary?
It's been 20-odd years, and the examples were pretty trivial... I don't
really recall exactly, but it would have been something like this:
* design a GUI involving lots of buttons on screen, each one with quite
similar but not identical code;
* since Hypercard didn't have a layout manager, write a script to
generate each button, place it where needed, and set the button's code.
Hypercard did have a message passing hierarchy (like inheritance for
objects), so often you could take the button's code and place it in a
higher level of the hierarchy (the card, the background, the stack), but
there were odd cases where that wasn't enough.
Another example: Hypercard had a very limited number of GUI elements
(text fields and buttons, basically) but I designed a slider control
using a few buttons, each button with a custom script. To avoid needing
to create and place the buttons by hand each time I wanted a slider, I
had a script that did it for me. The script not only created the buttons,
but it created the scripts used by the buttons. This wasn't as difficult
as it sounds -- it was basically taking a template and doing some text
replacements, then telling the button to use it as a script.
--
Steven
Aaron Brady
<ste...@REMOVE.THIS.cybersource.com.au> wrote:
> On Thu, 27 Nov 2008 06:41:47 +0200, Hendrik van Rooyen wrote:
> > I am using the term in the restricted sense of Python writing Python
> > source.
>
> > Given that, can anybody think of an example that you could not do with a
> > class? (excepting the "stored procedure" aspect)
>
> GUI designer. You write a program to let the user create code by clicking
> buttons, dragging objects, drawing lines, etc. The GUI designer may use
> classes, but the purpose of those classes is to generate source code.
I want to disagree with this example. I hold that the proper output
of a GUI designer is in XML, or another data format.
> Testing code speed... you might have some functions with a loop, and you
> want to unroll the loop as an optimization. If you have one function, you
> can unroll it yourself. If you have a hundred such functions, you might
> want to write a program to do it. (Yes, I'm stretching...)
>
> Don't like that Python doesn't optimize tail-recursion? Then write a
> source-code analyzer that detects tail-recursion and re-writes the
> function using a while loop.
I've seen a preprocessor come up a few times. The input is a
program's source, and the output is source. Of course you can do it
with quotes and 'exec' at run-time, but if your whole file is quoted,
it may be a better option.
(Part of my rationale was I wanted syntax-coloring.)
> >> Thinking further back, when I was young and programming in Apple's
> >> Hypercard 4GL, I used to frequently use Hypercard scripts to generate
> >> new Hypercard scripts. That was to work around the limitations of the
> >> scripting language.
>
> > What sort of stuff did you do, and would having had simple OO available
> > have rendered it unnecessary?
>
> It's been 20-odd years, and the examples were pretty trivial... I don't
> really recall exactly, but it would have been something like this:
>
> * design a GUI involving lots of buttons on screen, each one with quite
> similar but not identical code;
>
> * since Hypercard didn't have a layout manager, write a script to
> generate each button, place it where needed, and set the button's code.
snip
(Above.) I'm not sure if you'd count something like 'perlmodule':
import perl
# Simple arithmetics
six = perl.eval("3+3")
# Eval can also return functions
sum = perl.eval("sub { my $s = shift; $s += shift while @_; $s }")
print sum(1,2,3)
http://aspn.activestate.com/ASPN/CodeDoc/pyperl/perlmodule.html
but speaking of quoted code and all...
Rafe
wrote:
Generating a template for a specific script application. For example,
a script with pre-defined callbacks that only require the addition of
the contents.
I was really interested in exploring the idea of using python output,
instead of XML, to record something a user did in a GUI. I have seen
it done and it is really advantageous in the 3D industry because it
means the script files can be edited directly, in a pinch, to generate
something slightly different.
For example, say we have code which generates a cube by plotting it's
points. A user then changes a point position in the GUI. The change is
saved by outputting the function call to a file with new arguments
(the new point location). If I wanted to make 100s of copies of the
cube, but with a slightly different point position, I could edit the
custom cube's python code and hand it back for creation without using
the GUI. I could do this with XML, but it can be harder to work with
in a text editor (though I have seen some XML editors that make it a
bit easier.) In fact, in most 3D applications, the app prints
everything the user does to a log. Sometimes in a choice of languages,
so I guess I am looking to do the same thing with my own custom tools.
In a real situation the generated code file can build some pretty
complex 3D object hierarchies. It moves beyond simple storage of data
and becomes a real script that can be hacked as necessary.
It is nice to have everything as python scripts because we always have
a blend of GUI users and developers to get a job done.
- Rafe
Rafe
I was just thinking (hopefully i get some time to try this soon) that
it wouldn't be difficult to decorate a function so that when called, a
line of code is output. as long as the arguments can be stored as a
string (strings, numbers, lists, etc. but no 'object' instances) it
should be able to be executed to get the same result. I think it would
just have to:
1) Dynamically write the name with a '('
2) Gather all the args in a list and ", ".join(args)
3) Gather kwargs as a list of ['%s = %s' % key, value] and then and ",
".join(kwlist)
4) Add ')\n'
- Rafe
Lawrence D'Oliveiro
> In the name of self-education can anyone share some pointers, links,
> modules, etc that I might use to begin learning how to do some
> "metaprogramming".
Fred Brooks, in his classic "Mythical Man-Month", defined "metaprogramming" as simply a very high-level form of programming, using a language designed to call entire programs as opposed to procedures/subroutines.
He quoted AppleScript as a good example (this was back in 1995), but Perl would have been just as good. Or these days, even better, Python.
Michele Simionato
> Given that, can anybody think of an example that you could not do with
> a class? (excepting the "stored procedure" aspect)
The namedtuple recipe by Raymond Hettinger (http://
code.activestate.com/recipes/500261)
is an interesting example of code generation. My own decorator module
use a similar
trick. Here code generation (plus eval/exec) is needed since you need
control on
the signature of a function. If we could change the signature of a
function (I think
this is possible in Python 3.0, I have to check) then we would not
need code
generation.
Hendrik van Rooyen
This is kind of the wrong way around - this is an example of
OO jiggery pokery that will be difficult to do by generating the
source code - are there things that can be done by generating
the source code that cannot be done with OO?
- Hendrik
Hendrik van Rooyen
> Hendrik van Rooyen wrote:
>
> > I am using the term in the restricted sense of Python writing Python source.
> >
> > Given that, can anybody think of an example that you could not do with
> > a class? (excepting the "stored procedure" aspect)
>
> I am not sure I understand your question.
>
> def iterize(recursive_function_text):
> <code to parse input and fill a template>
> return equivalent_iterative_function_text
>
> where input and output are both Python code. If one were to implement
> this by compiling the input to AST form and then walking the tree, the
> AST node classes would be involved, but I would scarely say the
> translation was done by the classes, as opposed to functions which might
> or might not be attacked to a class as methods.
>
I am not sure I understand the answer - if the input and output are both
bits of python source, then the output must be stored and evaluated or
imported to get it executed, right?
Now if this is the case, my question has to do with whether it is always
possible to put the equivalent code in a class, and instantiating it so that
it is either immediately executed or ready for immediate execution. My
gut feel tells me that it should always be possible, but I have often had
wrong hunches.
Not sure if this makes it any clearer.
- Hendrik
Hendrik van Rooyen
"Steven D'Aprano" <ste...@REM.com.au>
>GUI designer. You write a program to let the user create code by clicking
>buttons, dragging objects, drawing lines, etc. The GUI designer may use
>classes, but the purpose of those classes is to generate source code.
>
Yikes, this is getting hairy- If "the problem" is to generate source code,
then you have to generate source code...
>Testing code speed... you might have some functions with a loop, and you
>want to unroll the loop as an optimization. If you have one function, you
>can unroll it yourself. If you have a hundred such functions, you might
>want to write a program to do it. (Yes, I'm stretching...)
Ok this one I'll buy - I can't think of a way to do this dynamically in a
class and get runnable code back. (but maybe I'm just not trying hard enough.)
>
>Don't like that Python doesn't optimize tail-recursion? Then write a
>source-code analyzer that detects tail-recursion and re-writes the
>function using a while loop.
This is like TJR's example (I think)
>>> Thinking further back, when I was young and programming in Apple's
>>> Hypercard 4GL, I used to frequently use Hypercard scripts to generate
>>> new Hypercard scripts. That was to work around the limitations of the
>>> scripting language.
>>
>> What sort of stuff did you do, and would having had simple OO available
>> have rendered it unnecessary?
>
>It's been 20-odd years, and the examples were pretty trivial... I don't
>really recall exactly, but it would have been something like this:
>
>* design a GUI involving lots of buttons on screen, each one with quite
>similar but not identical code;
>
>* since Hypercard didn't have a layout manager, write a script to
>generate each button, place it where needed, and set the button's code.
>
>Hypercard did have a message passing hierarchy (like inheritance for
>objects), so often you could take the button's code and place it in a
>higher level of the hierarchy (the card, the background, the stack), but
>there were odd cases where that wasn't enough.
>
>Another example: Hypercard had a very limited number of GUI elements
>(text fields and buttons, basically) but I designed a slider control
>using a few buttons, each button with a custom script. To avoid needing
>to create and place the buttons by hand each time I wanted a slider, I
>had a script that did it for me. The script not only created the buttons,
>but it created the scripts used by the buttons. This wasn't as difficult
>as it sounds -- it was basically taking a template and doing some text
>replacements, then telling the button to use it as a script.
>
Ok I think I am beginning to get the picture - when you have to do stuff
that the language does not directly support, then you use the simple
available elements, and create source code that strings them together
to make more complex stuff. -in this case its probably not trivial to
do it at run time.
The "make the source code" then run it, introduces a kind of compiler
stage.
For an old assembler programmer, this is starting to sound like macros.
So a different meta law would read like:
One uses Code Generation Techniques when the language does not
have macros.
*ducks*
- Hendrik
Hendrik van Rooyen
"Kay Schluehr" <kay.sc...@gmx.net> wrote:
>
> I just noticed that corepy 1.0 [1] has been released. Corepy is an
> embedded DSL for synthesizing machine code from chaining Python
> commands. This means it provides objects and exploits control
> structures used to create machine code that can finally be executed
> interactively.
>
> Let's say you have an ordinary Python function that computes a CRC 32.
> Now you could attempt to translate the function into other Python code
> that expresses a corepy routine. You could create a decorator that
> works as follows
>
> 1) reads the source of the decorated function
> 2) transforms the source into corepy source and compiles it or
> 3) if 2) fails it just returns the passed code object.
>
> Kay
>
> [1] http://www.corepy.org/
Thanks for the link.
This stuff sounds powerful - I will try to wrap my head around it.
- Hendrik
Hendrik van Rooyen
>The namedtuple recipe by Raymond Hettinger (http://
>code.activestate.com/recipes/500261)
>is an interesting example of code generation. My own decorator module
>use a similar
>trick. Here code generation (plus eval/exec) is needed since you need
>control on
>the signature of a function. If we could change the signature of a
>function (I think
>this is possible in Python 3.0, I have to check) then we would not
>need code
>generation.
Right that is a definite counterexample.
van Rooyen's folly is officially dead.
It is stillborn.
It has snuffed it.
It lives no more.
Thanks.
- Hendrik
Terry Reedy
> "Terry Reedy" <tjr...@udel.edu> wrote:
>
>
>> Hendrik van Rooyen wrote:
>>
>>> I am using the term in the restricted sense of Python writing Python source.
>>>
>>> Given that, can anybody think of an example that you could not do with
>>> a class? (excepting the "stored procedure" aspect)
>> I am not sure I understand your question.
>>
>> def iterize(recursive_function_text):
>> <code to parse input and fill a template>
>> return equivalent_iterative_function_text
>>
>> where input and output are both Python code. If one were to implement
>> this by compiling the input to AST form and then walking the tree, the
>> AST node classes would be involved, but I would scarely say the
>> translation was done by the classes, as opposed to functions which might
>> or might not be attacked to a class as methods.
>>
>
> I am not sure I understand the answer - if the input and output are both
> bits of python source, then the output must be stored and evaluated or
> imported to get it executed, right?
Yes. I would couple the above with
def iterfunc(rtext):
itext = iterize(rtext)
<code to exec itext to make function object>
return ifunc
The text transform is the hard part and should be separate for testing
and possible output for optimization or reuse elsewhere.
The scenario I have in mind is
1. write program with recursive function and test
2. triple-quote function text and wrap with function call; retest.
Yes, Steven's tail recursion example is example of above.
> Now if this is the case, my question has to do with whether it is always
> possible to put the equivalent code in a class, and instantiating it so that
> it is either immediately executed or ready for immediate execution. My
> gut feel tells me that it should always be possible, but I have often had
> wrong hunches.
>
> Not sure if this makes it any clearer.
I am not sure what you mean by 'do with a class'. A class is an active
namespace in that it creates and initializes instances and does a bit
extra with method calls, but it is otherwise a passive namespace container.
Any code can be put into class methods, but I do not see any reason to
do so for this example.
Terry Jan Reedy
Kay Schluehr
A rather advanced approach to deal with "writing XML in Python" and
generating "Python from XML" has been chosen in P4D:
http://pypi.python.org/pypi/P4D%20Langlet/1.2.4
It is of course a cultural mismatch to do such stuff because it
targets a proper superset of Python but I don't think it's worse than
what has been done in a plethora of template languages ( P4D is none
of them though ), YAML or related approaches. I did it mostly because
I wanted to learn Adobe Flex and I wanted to use it as a scripting
language, not as something I had to edit in a particular editor like
FlexBuilder.
Steven D'Aprano
> "Terry Reedy" <tjr...@udel.edu> wrote:
>
>
>> Hendrik van Rooyen wrote:
>>
>> > I am using the term in the restricted sense of Python writing Python
>> > source.
>> >
>> > Given that, can anybody think of an example that you could not do
>> > with a class? (excepting the "stored procedure" aspect)
>>
>> I am not sure I understand your question.
>>
>> def iterize(recursive_function_text):
>> <code to parse input and fill a template> return
>> equivalent_iterative_function_text
>>
>> where input and output are both Python code. If one were to implement
>> this by compiling the input to AST form and then walking the tree, the
>> AST node classes would be involved, but I would scarely say the
>> translation was done by the classes, as opposed to functions which
>> might or might not be attacked to a class as methods.
>>
>>
> I am not sure I understand the answer - if the input and output are both
> bits of python source, then the output must be stored and evaluated or
> imported to get it executed, right?
I think there's some confusion here. Python, like all general-purpose
programming languages, is able to process text. Python source code, like
that of almost all programming languages, is text.
Therefore Python can process Python source code as text: it can read it
and process it, or it can write it. There's nothing mysterious about
this, although how useful it is depends on the nature of the processing.
Once you have that Python source code, it is no different whether you
wrote it by hand, or wrote it with the aid of another program. To execute
it, you have to execute it.
Here's a trivial example of metaprogramming. Suppose I am interested in
the compilation time of Python code, rather than the execution time. I
might do this:
numitems = 100000
template = """def main():
# %s
%s%s
return L
"""
def make_file(name, comment, setup, body):
f = open(name, 'w')
f.write(template % (comment, setup, body))
f.close()
body = 'L.append(None)\n'
make_file('dumb.py', 'Create a big list the dumb way.',
'L = []\n', body*numitems)
make_file('smart.py', 'Create a big list the smart way.',
'', 'L = [None]*%d\n' % numitems)
At this point, I have two source files which I can do anything I like
with. It would have been a real pain to have written the first one by
hand, even with a clever editor. I can post-process them, run them
through other tools, even open them up in an editor and manipulate them
by hand (although that defeats the purpose of metaprogramming). Now I can
run the source files as input to another piece of code, and get a (very
rough) estimate of the compilation time:
from time import time
start = time()
execfile('dumb.py') # bypass the import mechanism
tdumb = time() - start
print "Time taken to compile dumb.py is %f seconds" % tdumb
start = time()
execfile('smart.py')
tsmart = time() - start
print "Time taken to compile smart.py is %f seconds" % tsmart
On my computer, I get:
Time taken to compile dumb.py is 2.236122 seconds
Time taken to compile smart.py is 0.003754 seconds
--
Steven
alex23
> Then another thing - it strikes me that any problem that can be solved
> by metaprogramming, can be solved by putting similar code into a class
> and instanciating an instance.
>
> Does anybody know if this is true?
>
> If it is, it limits the usefulness of metaprogramming to the creation
> of "stored procedures" for "later execution".
Take a look at Pythoscope, which generates unit test stubs for a given
piece of Python code:
Aaron Brady
> "Aaron Brady" <castiro...@gmail.com> wrote:
...
> >As you can see, the 'visit' method is mechanical for classes A and B.
> >One might want to autogenerate those in some languages, but Python has
> >introspection:
>
> >class BaseAB:
> > def visit( self, vis ):
> > getattr( vis, 'visit_%s'% self.__class__.__name__ )( self )
>
> >And it's easier to modify the default behavior this way than in
> >autogenerated code too.
>
> This is kind of the wrong way around - this is an example of
> OO jiggery pokery that will be difficult to do by generating the
> source code - are there things that can be done by generating
> the source code that cannot be done with OO?
Yes, I was examining the special case of generating OO code.
Aaron Brady
cybersource.com.au> wrote:
snip
> body = 'L.append(None)\n'
> make_file('dumb.py', 'Create a big list the dumb way.',
> 'L = []\n', body*numitems)
> make_file('smart.py', 'Create a big list the smart way.',
> '', 'L = [None]*%d\n' % numitems)
>
> At this point, I have two source files which I can do anything I like
> with. It would have been a real pain to have written the first one by
> hand, even with a clever editor. I can post-process them, run them
> through other tools, even open them up in an editor and manipulate them
> by hand (although that defeats the purpose of metaprogramming).
I tempt that one's metaprogram is in a metalanguage, and if it's
generating two programs, it has twice the power of the language it's
in, giving it a leverage of two units.
I feel like I always do that, manipulate the metaprogram output by
hand. I forget my use cases though, except for one that did low-level
structure handling.
P.S. tempt: '5. Obsolete. to try or test.' http://dictionary.reference.com/browse/tempt
Eric_...@msn.com
>
> - Show quoted text -
I have been converting stuff like
sound 100, 1
exc... and just writing like
100, 1
for a number of languages and then just loading it into a spreadsheet
so that I can save little pieces of songs exc.. I can even use
different compilers or libraries that way.. I have started doing that
for quickbasic, qbasic, free basic, qb64, c++.... All I do is use
strings and '\n'.. I get to use a large number of older sounds and
effects (for basic) on newer compilers that have aditional options..
It just looks like another music tracker.. I am not sure if that fits
what you are trying to do though.