Q: newbie wants to eval()
lynx
to a regexp match part, and if the comparison is equal, set the value of
the variable to the value given in another part of the same regexp
match; which seems most obviously done with eval().
i.e., i want to parse a configuration file of the rough form
variable_name = variable_value, and i want to make sure the variable
being set is on a list of variables i'm willing to allow setting, so i
don't want to just eval() the whole line - hence the list iteration.
trouble is, i've never used eval() for anything before, and i can't seem
to get the syntax right. given a loop iterator variable i, which
iterates over a list of variable names, and a regexp match variable m,
which can be assumed to hold the value i'd like to assign to the named
variable, what should the eval expression look like, and why - how would
the bits and pieces of the complete, correct expression work together? i
ask because none of my current efforts work at all, and i can't figure
out why.
one little additional twist is that the variables i'm trying to set are
globals, defined outside the configuration-file-reading-function itself.
does this change some argument to the eval(), and if so, how?
--
PGP/GnuPG key (ID 1024D/BFE0D6D0) available from keyservers everywhere
Key fingerprint = 3EBC 97FC 68AA 65F1 65E6 3D36 35F6 4213 BFE0 D6D0
"...life goes on
long after the thrill of living is gone..."
Sean 'Shaleh' Perry
known_vars = ('foo', 'bar', 'baz')
input = 'foo = 2'
var, data = string.split(input)
if var in known_vars:
exec(input) # eval takes an expression, assignment is not
else:
print "unknown variable: %s" % var
for key in globals().keys():
if key[0] != '_':
print key
Alex Martelli
news:3b3e3e81$0$88189$2c3e...@news.voyager.net...
> i want to iterate over a list of variable names, comparing each in turn
> to a regexp match part, and if the comparison is equal, set the value of
> the variable to the value given in another part of the same regexp
> match; which seems most obviously done with eval().
If it so seems, appearances are deceptive. eval() evaluates an
EXPRESSION, and variable binding and rebinding is normally done
by assignment, which is a STATEMENT. In Python expressions and
statements are separate concepts: an expression can always be
used as a statement, but a statement cannot be used as an expression.
You'd need to use the 'exec' statement (shudder), but fortunately
you can easily do better than that, since...:
> one little additional twist is that the variables i'm trying to set are
> globals, defined outside the configuration-file-reading-function itself.
> does this change some argument to the eval(), and if so, how?
Global variables are really attributes of an object, the module
object to be precise. Setting any object's attributes is most
easily done via built-in function setattr().
So you need a reference to the object that's the module you
want to affect. Assuming this is the module containing the
function you're writing, simplest may be:
import sys
mymod = sys.modules[__name__]
and now mymod is the reference you require.
Now, if your variable x refers to a string that names the variable
you want to set, and y refers to the value you want to bind your
variable to,
setattr(mymod, x, y)
will do what you require.
Alex
Sean 'Shaleh' Perry
> var, data = string.split(input)
>
oops (-: This should read:
var, symbol, data = string.split(input) # duh!
lynx
<mailman.993937666...@python.org>:
[...]
> exec(input) # eval takes an expression, assignment is not
<blush> yes, so i found out. boy do i have egg on my face... i guess i'm
still thinking too Perl-ish, i couldn't get it out of my head that "eval"
is the way to execute arbitrary code. thanks to everybody for their help,
i've got it pretty much working now.
lynx
[...]
> You'd need to use the 'exec' statement (shudder), but fortunately you
> can easily do better than that, since...:
hmm. is there something inherently evil about exec that i should know
about...?
> Global variables are really attributes of an object, the module object
> to be precise. Setting any object's attributes is most easily done via
> built-in function setattr().
> So you need a reference to the object that's the module you want to
> affect. Assuming this is the module containing the function you're
> writing, simplest may be:
> import sys
> mymod = sys.modules[__name__]
> and now mymod is the reference you require.
looks interesting. should that snippet be run in the function where i'm
trying to change these globals, or in the main program and that reference
passed in as an argument to the function?
honestly, myself, i'd be more upset and shuddering at this kind of
manual rooting around in Python's own internal workings than at the
exec() statement, which seems designed to hide this sort of magic from
me; unless, of course, there's something strange about exec that i don't
know of...?
Alex Martelli
...
> > You'd need to use the 'exec' statement (shudder), but fortunately you
> > can easily do better than that, since...:
>
> hmm. is there something inherently evil about exec that i should know
> about...?
In a sense, it's "too powerful" -- it can do so many things that it's bound
to have unpleasant fallback in some way or other. Clarity and ease of
understanding your program is likely to be the main casualty, but there
may be others.
For example: usually, the compiler is able to perform a simple but
crucial optimization for local variables of your functions -- it knows
what variables are local (all that are bound in local scope), and it
needs not insert lookup operations for local variable access into the
bytecode it generates. But if the function contains an exec, all
bets are off -- the compiler DOES need to insert explicit lookups,
so all of the function is slowed down.
So, use exec when you must, but be aware that you RARELY 'must',
and there are usually better alternatives. If and when you do use
exec, it's generally best to use explicit dictionaries as namespaces,
to avoid accidents and slow-downs.
> > Global variables are really attributes of an object, the module object
> > to be precise. Setting any object's attributes is most easily done via
> > built-in function setattr().
>
> > So you need a reference to the object that's the module you want to
> > affect. Assuming this is the module containing the function you're
> > writing, simplest may be:
> > import sys
> > mymod = sys.modules[__name__]
> > and now mymod is the reference you require.
>
> looks interesting. should that snippet be run in the function where i'm
> trying to change these globals, or in the main program and that reference
> passed in as an argument to the function?
Either will work. Having it in your function will make it simpler to use.
Having it on the outside makes your function more general, since it
can be called with different module-object arguments to set global
variables in different modules (or attributes of other objects yet). A
person with any knack for overgeneralization would have mymod as
an optional argument with a default value of None and do the above
only if mymod is None...:-).
> honestly, myself, i'd be more upset and shuddering at this kind of
> manual rooting around in Python's own internal workings than at the
> exec() statement, which seems designed to hide this sort of magic from
> me; unless, of course, there's something strange about exec that i don't
> know of...?
Python is an unusual high-level language in that it DOES expose and
document a wide part of how it internally goes about things, and quite
explicitly empowers you to "root around" that part. It does mean one
must employ good taste and common sense to avoid making one's
code a mess of hard-to-understand, hard-to-maintain black magic, yes.
But "exec" is still overkill for most things you can do by other means.
Specifically, when you think you need exec or eval to bind or access
variables whose names you don't know in advance, setattr and getattr
are very likely to be a better choice. Whether they must go to a module
object is another issue; often you're better served by keeping such
variables as attributes of _your own_ object that you fully control, and
that will most often be an instance object rather than a module object,
because instance objects can check their own attributes via special
methods __getattr__ and __setattr__ -- and you may not even need
that much.
Suppose you keep all of your configuration variables this way:
class Config:
one = 1
another = 23
andalso = 'feefie'
config=Config()
Now you can elsewhere in your code check and use config.one,
config.another, config.andalso -- as the attributes are not specifically
bound in the instance, you'll get the class ones. You can also
allow user-configuration code to set ANY attribute of config, no
checks needed -- if the user sets an attribute you're not using,
that's gonna be a no-op. So you can for example:
execfile("userconf.py", vars(config))
and then use config.one, etc, and get either the instance variable
if user code has set it, or else the fallback class value.
Alex
Kragen Sitaker
lynx <no...@nowhere.net> wrote:
>hmm. is there something inherently evil about exec that i should know
>about...?
exec and eval are an easy way to write code that works some of the time
and breaks in spectacular and completely unexpected ways when you get
unexpected input.
Their effect depends entirely on their arguments. They can do anything
--- literally anything at all. This makes it hard for you to determine
why your program is breaking.
Say you find that a variable named 'max_pool' is getting set to some
incorrect value and this is breaking your program. Without 'exec' or
'eval' (or other similar, but less dangerous, things like setattr), you
can look in the scope of max_pool to see where max_pool gets set ---
only in statements that look like this:
max_pool = <something>
There are probably only three or four of these, and you can probably
look at the <something> to see if it could possibly be evaluating to
the incorrect value you're seeing. Probably one or two of them could,
so you put a print statement above them to see what's happening.
Voila, you found the bug.
Now, if, in addition to these three or four assignments, there are also
five or six places you're calling exec or eval, any one of those places
could be messing with max_pool. Whether they are or not probably
depends on some data you computed earlier, or possibly read in from a
file or something. Your problem may have gotten as much as three times
harder.
What's worse, it's probably hard to tell what's being assigned to
max_pool in your exec strings; you probably have to look at some stuff
that's being interpolated into a string with %, figure out what it
could evaluate to, and then look at the resulting expression and figure
out what *it* could evaluate to. This extra level of indirection isn't
trivial for the smartest of us.
Also, as someone (Alex?) pointed out, not only can't you easily reason
about what your code is doing once you introduce exec or eval, neither
can the compiler, so it tends to produce slower code.
So use exec and eval only when it saves you pages of code, because the
price you pay is bigger than you think.
This advice applies to Perl and Tcl, too, and any other language that
gives you eval "string"; some of it applies to Lisp, but some probably
doesn't.
--
<kra...@pobox.com> Kragen Sitaker <http://www.pobox.com/~kragen/>
Perilous to all of us are the devices of an art deeper than we possess
ourselves.
-- Gandalf the White [J.R.R. Tolkien, "The Two Towers", Bk 3, Ch. XI]