problem with variable scoping
Michael Dingler
an embarassing question. Yes folks, I'm writing my first
Python program (a text to HTML converter) and I have a
slight misunderstanding with the variable scoping...
Let me present some code fragments:
last_line_empty = 0
.
.
.
def process(line):
...
if last_line_empty:
pass
else:
print "<P>"
last_line_empty = 1
You see, I want to replace empty lines with paragraph symbols,
but if there is more than one, I want just one <p>-tag... If
I leave out the last statement (setting last_line_empty to one),
it works, but otherwise I get a 'name error' on the 'if' line.
As last_line_empty isn't a string it should be mutable, is my
grasp on some Python concepts that weak or do I just have a
'bad brain day'?
...Michael...
tav...@connix.com
Don't worry, it's not that bad.
It looks to me like the problem is that last_line_empty is a global
variable, and you want to assign to it inside a function, right? Here's
the missing incantation:
def process(line):
global last_line_empty # <--- YOU NEED THIS
...
Without the global declaration, assignments will always go into the
local scope. So, when you do "last_line_empty = 1", it actually created
a new last_line_empty local variable that completely masked the global
one.
Now, I'll also bet you were getting an error on the "if
last_line_empty:" and wondering why. Well, the Python compiler does some
tricks to optimize local variable access. One of those tricks is that it
knows wether access to a variable will be global or local depending on
the assignments. So, what the function tried to do is check the local
last_line_empty variable, which doesn't exist yet since it hasn't been
assigned to yet.
Sufficiently confused, I hope? Don't worry, it's not Python, just my
lousy explanations.
-Chris
--== Sent via Deja.com http://www.deja.com/ ==--
---Share what you know. Learn what you don't.---
Daniel E. Wilson
> It's...
>
> an embarassing question. Yes folks, I'm writing my first
> Python program (a text to HTML converter) and I have a
> slight misunderstanding with the variable scoping...
>
> Let me present some code fragments:
>
> last_line_empty = 0
> .
> .
> .
> def process(line):
> ...
> if last_line_empty:
> pass
> else:
> print "<P>"
> last_line_empty = 1
Your thinking like this is a C or Pascal program. The assignment of
last_line_empty creates an instance in the local scope. It does not
modify the instance in the enclosing scope.
Add 'global last_line_empty' at the top of the function and the problem
should be fixed.
--
Daniel E. Wilson
The idea that Bill Gates has appeared like a knight in shining armour to lead
all customers out of a mire of technological chaos neatly ignores the fact
that it was he who, by peddling second-rate technology, led them into it in
the first place.
- Douglas Adams
Martijn Faassen
>
> It's...
>
> an embarassing question. Yes folks, I'm writing my first
> Python program (a text to HTML converter) and I have a
> slight misunderstanding with the variable scoping...
>
> Let me present some code fragments:
>
> last_line_empty = 0
> .
> .
> .
> def process(line):
> ...
> if last_line_empty:
> pass
> else:
> print "<P>"
> last_line_empty = 1
>
> but if there is more than one, I want just one <p>-tag... If
> I leave out the last statement (setting last_line_empty to one),
> it works, but otherwise I get a 'name error' on the 'if' line.
Short answer:
Try using 'global':
def process(line):
# tell this function that last_line_empty exists and is a global
global last_line_empty
if last_line_empty:
...
Long explanation:
Whenever an assignment to a local variable occurs in a Python function
(such as last_line_empty = 1, Python by default always assigns to the
innermost scope), that variable is assumed to be in the function's local
namespace. Any lookups of that variable inside the function that occur
before the variable is defined fail, identical to the case where you
don't define the global variable last_line_empty.
This, for instance, will do unexpected things:
myglob = 1
def foo():
myglob = 0
foo()
print myglob # prints 1!
This does what you'd expect:
myglob = 1
def foo():
global myglob
myglob = 0
foo()
print myglob # prints 0
The tricky bit is that this works:
myglob = 1
def foo():
print myglob
foo() # prints 1, no NameError
If no assignment to myglob occurs in the function's namespace the global
namespace is used.
This all has interesting reasons and consequences that Tim Peters can
tell you the advantages of. :)
Note that it's good practice to try to avoid global variables in your
program (though they're useful sometimes, of course). Python provides
classes that often provide a more elegant solution; you can use a member
variable instead of a global variable.
Also take a look at StructuredText, a Python module that's part of
Digital Creation's Zope (www.zope.org), but can be used independently.
It does something similar to what you are trying to accomplish, I think.
Regards,
Martijn
Michael Dingler
> If no assignment to myglob occurs in the function's namespace the global
> namespace is used.
>
> This all has interesting reasons and consequences that Tim Peters can
> tell you the advantages of. :)
I'd certainly like to hear them. I'd like to know how other languages
handle this. Doesn't Perl do it the other way round, i.e. a _local_
keyword?
> Note that it's good practice to try to avoid global variables in your
> program (though they're useful sometimes, of course). Python provides
> classes that often provide a more elegant solution; you can use a member
> variable instead of a global variable.
I will. This being my first Python program, I just wanted to test the
basic concepts before I try to restructere it. Heck, most of the
program consists of print statements ;)
> Also take a look at StructuredText, a Python module that's part of
> Digital Creation's Zope (www.zope.org), but can be used independently.
> It does something similar to what you are trying to accomplish, I think.
htmlpp has a thing called 'guru' mode. But as it's written in Perl, I just
get into point haired boss mode (i.e. dollar signs on my eyes...)
...Michael...
Michael Dingler
> Your thinking like this is a C or Pascal program. The assignment of
> last_line_empty creates an instance in the local scope. It does not
> modify the instance in the enclosing scope.
>
> Add 'global last_line_empty' at the top of the function and the problem
> should be fixed.
Thanks.
Indeed my background is more towards C and Pascal, and as lots of scripting
languages try to emulate the idiosyncrasies of them, I kinda suspected
Python was doing the same. Well, one never should think that all algol-like
languages are the same ;)
...Michael...
Hans Nowak
On 26 May 99, Michael Dingler wrote:
> It's...
>
> an embarassing question. Yes folks, I'm writing my first
> Python program (a text to HTML converter) and I have a
> slight misunderstanding with the variable scoping...
>
> Let me present some code fragments:
>
> last_line_empty = 0
> .
> .
> .
> def process(line):
> ...
> if last_line_empty:
> pass
> else:
> print "<P>"
> last_line_empty = 1
Add this line
global last_line_empty
as the first line in your process function. Python's scoping is
different from, say, C's. Let's say I have this code:
x = 1
def p():
x = 2
then the x in function p is considered *local*. A similar construct
in C or Pascal would access the global variable x; not so in Python.
If you want to change this behavior, you can use the global
specifier. So this:
def q():
global x
x = 2
*would* access global variable x.
I'll be darned if this isn't in the FAQ somewhere. :o) [looks it up]
Try questions 4.36 and 4.57. They pretty much deal with this problem.
It used to bite me too when I started with Python.
Veel liefs,
--Hans Nowak (ivn...@hvision.nl)
Homepage: http://fly.to/zephyrfalcon
Tim Peters
> I'd certainly like to hear them. I'd like to know how other languages
> handle this. Doesn't Perl do it the other way round, i.e. a _local_
> keyword?
Perl started life with dynamic scoping and grafted on sensible semantics
later <0.5 wink -- but "my" rather than "local" is almost always want you
want to use in Perl5>).
Python's local rules will make instant sense to the C part of your brain if
you consider that Python acts exactly like C does, except that there's no
local *declaration* kicking you in the face.
> last_line_empty = 0
> .
> .
> .
> def process(line):
> ...
> if last_line_empty:
> pass
> else:
> print "<P>"
> last_line_empty = 1
C:
int last_line_empty = 0;
void process(const char* line)
{
int last_line_empty;
if (last_line_empty)
;
else
printf("<P>\n");
last_line_empty = 1;
}
In C too, the local last_line_empty shadows the global one, and the code is
nonsense as a result. Because it has no type declarations, Python has to
use *some* rule to decide which names are local (and happens to use a rule
based on whether a name is bound in the function body), but having made that
decision acts the same way as C. Well, not quite: Python yells at you if
you reference an uninitialized local, while C entertains you in creative
ways <wink>.
it's-a-feature-and-a-dessert-topping-ly y'rs - tim
Moshe Zadka
> > def process(line):
> > ...
> > if last_line_empty:
> > pass
> > else:
> > print "<P>"
> > last_line_empty = 1
>
> C:
>
> int last_line_empty = 0;
>
> void process(const char* line)
> {
> int last_line_empty;
> if (last_line_empty)
> ;
> else
> printf("<P>\n");
> last_line_empty = 1;
> }
Ummmm.....Tim, I beg to differ: it is more like the C++
void process(const char *line)
{
if (last_line_empty)
;
else
printf("<P>\n");
int last_line_empty;
last_line_empty = 1;
}
The Python code shadows last_line_empty only *after* the "decleration"
would-be-interesting-if-python-had-set!-and-not-just-define-ly y'rs, Z.
--
Moshe Zadka <mza...@geocities.com>.
QOTD: My own exit is more likely to be horizontal then perpendicular.
Hrvoje Niksic
> Ummmm.....Tim, I beg to differ: it is more like the C++
[...]
> The Python code shadows last_line_empty only *after* the "decleration"
Not quite. Try this:
>>> a = 3
>>> def x():
... print a
... a = 10
... print a
...
What do you expect calling x() will do? I'd "normally" expect it to
print 3, then 10. What really happens is this:
>>> x()
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in x
NameError: a
Fredrik Lundh
away with things like this while I'm off-line ;-)
> Ummmm.....Tim, I beg to differ: it is more like the C++
>
> void process(const char *line)
> {
> if (last_line_empty)
> ;
> else
> printf("<P>\n");
> int last_line_empty;
>
> last_line_empty = 1;
> }
>
> The Python code shadows last_line_empty only *after* the "decleration"
if it worked this way, do you really think Michael would have
gotten a NameError in the first place?
here's the full story:
http://www.python.org/doc/current/ref/execframes.html says:
Whether a name is local or global in a code block is determined
by static inspection of the source text for the code block: in the
absence of global statements, a name that is bound anywhere in
the code block is local in the entire code block; all other names
are considered global. The global statement forces global inter-
pretation of selected names throughout the code block.
...
Local names are searched only on the local namespace.
..
When a name is not found at all, a NameError exception is
raised.
note the use of "anywhere", "throughout", and "only".
</F>
Moshe Zadka
> (I might be busy, but don't expect me to let you guys get
> away with things like this while I'm off-line ;-)
>
> > Ummmm.....Tim, I beg to differ: it is more like the C++
<snipped stupid mistake made by me>
I humble myself before Fredrik and the Tim, and ask their forgiveness
for the nonsense I have said. I vow that from now on I will not post
without checking my programs...
now-who-can-be-mad-at-me-after-a-self-abasement-like-that<wink>-ly y'rs,
Moshe Zadka
> Static scoping still makes sense in the absence of indefinite extent (what
> you really mean by "closure"), and that's what most languages outside the
> Lisp world adopt (from C to Java). Arbitrary namespace nesting + indefinite
> extent may be in Python2; a technical hangup with garbage collection
> prevents it in Python1, unless you simulate it by hand via default-argument
> abuse (search DejaNews if you're intrigued).
Yes, I know that trick. And actually, IMHO, a better simulation would be
via classes and that wonderful __call__ special form.
(Reason: works better with variable length argument lists)
> > still-waiting-for-arbitary-expressions-in-lambdas-
>
> Already there: they're called "def". Honest! lambda is nothing but
> syntactic sugar for a particularly simple form of def in Python, and adds no
> power to the language. All in all, lambda shouldn't have been added; it's
> not what people expect it to be, so just provokes endlessly useless
> arguments (useless because it's too late to change).
Oh, come on, you expect me to write:
def incr(x): return x+1
l=map(incr, l)
Instead of
l=map(lambda x: x+1, l)
Ouch!!!!
> > and-tail-call-optimization-which-is-possible-in-python-ly y'rs, Z.
>
> It's not possible to identify a tail call at compile time (no name->object
> binding is known at compile-time, and e.g. the "f(n-1)" appearing inside a
> "def f:" may indeed notrefer to the function f, or even to a function at
> all, at runtime).
Hey, Tim, you know I put that part in just to cause you to say it isn't so
I could answer more fully <wink>.
OK: Notice ``tail *call* optimization'', not ``tail *recursion*
optimization'', which is a special case anyway (and not resembling a blue
sphere at all).
Whenever the python interpreter sees something like
def f(...):
.
.
.
return g(...)
It could generate code for the stack popping *before*, rather then *after*
the evaluation of g(...), since f(..) and its parameters are useless
(unless, of course, someone decides to play with the frames inside g, in
which case, he/she deserve to get shot)
> Bottom line: Anyone who intends to stick with Python is well-advised to
> learn how to write a loop <wink>.
>
> the-best-language-to-code-scheme-in-remains-scheme-ly y'rs - tim
Well, my paradigm is ``think in Scheme, write in Python, optimize in C''.
It sort of works, and though Guido would hate me for that, I'm sad that
too much gets lost in the translation.
though-i-find-i-can-live-w/o-continuations-so-guido-needn't-worry-ly y'rs,
Tim Peters
>> ...
>> Arbitrary namespace nesting + indefinite extent may be in Python2; a
>> technical hangup with garbage collection prevents it in Python1,
>> unless you simulate it by hand via default-argument abuse (search
>> DejaNews if you're intrigued).
[Moshe Zadka]
> Yes, I know that trick. And actually, IMHO, a better simulation would be
> via classes and that wonderful __call__ special form.
> (Reason: works better with variable length argument lists)
Agreed classes are better (& it can be done with or without __call__), but
for a different reason: classes in Python were designed to carry state, and
arglist tricks weren't. So it's a matter of playing along with the basic
design instead of fighting it. This is a Social Good.
>>> still-waiting-for-arbitary-expressions-in-lambdas-
>> Already there: they're called "def".
> Oh, come on, you expect me to write:
> def incr(x): return x+1
> l=map(incr, l)
>
> Instead of
> l=map(lambda x: x+1, l)
>
> Ouch!!!!
If you have trivial expressions you want to apply across lists, I "expect
you" to install the NumPy extension and use the array syntax designed for
this purpose.
If you have complex expressions you want to apply across lists, you're much
better off using an explict "for" loop in Python. Generally much faster,
and for most Python programmers much clearer too.
for x, y in sides:
hypot = math.sqrt(x**2 + y**2)
if hypot > max_hypot:
max_hypot = hypot
best_pair = x, y
Anyone who wants to write that as a nest of function applications instead is
simply diseased <wink>.
BTW, check out the old-but-not-so-old c.l.py threads on "list
comprehensions". That's a gimmick borrowed from functional languages (not
Scheme-ish, Haskell-ish) that strikes me as much more Pythonic than lambda
and filter.
>>> and-tail-call-optimization-which-is-possible-in-python-ly y'rs, Z.
>> It's not possible to identify a tail call at compile time ...
> Hey, Tim, you know I put that part in just to cause you to say it
> isn't so I could answer more fully <wink>.
>
> OK: Notice ``tail *call* optimization'', not ``tail *recursion*
> optimization'', which is a special case anyway (and not resembling a blue
> sphere at all).
Doesn't matter, though -- recursive or not, Python can't recognize a tail
call at compile time. The example I gave was recursive in order to
illustrate that it can't even handle the most obvious case <wink>.
> Whenever the python interpreter sees something like
>
> def f(...):
> .
> .
> .
> return g(...)
>
> It could generate code for the stack popping *before*, rather then
> *after* the evaluation of g(...),
1) It doesn't know at compile time that g(...) is a call. It's either that
or a runtime error, though, so it could pump out a special opcode that
considered only those cases. For that matter, it already does <wink>.
2) The Python compiler doesn't generate code to pop stack frames. That's a
small part of what I was trying to get across by saying tail-call
optimization wouldn't do you any good anyway given the current global
structure of the PVM; the latter wasn't designed to support it; it's a
battle.
3) "Stack frames" come in two flavors today, the kind Python explicitly
generates to hold Python info, and the ones implicitly constructed by C to
hold C info. Python is implemented in C, and *both* kinds of frames get
mixed into the bag during a Python-level call today; there's nothing the PVM
can do today to tail-call away the C frame sitting on the C stack (well, at
least not before C does tail-call optimization too ...). Again part of the
current global PVM structure.
4) What's the point? If it's just a matter of not growing the stack depth,
with no speed advantage, hardly anyone would care. There are real downsides
to doing this too (see below), so almost everyone would care a lot <wink>.
> since f(..) and its parameters are useless (unless, of course, someone
> decides to play with the frames inside g, in which case, he/she deserve
> to get shot)
That's the major downside for me: playing with f's frame inside g is
*extremely* useful when g raises an unhandled exception. Think "debugger"
here. If "f" went missing from the traceback report that would be confusing
enough, but not being able to examine f's locals in a post-mortem analysis
would be a disaster (given that as often as not, it was some local
computation of f's (or something up *its* call chain) that caused a bad
argument to g that caused g to blow up).
The notion that f's locals are dead after it tail-calls to g is true only in
an anal theoretical sense <wink>; in practice, no local is really dead
until f returns without exception, and keeping f's frame alive reflects the
reality that when things go wrong people desperately do want to look at f's
locals. This doesn't preclude keeping f's frame alive but unhooking it from
the call stack, though. There! We can give you unbounded memory use *and*
confusing tracebacks, all at the same time <wink>.
> Well, my paradigm is ``think in Scheme, write in Python, optimize in C''.
> It sort of works, and though Guido would hate me for that, I'm sad that
> too much gets lost in the translation.
Over time you're likely to think more in Python, and less will get lost.
> though-i-find-i-can-live-w/o-continuations-so-guido-needn't-worry-ly y'rs,
> Z.
See? You're thinking more in Python already <wink>.
idiomatic-scheme-is-a-treat-and-so-is-idiomatic-python-ly y'rs - tim