Name resolution and the (wrong?) LEGB rule

[Marco Buttu]

Sometimes the Python name resolution is explained using a LEGB rule.
For instance, in [1] (I think also the Learning Python book gives the same):

"if a particular name:object mapping cannot be found in the local
namespaces, the namespaces of the enclosed scope are being searched
next. If the search in the enclosed scope is unsuccessful, too, Python
moves on to the global namespace, and eventually, it will search the
built-in namespace (side note: if a name cannot found in any of the
namespaces, a NameError will is raised)."

AFAIK, Python has static scoping: the scope for a name is given during
the bytecode compilation. This means that before executing the program,
Python already know in which namespace to look for an object. So, it
seems to me that the LEGB rule is wrong, and this is what actually happens:

* local name (LOAD_FAST instruction): Python looks (only) in the local
namespace, and if it does not find the name, then it raises an
UnboundLocalError

* global name (LOAD_GLOBAL): at first Python looks in the globals(), and
in case the name is not there, it looks in the builtins namespace; if
the name is neither in the global nor in the builtin namespace, it
raises a NameError

* enclosing scope (LOAD_DEREF): there is a closure, and Python looks for
the name in the enclosing namespace

Is that right, or am I missing something? Thanks, Marco

[1]
http://sebastianraschka.com/Articles/2014_python_scope_and_namespaces.html

--
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbu...@oa-cagliari.inaf.it

[Chris Angelico]

On Fri, Dec 9, 2016 at 1:51 AM, Marco Buttu <marco...@gmail.com> wrote:
> "if a particular name:object mapping cannot be found in the local
> namespaces, the namespaces of the enclosed scope are being searched next. If
> the search in the enclosed scope is unsuccessful, too, Python moves on to
> the global namespace, and eventually, it will search the built-in namespace
> (side note: if a name cannot found in any of the namespaces, a NameError
> will is raised)."
>
> AFAIK, Python has static scoping: the scope for a name is given during the
> bytecode compilation. This means that before executing the program, Python
> already know in which namespace to look for an object. So, it seems to me
> that the LEGB rule is wrong,

It isn't wrong, but there are some parts of it that can be resolved at
compile time. Once a function is compiled, it cannot normally gain or
lose local names. There are a few situations that can mess with that
(a star import or 'exec' statement, in Python 2), and when the
compiler detects one of those, it has to avoid the usual optimization.

The "usual optimization" is exactly what you describe: that different
bytecodes represent Local, Enclosing, and Global/Built-in scope
lookups. (Globals can be created or removed at run-time, so there's no
optimization possible there.) But in terms of language specifications,
the lookup rules are the same; it's just that the CPython compiler
takes advantage of information that it can see ("these are the only
locals for this function") to speed up execution.

ChrisA

[Peter Otten]

Chris Angelico wrote:

> On Fri, Dec 9, 2016 at 1:51 AM, Marco Buttu <marco...@gmail.com> wrote:
>> "if a particular name:object mapping cannot be found in the local
>> namespaces, the namespaces of the enclosed scope are being searched next.
>> If the search in the enclosed scope is unsuccessful, too, Python moves on
>> to the global namespace, and eventually, it will search the built-in
>> namespace (side note: if a name cannot found in any of the namespaces, a
>> NameError will is raised)."
>>
>> AFAIK, Python has static scoping: the scope for a name is given during
>> the bytecode compilation. This means that before executing the program,
>> Python already know in which namespace to look for an object. So, it
>> seems to me that the LEGB rule is wrong,

It might be sufficient to say that the LE part is usually statically
determined while the GB part is dynamic.

The odd beast is the class namespace:

>>> x = "outer"
>>> class C:
... print(x)
... x = "inner"
... print(x)
... del x
... print(x)
...
outer
inner
outer

> It isn't wrong, but there are some parts of it that can be resolved at
> compile time. Once a function is compiled, it cannot normally gain or
> lose local names. There are a few situations that can mess with that
> (a star import or 'exec' statement, in Python 2), and when the
> compiler detects one of those, it has to avoid the usual optimization.
>
> The "usual optimization" is exactly what you describe: that different
> bytecodes represent Local, Enclosing, and Global/Built-in scope
> lookups. (Globals can be created or removed at run-time, so there's no
> optimization possible there.) But in terms of language specifications,
> the lookup rules are the same; it's just that the CPython compiler
> takes advantage of information that it can see ("these are the only
> locals for this function") to speed up execution.

If it is only an optimization why doesn't a failing local lookup fall back
to the global namespace?

[Chris Angelico]

On Fri, Dec 9, 2016 at 5:03 AM, Peter Otten <__pet...@web.de> wrote:
>> The "usual optimization" is exactly what you describe: that different
>> bytecodes represent Local, Enclosing, and Global/Built-in scope
>> lookups. (Globals can be created or removed at run-time, so there's no
>> optimization possible there.) But in terms of language specifications,
>> the lookup rules are the same; it's just that the CPython compiler
>> takes advantage of information that it can see ("these are the only
>> locals for this function") to speed up execution.
>
> If it is only an optimization why doesn't a failing local lookup fall back
> to the global namespace?

Define "failing". Do you mean that this should print "outer"?

x = "outer"
def f():
print(x)
x = "inner"
f()

There are plenty of languages where this is true, but they work
because the defined scope of a variable is "from its declaration
down". Python doesn't work like that. Neither does JavaScript,
although it's a bit bizarre in a few ways. The lookup doesn't fail; it
finds a local variable that doesn't have a value. At least, I'm pretty
sure that's how it works. Is there a language specification stating
this?

ChrisA

[Peter Otten]

Chris Angelico wrote:

> On Fri, Dec 9, 2016 at 5:03 AM, Peter Otten <__pet...@web.de> wrote:
>>> The "usual optimization" is exactly what you describe: that different
>>> bytecodes represent Local, Enclosing, and Global/Built-in scope
>>> lookups. (Globals can be created or removed at run-time, so there's no
>>> optimization possible there.) But in terms of language specifications,
>>> the lookup rules are the same; it's just that the CPython compiler
>>> takes advantage of information that it can see ("these are the only
>>> locals for this function") to speed up execution.
>>
>> If it is only an optimization why doesn't a failing local lookup fall
>> back to the global namespace?
>
> Define "failing". Do you mean that this should print "outer"?
>
> x = "outer"
> def f():
> print(x)
> x = "inner"
> f()

I mean it could, as classes already work that way. I think the current
behaviour is a design decision rather than an implementation accident.

> There are plenty of languages where this is true, but they work
> because the defined scope of a variable is "from its declaration
> down". Python doesn't work like that. Neither does JavaScript,
> although it's a bit bizarre in a few ways. The lookup doesn't fail; it
> finds a local variable that doesn't have a value. At least, I'm pretty
> sure that's how it works. Is there a language specification stating
> this?

I found

https://www.python.org/dev/peps/pep-0227/

[Steve D'Aprano]

You are partly right, but you are also missing a few things.

(1) Technically, the LOAD_* byte codes are implementation details, and there
is no requirement for Python interpreters to use them. In fact, Jython
being built on the JVM and IronPython being built on the .Net runtime
cannot use them. Even a C-based interpreter is not required to implement
name resolution in the same way.

(2) There are some odd corner cases in Python 2 where you use exec() or
import * inside a function to create local variables, where the CPython
interpreter no longer uses LOAD_FAST to resolve locals. (I'm going by
memory here, so I may have some of the details wrong.) You can experiment
with that if you like, but the behaviour has changed in Python 3 so its no
longer relevant except as a historical detail for those stuck on Python 2.

(To be precise, import * is forbidden inside functions, and exec() must take
an explicit namespace argument rather than implicitly applying to the local
function scope.)


(3) However, the most important case you've missed is code executed inside a
class namespace during class creation time:


class C(object):

a = 1

def test(): # intentionally no self
b = 2
return a + b

c = 3
d = test() + c



Don't forget classes nested inside classes...


I consider the LEGB rule to be a statement of intent, describing how a
Python implementation is expected to behave, *as if* it were following the
LEGB rule, not necessarily a statement of the implementation of any
specific Python implementation.





--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.