Python, Parrot, and lexical scopes
Sam Ruby
Parrot, and posted a few thoughts on my weblog:
http://www.intertwingly.net/blog/2004/10/18/Python-Parrot-and-Lexical-Scopes
While I posted it on my weblog for formatting and linking reasons, feel
free to respond on the mailing list. Suggestions welcome, in
particular, a PIR equivalent to the Perl would be most helpful.
- Sam Ruby
Aaron Sherman
> I've been trying to make sense of Python's scoping in the context of
> Parrot, and posted a few thoughts on my weblog:
>
> http://www.intertwingly.net/blog/2004/10/18/Python-Parrot-and-Lexical-Scopes
It seems like everything on that page boils down to: all functions are
module-scoped closures.
Your example:
Consider the following scope1.py:
from scope2 import *
print f(), foo
foo = 1
print f(), foo
and scope2.py:
foo = 2
def f(): return foo
The expected output is:
2 2
2 1
Is also useful for context, but I don't think you need the Perl
translation to explain it.
Sam Ruby
> On Mon, 2004-10-18 at 07:55, Sam Ruby wrote:
>
>> I've been trying to make sense of Python's scoping in the context
>> of Parrot, and posted a few thoughts on my weblog:
>>
>> http://www.intertwingly.net/blog/2004/10/18/Python-Parrot-and-Lexical-Scopes
>
> It seems like everything on that page boils down to: all functions
> are module-scoped closures.
A closer translation: "How do I implement module-scoped closures in Parrot?"
> Your example: [snip]
>
> Is also useful for context, but I don't think you need the Perl
> translation to explain it.
You elided the reason why I included it:
> Suggestions welcome, in particular, a PIR equivalent to the Perl
> would be most helpful.
If I look at the description of the scratchpad opcodes, I don't see all
the pieces that I need (save_context, restore_context, mark_context,
swap_context, etc). However, by poking around enough, and with a little
bit of dumb luck, I have stumbled across src/sub.c.
The functions it defines aren't used by any opcodes, but are used by a
few PMCs. Those PMCs have delightful names like continuation,
coroutine, and retcontinuation. So the prefered approach is either use
one of these, or package the desired functionality into a pyfunction.pmc?
Unfortunately, these PMCs don't seem to have test cases.
Clearly, I'm fumbling around in the dark. A well placed RTFM (including
an indication of *which* FM) would be most welcome. Until then, I will
continue to ask questions, make observations, and submit patches to
bring the code base in line of where I'm guessing it wants to go - even
if many or most of these get rejected.
- Sam Ruby
Sam Ruby
>
> Unfortunately, these PMCs don't seem to have test cases.
Correction: t/pmc/sub.t
- Sam Ruby
Sam Ruby
>> It seems like everything on that page boils down to: all functions
>> are module-scoped closures.
>
> A closer translation: "How do I implement module-scoped closures in
> Parrot?"
OK, I've roughed out an implementation:
http://intertwingly.net/stories/2004/10/18/scopes.pir
http://intertwingly.net/stories/2004/10/18/pymodule.pmc
In the process, I've made a large number of assumptions. Undoubtably,
many of them are wrong.
- Sam Ruby
Dan Sugalski
I responded (sorta) on the weblog, but I'll redo it here since it
gets into some of the fundamental bits of namespaces, which checking
the calendar I see we're scheduled to grovel over again.
The code (for folks playing along at home) is:
scope1.py:
from scope2 import *
print f(), foo
foo = 1
print f(), foo
and scope2.py:
foo = 2
def f(): return foo
I'll make two assumptions in the explanation here.
First, that lexical scopes are inappropriate. In this case, they'd do
exactly what you want them, but the problem there is that you can't
really have multiple files sharing the same scope, so that makes
splitting modules into multiple files untenable.
Second, that named namespaces are inappropriate. Again, in this case
they could do what you wanted if scope1.py and scope2.py were in
different basic namespaces. Modules split across multiple files
could, with named namespaces (that is, things in the main module have
their variables in main:, the foo module in foo:, and so forth), work
just fine, but that's not what we need here, since python wants
un-qualified names to look up, at runtime, in the current module
namespace and then the main namespace.
So, the solution here is to have a chain of overlapping namespaces.
Each sub or method has a handle on a namespace, which itself has a
link to the namespace it's occluding, and so on up to the top, basic
namespace. (If, indeed, we even have one that's universal -- code
could twiddle with that if it really wanted to) The namespaces act
much like the lexical pads do (or would, if they were fully
functional) only with globally visible names instead.
The nice thing here is that this is transparent to the code -- the
find_global and store_global ops may have to jump through some hoops
to do the right thing, but most bytecode won't know it's happening.
What we need to do is define and add the ops to add in and remove
layers of namespaces, and get the packfile format set so that the
proper layers can be anchored to the sub PMCs when bytecode's loaded
in from wherever.
--
Dan
--------------------------------------it's like this-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Sam Ruby
>
> Consider the following code:
>
> def f(x): return len(x)
> for i in [0,1]:
> print f("foo")
> f = lambda x: x.upper()
No, don't. Consider the following code instead:
def f(x): return len(x)
for i in [0,1]:
print f("foo")
len = lambda x: x.upper()
Key difference is the last line. In this example, there is only one
definition for f, one that will call whatever function is defined as
"len" at the time of the call.
- Sam Ruby
Leopold Toetsch
> ... Suggestions welcome, in
> particular, a PIR equivalent to the Perl would be most helpful.
It could be something like below. Some remarks:
* we don't have a notion to create a Closure PMC, so these closures are
handcrafted. (NB: a subroutine with a .yield inside gets already
created as a Coroutine PMC constant)
* the import statement is simulated too by storing the lexicals into the
caller's frame. This would very likely be another Python opcode.
* the builtins would be at the top-level pad. The lexical opcodes do not
fully comply with Python's name lookup, which is something like this:
for d in (locals, globals, builtins):
try:
f = d[wrapped_name]:
return f
except KeyError:
pass
except:
raise
# raise NameError
# scope1.pir
.namespace [""]
.sub __main__ @MAIN
new_pad 0
# from scope2 import *
load_bytecode "scope2.pir"
$P0 = find_global "scope2", "_scope2__import"
$P0()
# print f(), foo
.local pmc f
f = find_lex "f"
$P2 = f()
.local pmc foo
foo = find_lex "foo"
print_item $P2
print_item foo
print_newline
# foo = 1
$P4 = new Undef
$P4 = 1
store_lex -1, "foo", $P4
foo = find_lex "foo"
#print f(), foo
$P3 = f()
print_item $P3
print_item foo
print_newline
.end
# scope2.pir
.namespace ["scope2"]
.sub _scope2__init @LOAD
new_pad -1
# foo = 2
$P0 = new Undef
$P0 = 2
store_lex -1, "foo", $P0
# def f(): return foo
.local pmc f
f = new Closure
set_addr f, _f
store_lex -1, "f", f
$P1 = new Closure
set_addr $P1, _scope2__import
store_global "scope2", "_scope2__import", $P1
.end
.sub _scope2__import # @CLOSURE
$P0 = find_lex "foo"
store_lex -2, "foo", $P0
$P0 = find_lex "f"
store_lex -2, "f", $P0
.end
.sub _f # @CLOSURE
$P0 = find_lex "foo"
.pcc_begin_return
.return $P0
.pcc_end_return
.end
leo
Allen Short
> * the import statement is simulated too by storing the lexicals into the
> caller's frame. This would very likely be another Python opcode.
I should point out that this is much more like Python's semantics for
"import *" than Dan's overlapping-namespaces idea -- "import" really
means "copy these bindings into my current module". In particular,
after doing "from foo import *", subsequent additions or removals of
names to foo will not be reflected in the bindings of the module
importing them.
Allen
Dan Sugalski
Right, but the overlapping namespaces stuff isn't for the import.
It's certainly possible that I overestimated the complexity involved
in the semantics -- wouldn't be the first time, probably won't be the
last. If lexicals actually work just fine for python I'm fine with
tossing (or never getting around to implementing) the layered
namespace stuff, though if its implemented and never used it has no
performance penalties for anything but failed name lookups.