Namespaces (At Long Last)
Matt Diephouse
latest namespace spec draft. Comments are most welcome: to quote Chip,
"The rest of the discussion would benefit from more eyes."
Thanks,
--
matt diephouse
http://matt.diephouse.com
Synopsis
- Languages should contain their namespaces
- Namespaces should be hierarchical
- Add a get_namespace opcode (that takes an array or a multidimensional
hash index)
- Namespaces follow the semantics of the HLL in which they're defined
- Imports follow the semantics of the library's language
- Two interfaces: typed and generic
Namespace PMC API
There are many different ways to implement a namespace and Parrot's
target languages display a wide variety of them. By implementing an API,
it should be possible to allow interoperability while still allowing
each one choose the best internal representation.
Definitions
HLL A High Level Language, such as Perl, Python, or Tcl. This is in
contrast to PIR.
Current namespace
The current namespace is the namespace associated with the currently
executing subroutine.
Naming Conventions
For interoperability, languages should use hierarchical namespaces.
There's no way to outlaw flat namespaces, of course, and that remains an
option for language implementors, but a standard must be chosen for easy
interoperability, and hierarchical seems to be the better choice.
HLL Private Namespaces
HLLs should use a namespace with an underscore and the lowercased
name of the HLL to store any private items. For instance, Tcl should
use the "_tcl" namespace to store the subroutines that make up the
compiler.
User Defined Namespaces
All HLLs should prefix any namespaces with the lowercased name of
the HLL (so there's no question of what to capitalize). So Perl 5's
CGI module should be stored under perl5::CGI. This eliminates any
accidental collisions between languages.
Interfaces: Generic and Typed
Most languages leave their symbols plain, which makes lookups quite
straightforward. Others use sigils or other mangling techniques,
complicating the problem of interoperability.
Parrot namespaces assist with interoperability by providing two
interface subsets: the *raw interface* and the *typed interface*.
Raw Interface
Each HLL may, when working with its own namespace objects, use the *raw
interface*, which allows direct naming in the native style of the
namespace's HLL. This interface consists of the following methods:
TODO: Decide whether this is best exposed by "does hash".
add($S0, $P0)
Store the PMC in $P0 under the raw name $S0.
del($S0)
Delete the entry named $S0.
$P0 = find($S0)
Find the sub, namespace, or variable named $S0. Return a null PMC on
lookup failure.
$P0 = names()
Get an array of the names contained in the namespace.
NOTE: The return value may be a psuedo-array holding a reference to
the namespace and some kind of internal iterator.
Typed Interface
When a given namespace's HLL is either different from the current HLL or
unknown, an HLL should generally use only the language-agnostic
namespace interface. This interface isolates HLLs from each others'
naming quirks. It consists of add_foo(), find_foo(), and del_foo()
methods, for values of "foo" including "sub" (something executable),
"namespace" (something in which to find more names), and "var"
(anything).
NOTE: The job of the typed interface is to bridge *naming* differences,
and *only* naming differences. It does not enforce, nor even notice, the
interface requirements of "sub" or "namespace". Thus, for example,
successful execution of add_sub("foo", $P0) does *not* automatically
guarantee that $P0 is an invokable subroutine. Caveat usor.
add_sub($S0, $P0)
Store $P0 as a subroutine with the name of $S0.
add_namespace($S0, $P0)
Store $P0 as a sub-namespace with the name of $S0. Note that to add
Bar::Baz to the Foo namespace, one must add Bar to Foo and then Baz
to Foo::Bar.
add_var($S0, $P0)
Store $P0 as a variable under the name of $S0.
IMPLEMENTATION NOTE: perl6::namespace.add_var may choose to check
which parts of the variable interface are implemented by $P0 so it
can decide on an appropriate sigil.
del_sub($S0)
del_namespace($S0)
del_var($S0)
Delete the sub, namespace, or variable named $S0.
$P0 = find_sub($S0)
$P0 = find_namespace($S0)
$P0 = find_var($S0)
Find the sub, namespace, or variable named $S0.
IMPLEMENTATION NOTE: perl6::namespace.find_var should check all
variable sigils, but the order is not to be counted on by users. If
you're planning to let Python code see your module, don't have both
"our $A" and "our @A".
$P0 = subs()
$P0 = namespaces()
$P0 = vars()
Get an array of the subs, namespaces, or variables in the namespace.
import_into($P0, ...)
Import items from the current namespace into the namespace in $P0.
The namespace may choose to take any number of arguments for the
import routine, such as the name of subroutines, patterns, or tags.
These always follow the conventions of the callee and not the
caller.
This aligns with the behavior that users will expect, so that `use
tcl:Some::Module 'c*'` will DTRT and import all the commands that
start with 'c' from the Tcl namespace into the Perl namespace.
IMPLEMENTATION NOTE: Most namespace import_into implementations will
restrict themselves to using the typed interface on the target
namespace. However, they may also decide to check the type of the
target namespace and, if it turns out to be of a compatible type, to
use same-language shortcuts.
store_var($S0, $P0)
Store the variable $P0 under the name $S0.
Non-interface Methods
These methods don't belong to either the typed or the generic interface.
$P0 = name()
Returns the name of the namespace as an array of strings. So
perl5:Some::Module would return an array containing "Some",
"Module".
NOTE: This is a naive method. It does not account for any aliasing.
Class PMC API
Class should inherit from the Namespace PMC.
Methods
add_method($S0, $P0)
Add a method $P0 under the name $S0.
del_method($S0)
Delete the method named $S0.
$P0 = find_method($S0)
Find the method named $S0.
$P0 = methods()
Return an array of the methods in the class.
Compiler PMC API
Methods
load_library($S0, $S1, ...)
Ask this compiler to load a library/module named by the strings $S0,
$S1, ... So perl5:Some::Module should be loaded using
"perl5.load_library("Some", "Module")".
Subroutine PMC API
Some information must be available about subroutines to implement the
correct behavior about namespaces.
Data
get_namespace
The namespace where the subroutine was defined. (As opposed to a
namespace that it was imported into.)
Namespace Opcodes
add_namespace $P0, $P1
Add the namespace PMC $P1 as the namespace $P0 (an array of names or
a multidimensional hash index).
del_namespace $P0
Delete the namespace $P0 (an array of names or a multidimensional
hash index).
$P0 = find_global $P1, $S0
$P0 = find_global $S0
Find the variable $S0 in $P1 or the current namespace.
$P0 = get_namespace $P1
$P0 = get_namespace
Get the namespace $P1 (an array of names or a multidimensional hash
index) or the current namespace. To get the "Foo::Bar" namespace,
one would use this:
$P0 = split "::", "Foo::Bar"
$P1 = get_namespace $P0
or this:
$P1 = get_namespace ["Foo"; "Bar"]
store_global $P1, $S0, $P0
store_global $S0, $P0
Store $P0 as the variable $S0 in $P1 or the current namespace.
HLL Namespace Mapping
In order to make this work, Parrot must somehow figure out what type of
namespace pmc to create.
Default Namespace
The default namespace PMC will implement Parrot's current behavior.
Compile-time Creation
This perl:
#!/usr/bin/perl
package Foo;
$x = 5;
should map roughly to this PIR:
.HLL "Perl5", "perl5_group"
.namespace [ "Foo" ]
.sub main :main
$P0 = new .PerlInt
$P0 = 5
store_global "$x", $P0
.end
In this case, the "main" sub would be tied to Perl5 by the ".HLL"
directive, so a Perl5 namespace would be created.
Run-time Creation
Consider the following Perl5 program:
#!/usr/bin/perl
$a = 'x';
${"Foo::$a"} = 5;
The Foo:: namespace is created at run-time (without any optimizations).
In these cases, Parrot should create the namespace based on the HLL of
the PIR subroutine that calls the store function.
.HLL "Perl5", "perl5_group"
.sub main :main
# $a = 'x';
$P0 = new .PerlString
$P0 = "x"
store_global "$a", a
# ${"Foo::$a"} = 5;
$P1 = new PerlString
$P1 = "Foo::"
$P1 .= $P0
$S0 = $P1
$P2 = split "::", $S0
$S0 = pop $P2
$S0 = "$" . $S0
$P3 = new .PerlInt
$P3 = 5
store_global $P2, $S0, $P3
.end
In this case, "store_global" should see that it was called from "main",
which is in a Perl5 namespace, so "Foo::" should be also created as a
Perl 5 namespace.
Language Notes
Perl 6
Sigils
Perl6 may wish to be able to access the namespace as a hash with
sigils. That is certainly possible, even with subroutines and
methods. It's not important that a HLL use the typed namespace
API, it is only important that it provides it for others to use.
So Perl6 may implement get_keyed and set_keyed VTABLE slots that
allow the namespace PMC to be used as a hash. The "find_sub" and
"find_method" methods would, in this case, would append a "&"
sigil to the front of the sub/method name and search in the
internal hash.
Submethods
Submethods should be handled by adding an additional Class
method "add_submethod" and adding another internal hash
(submethods can have the same name as methods, I believe).
Python
Subroutines and Namespaces
Since Python's subroutines and namespaces are just variables
(the namespace collides there), the Python PMC's "find_var"
method may return subroutines as variables.
Examples
Aliasing
Perl:
#!/usr/bin/perl6
sub foo {...}
%Foo::{"&bar"} = &foo;
PIR:
.sub main :main
$P0 = find_name "&foo"
$P1 = get_namespace ["perl6"; "Foo"]
# A smart perl6 compiler would emit this,
# because it knows that Foo is a perl6 namespace:
# $P1["&bar"] = $P0
# But a naive one would emit this:
$P1.add_sub("bar", $P0)
end
.end
.sub foo
...
.end
Cross-language Importing
Perl:
#!/usr/bin/perl
use tcl:Some::Module 'w*';
write("this is a tcl command");
PIR:
.sub main :main
.local pmc tcl
tcl = compreg "tcl"
tcl.load_library("Some", "Module")
$P0 = get_namespace
$P1 = get_namespace ["tcl"; "Some"; "Module"]
add_namespace ["perl5"; "Some"; "Module"], $P1
$P1.import_into($P0, 'w*')
write("this is a tcl command")
end
.end
Roger Browne
already I have a question:
> Synopsis
> - Languages should contain their namespaces
Suppose my application is multi-HLL. For example, some parts of it are
written in Python and some in Ruby. Suppose I take one small part that
is written in Python and decide to re-implement it in Ruby.
Does this mean that I _must_ change its namespace? Or may I write it in
Ruby but keep it within a Python-compliant namespace?
In other words, shouldn't namespaces and HLLs be orthogonal from
Parrot's point of view? I understand that most HLLs will want to use
their own conventions, and some will want a unique namespace prefix, but
should this be forced by Parrot?
Regards,
Roger Browne
Michael Lacey
From an application programmer's (which is what I used to be) point of view
I'd want to be able to write/use code from multiple HLLs without any danger
of them stamping all over each other's data.
I'd assumed Parrot would be enforcing namespace integrity and not assuming
the "next layer up" was doing it correctly.
What did I miss?
Mike
--
Mike Lacey
Project Manager
Partner Services
07717 458 268
--
Mike Lacey
Project Manager
Partner Services
07717 458 268
Roger Browne
> I'd want to be able to write/use code from multiple HLLs without any danger
> of them stamping all over each other's data.
As I see it, HLLs are just languages, and it's not the _language_ that
"stamps all over" the data. It's really the code-generating tools that
have the potential to mess about with data - and those tools don't
necessarily have a one-to-one relationship with languages.
Consider, for example, Perl 6.0 and Perl 6.1 - they will almost
certainly use namespaces in a compatible way - but there is no way that
Parrot can enforce that. Should Parrot force them to use different
namespaces, or should Parrot leave that to the discretion of the Perl
6.1 tool designer?
And what about some future "Python for Parrot" and "Iron Python for
Parrot"? Should Parrot force them to both use namespace "_python"? Or
should Parrot require them to use different namespaces? Or should Parrot
just leave that to the discretion of the tool designers?
> I'd assumed Parrot would be enforcing namespace integrity...
If we can define some rigorous notion of "namespace integrity", then we
can consider whether that integrity can be best protected by Parrot, or
by HLL code-generating tools, or by language-neutral tools (for example
some kind of "namespace lint" tool).
In the meantime, I think the best we can do is to leave it up to each
tool author to use the namespaces that they are competent to use. If a
certain tool doesn't play nicely with the namespaces that it touches,
then people will simply stop using that tool.
Regards,
Roger Browne
Michael Lacey
> that "stamps all over" the data. It's really the code-generating
> tools that have the potential to mess about with data
Quite so, tho I wasn't aware that the tools might not have a one-to-one
relationship with languages.
I was expecting Parrot to force code generating tools to specify the
namespace they want to work in.
I'm probably looking at this far too simplistically but I was expecting to
read something like this.
Naming Conventions
For interoperability, languages will use hierarchical namespaces; this
will be enforced using HLL Private Namespaces.
HLL Private Namespaces
HLLs must specify their private namespace using the
".hll_prv_namespace" directive
User Defined Namespaces
All HLLs will create namespaces within the HLL's Private Namespace.
This eliminates any
accidental collisions between languages.
I hope people will pardon me butting in. I'm not a language designer nor
even much of a language *user* anymore; I help people fit things together
that I don't begin to understand and I watch this list because I find the
way that people work together here interesting.
Mike
--
Mike Lacey
Project Manager
Partner Services
07717 458 268
On 02/12/05, Roger Browne <ro...@eiffel.demon.co.uk> wrote:
>
Jerry Gay
> After many months and lots of work, I'm happy to present you with the
> latest namespace spec draft. Comments are most welcome: to quote Chip,
> "The rest of the discussion would benefit from more eyes."
>
be like everyone else and say this will take some time to digest, but
i have a question now.
> Naming Conventions
> For interoperability, languages should use hierarchical namespaces.
> There's no way to outlaw flat namespaces, of course, and that remains an
> option for language implementors, but a standard must be chosen for easy
> interoperability, and hierarchical seems to be the better choice.
>
> HLL Private Namespaces
> HLLs should use a namespace with an underscore and the lowercased
> name of the HLL to store any private items. For instance, Tcl should
> use the "_tcl" namespace to store the subroutines that make up the
> compiler.
>
i don't understand the reason for lowercasing in namespaces. i mean,
won't there be collisions if someone creates a PERL compiler? perhaps
you could elaborate on this point a bit.
> User Defined Namespaces
> All HLLs should prefix any namespaces with the lowercased name of
> the HLL (so there's no question of what to capitalize). So Perl 5's
> CGI module should be stored under perl5::CGI. This eliminates any
> accidental collisions between languages.
>
why would there be a question of capitalization if the HLL name is
used as is, instead of lowercased? (e.g. Perl5::CGI.) or, in the case
of my example above, PERL::CGI and Perl::CGI. i understand it looks
like there won't be a Perl namespace, but one for each version, but
i'm sure you get the point. maybe i don't understand who (or what) is
responsible for doing the capitalization.
~jerry
Matt Diephouse
Alright, let me lay out my motivation for saying this:
Pugs already has (limited?) support for using Perl 5 modules inside of
Perl 6. The syntax they've used for this is C<use perl5:DBI>, which is
to say that they've lowercased the name of the language. And I rather
like that.
With that in mind, there are two possible ways to name namespaces and compilers:
1. Lowercase or uppercase them all. The Pugs code works with little or
no effort.
2. Capitalize them properly (ie, Perl5). Now there needs to be a
special mapping from "perl5" to "Perl5". And Pugs now has to learn
about each language that it wants to use before it can use it.
(Alternately, Pugs could start using C<use Perl5:DBI>, but I'm not
really considering that.)
(2) is what I'm hoping to avoid when I say that we should lowercase
them (and lowercase the names of compilers).
Nicholas Clark
> With that in mind, there are two possible ways to name namespaces and compilers:
>
> 1. Lowercase or uppercase them all. The Pugs code works with little or
> no effort.
And always use Unicode Normalised form C?
Nicholas Clark
Leopold Toetsch
On Dec 2, 2005, at 7:31, Matt Diephouse wrote:
[ Just a few notes, more to come. I've to read it some more times. ]
> Naming Conventions
> HLL Private Namespaces
> HLLs should use a namespace with an underscore and the
> lowercased
> name of the HLL to store any private items. For instance, Tcl
> should
> use the "_tcl" namespace to store the subroutines that make up
> the
> compiler.
A compiler can use whatever. The created code should conform to the
final namespace spec.
> Interfaces: Generic and Typed
This seems to add complexity and I doubt it solves anything. The
import/export code has to deal with sigils and whatever. $Module::bar
variables are rare and usually there is another method to access such a
var.
> .... This interface consists of the following methods:
Additionally to the debatable name 'add' ...
> add($S0, $P0)
... in combination with ...
> Class PMC API
> Class should inherit from the Namespace PMC.
... it seems to be unimplementable. A namespace is a class (PMC) that
defines it's API in terms of methods, which need a namespace for method
lookup, to locate the 'add' method, ...
To me the whole namespace stuff can just be defined in terms of
interoperbility. One compiler alone can do whatever it likes. but *if*
the code should be usable by other languages, it has to conform to
fixed rules.
E.g.
- Namespaces should be hierarchical
must be enforced for the 2nd case.
E.g. 'Perl5::CGI' can't be accessed by Python, but a canonical form
['Perl5'; 'CGI'] can easily be created by the Python importer from a
pythonic name Perl5.CGI (which doesn't have quotes).
I'm missing the policy in this proposal, e.g. what is allowed to be a
top-level global, how are HLL namespaces organized. And of course:
where is the Parrot namespace for it's PMCs.
And then we should look at implementation details.
My 2c so far,
leo
Matt Diephouse
>
> On Dec 2, 2005, at 7:31, Matt Diephouse wrote:
>
> [ Just a few notes, more to come. I've to read it some more times. ]
>
> > Naming Conventions
>
> > HLL Private Namespaces
> > HLLs should use a namespace with an underscore and the
> > lowercased
> > name of the HLL to store any private items. For instance, Tcl
> > should
> > use the "_tcl" namespace to store the subroutines that make up
> > the
> > compiler.
>
> A compiler can use whatever. The created code should conform to the
> final namespace spec.
The point of this is that if you need more than one compiler to
compile some code, you don't want the compilers trouncing all over
each other. The compilers need to use different namespaces for their
internals and this is the only way that really makes sense.
> > Class PMC API
> > Class should inherit from the Namespace PMC.
>
> ... it seems to be unimplementable. A namespace is a class (PMC) that
> defines it's API in terms of methods, which need a namespace for method
> lookup, to locate the 'add' method, ...
I knew this part was a bit shaky, I just didn't see quite why or how
much. Thanks.
> To me the whole namespace stuff can just be defined in terms of
> interoperbility. One compiler alone can do whatever it likes. but *if*
> the code should be usable by other languages, it has to conform to
> fixed rules.
> E.g.
> - Namespaces should be hierarchical
>
> must be enforced for the 2nd case.
> E.g. 'Perl5::CGI' can't be accessed by Python, but a canonical form
> ['Perl5'; 'CGI'] can easily be created by the Python importer from a
> pythonic name Perl5.CGI (which doesn't have quotes).
That's exactly the point of all this.
> I'm missing the policy in this proposal, e.g. what is allowed to be a
> top-level global, how are HLL namespaces organized. And of course:
> where is the Parrot namespace for it's PMCs.
I don't think I follow this part.
Roger Browne
In my previous messages, I was concerned that languages were being
shoehorned too tightly into their own namespaces.
But after a careful reading about "import_into", I am happy that each
language has sufficient ability to insert its symbols into the namespace
of another language (when asked to in the proper way, of course).
So I retract the concerns that I expressed. But I still find the intent
of the spec unclear here:
- Languages should contain their namespaces
because the "add_namespace" opcode seems to allow arbitrary namespaces
to be created. Maybe the strongest thing we can say is:
- All namespaces are organized under first-level namespaces
whose names will typically correspond to HLL names
If the intention is to say anything stronger, then we must say from
_where_ the first-level namespace names are derived.
2. TYPED INTERFACE
I assume it's an error that both "add_var($S0, $P0)" and "store_var($S0,
$P0)" are listed in the typed interface API for namespaces. I assume
that "add_var" is the intended name.
3. "name()" METHOD
The example suggests that perl5:Some::Module would return an array
containing "Some", "Module". Instead, don't we want it to return
"perl5", "Some", "Module" to match the params required by the
"get_namespace" opcode?
If not, then we need some other way to get the language name.
4. LANGUAGES AND COMPILERS
There are at least four notions of "Language" here:
(1) The language named in the .HLL directive.
(2) The name registered with "compreg".
(3) The name of the first-level namespace.
(4) The programming language in which the source code
was originally written.
Presumably we can ignore (4) because, by the time the code is compiled,
Parrot doesn't know or care what the original language was.
Of the other three, which of them are necessarily the same? If (3) must
be the same as (1) and/or (2), then we probably cannot force (3) to be
all-lowercase unless we force the others to be all-lowercase too.
5. THE FINAL EXAMPLE
In the final example, what is the purpose of this line:
add_namespace ["perl5"; "Some"; "Module"], $P1
As far as I can tell, the intent of the following line is to import
tcl:Some::Module 'w*' into the _current_ perl namespace, not into
["perl5"; "Some"; "Module"]. Am I missing the point here?
Regards,
Roger Browne
Matt Diephouse
> 1. LANGUAGES AND NAMESPACES
>
> In my previous messages, I was concerned that languages were being
> shoehorned too tightly into their own namespaces.
>
> But after a careful reading about "import_into", I am happy that each
> language has sufficient ability to insert its symbols into the namespace
> of another language (when asked to in the proper way, of course).
>
> So I retract the concerns that I expressed. But I still find the intent
> of the spec unclear here:
>
> - Languages should contain their namespaces
Yes, I wasn't sure how to word that best.
> because the "add_namespace" opcode seems to allow arbitrary namespaces
> to be created. Maybe the strongest thing we can say is:
>
> - All namespaces are organized under first-level namespaces
> whose names will typically correspond to HLL names
That's better, thanks.
> 2. TYPED INTERFACE
>
> I assume it's an error that both "add_var($S0, $P0)" and "store_var($S0,
> $P0)" are listed in the typed interface API for namespaces. I assume
> that "add_var" is the intended name.
Good catch.
> 3. "name()" METHOD
>
> The example suggests that perl5:Some::Module would return an array
> containing "Some", "Module". Instead, don't we want it to return
> "perl5", "Some", "Module" to match the params required by the
> "get_namespace" opcode?
Another good catch. This method was new in the latest draft.
> 4. LANGUAGES AND COMPILERS
>
> There are at least four notions of "Language" here:
>
> (1) The language named in the .HLL directive.
>
> (2) The name registered with "compreg".
>
> (3) The name of the first-level namespace.
>
> (4) The programming language in which the source code
> was originally written.
>
> Presumably we can ignore (4) because, by the time the code is compiled,
> Parrot doesn't know or care what the original language was.
>
> Of the other three, which of them are necessarily the same? If (3) must
> be the same as (1) and/or (2), then we probably cannot force (3) to be
> all-lowercase unless we force the others to be all-lowercase too.
I was advocating having (2) and (3) be all-lowercase. I'm not sure
which are necessarily the same.
> 5. THE FINAL EXAMPLE
>
> In the final example, what is the purpose of this line:
>
> add_namespace ["perl5"; "Some"; "Module"], $P1
>
> As far as I can tell, the intent of the following line is to import
> tcl:Some::Module 'w*' into the _current_ perl namespace, not into
> ["perl5"; "Some"; "Module"]. Am I missing the point here?
No, I meant to take that line out but I forgot.
Thanks, Roger. Your email was helpful.
Roger Browne
> HLLs should use a namespace with an underscore and the lowercased
> name of the HLL to store any private items. For instance, Tcl should
> use the "_tcl" namespace to store the subroutines that make up the
> compiler.
Is it the intention that Parrot will enforce that these names are kept
private?
If so, it would be good to say so.
If not, then the HLL can store its private stuff however it wants (under
its own first-level namespace), and define "import_into" so that it will
refuse to share its private stuff. In that case, the HLL is responsible
for keeping its own private stuff private, and there is no need for the
spec to refer to unenforced private namespaces.
Regards,
Roger Browne
Roger Browne
> import_into($P0, ...)
> Import items from the current namespace into the namespace in $P0.
It might be clearer to change the name to "export_into", because it is
being called on - and controlled by - the source:
source_namespace.export_into(target_namespace, options...)
Regards,
Roger Browne
Matt Diephouse
> > HLL Private Namespaces
> > HLLs should use a namespace with an underscore and the lowercased
> > name of the HLL to store any private items. For instance, Tcl should
> > use the "_tcl" namespace to store the subroutines that make up the
> > compiler.
>
> Is it the intention that Parrot will enforce that these names are kept
> private?
Nope.
ParTcl is written in PIR. Other compilers may also be written in PIR.
There are two things we want to avoid: conflicting symbols between
different compilers and conflict symbols between compilers and
user-defined code.
If compilers don't place their symbols in a namespace other than the
root namespace, two compilers may end up defining different symbols
that have the same name. ParTcl has a compile() sub, for instance. If
another compiler were to introduce a compile() sub in the same
namespace, things would break. So we need to put these somewhere.
You've suggested putting these internal subs under "tcl". But what
happens if the user tries to define a compile() sub in Tcl's root
namespace? Now we have the some problem: 2 different symbols with the
same name. We can't just change the internal sub to be _compile(),
because the user could also define a _compile(). We could prefix all
user-defined subs with some char, but that's a far amount of work.
Which is why I've suggested private HLL namespaces for compiler internals.
> If so, it would be good to say so.
>
> If not, then the HLL can store its private stuff however it wants (under
> its own first-level namespace), and define "import_into" so that it will
> refuse to share its private stuff. In that case, the HLL is responsible
> for keeping its own private stuff private, and there is no need for the
> spec to refer to unenforced private namespaces.
I don't think you'll really be interested in importing the guts of the
Tcl compiler into your program. But if that's what you really want,
I'm not going to stop you. (And neither should Parrot.)
Leopold Toetsch
On Dec 2, 2005, at 19:44, Matt Diephouse wrote:
> Leopold Toetsch <l...@toetsch.at> wrote:
>> I'm missing the policy in this proposal, e.g. what is allowed to be a
>> top-level global, how are HLL namespaces organized. And of course:
>> where is the Parrot namespace for it's PMCs.
>
> I don't think I follow this part.
Parrot PMCs are occupying top-level namespaces (now):
$ cat i.pir
.sub main :main
.local pmc i,m
i = new .Integer
i."bark"()
.end
.namespace ['Integer']
.sub bark :method
print "miau\n"
.end
$ ./parrot i.pir
miau
Further: all internal multisubs for the infix ops like "__add" need a
namespace for storage, currently it is: "__parrot_core".
leo
Leopold Toetsch
On Dec 2, 2005, at 7:31, Matt Diephouse wrote:
> Typed Interface
> add_sub($S0, $P0)
> add_namespace($S0, $P0)
> add_var($S0, $P0)
Which HLLs would use these interfaces? Can you please provide some
examples of HLLs with the usage of these interfaces.
Thanks,
leo
Roger Browne
> > add_sub($S0, $P0)
>
> > add_namespace($S0, $P0)
>
> > add_var($S0, $P0)
>
> Which HLLs would use these interfaces?
Maybe I'm missing the point, but I see these being used in the
implementation of "import_into" as a way for the source HLL to tell the
target HLL whether to treat each name as a sub, namespace, variable or
method.
Regards,
Roger Browne
Matt Diephouse
Yes, that's correct.
Leopold Toetsch
And it doesn't answer my question at all, sorry. Which HLLs are able to
divide their symbols into above categories? Further: as this proposals
deals with the managment of namespaces, a special typed interface for a
'namespace' symbol name seems not to be necessary, because if it
weren't evident, where a namespace is used, we couldn't deal with
namespaces at all.
The current implementation disambiguates namespace names from symbol
names by prepending a '\0' char to the former. Therefore it's always
clear if a namespace or a 'variable' symbol is used.
Remaining for me is the distiction between a variable and a function
symbol:
- python: no (all names are just names)
- tcl: afaik no (all names are strings)
- perl5: sometimes (via sigil, but $ref_tosub)
- perl6: maybe (sigil is part of the symbol name, but $ref)
- ???
To make use of this distinction (var, function) we'd need a pair of
such languages, one that can do a typed export, and one that can do a
typed import of symbols.
What am I missing?
leo
Matt Fowles
LISP - yes
Matt
--
"Computer Science is merely the post-Turing Decline of Formal Systems Theory."
-Stan Kelly-Bootle, The Devil's DP Dictionary
Bob Rogers
Date: Sun, 4 Dec 2005 12:35:24 -0500
. . .
On 12/4/05, Leopold Toetsch <l...@toetsch.at> wrote:
>
> . . . Which HLLs are able to
> divide their symbols into above categories? . . .
>
> Remaining for me is the distiction between a variable and a function
> symbol:
>
> - python: no (all names are just names)
> - tcl: afaik no (all names are strings)
> - perl5: sometimes (via sigil, but $ref_tosub)
> - perl6: maybe (sigil is part of the symbol name, but $ref)
> - ???
LISP - yes
Matt
Actually, it's "yes" for Common Lisp, and "no" for Scheme. But there's
a bit more to it than that: Namespaces in Common Lisp map a name
(string) to a symbol, which is the object that holds the name's global
function and/or variable bindings, etc. The add_sub/add_var interface
can be implemented in terms of symbols, though; I'm still digesting the
consequences, but I don't believe it will require anything more of
Parrot.
-- Bob Rogers
http://rgrjr.dyndns.org/
Matt Diephouse
> And it doesn't answer my question at all, sorry. Which HLLs are able to
> divide their symbols into above categories? Further: as this proposals
> deals with the managment of namespaces, a special typed interface for a
> 'namespace' symbol name seems not to be necessary, because if it
> weren't evident, where a namespace is used, we couldn't deal with
> namespaces at all.
I added that interface specifically because of Python. Namespaces in
Python are also variables, so any time a namespace is created, it must
be stored as a variable. This wouldn't be a problem, except that if
anything besides Python itself creates the namespace, it wouldn't be
added as a variable.
Of course, now that I think about it more, it's possible that nothing
else will be adding namespaces for Python. In which case I'd advocate
having Parrot keep track of namespaces in a separate hash.
> Remaining for me is the distiction between a variable and a function
> symbol:
>
> - python: no (all names are just names)
Correct. Variables, functions, and namespaces all collide.
> - tcl: afaik no (all names are strings)
Incorrect. You can have a variable, a function, and namespace with the
same name.
> - perl5: sometimes (via sigil, but $ref_tosub)
> - perl6: maybe (sigil is part of the symbol name, but $ref)
Functions, variables, and namespaces _are_ separate here. Don't let
the fact you can make a variable reference a sub throw you off. They
are separate entries in the symbol table and separate entities.
> To make use of this distinction (var, function) we'd need a pair of
> such languages, one that can do a typed export, and one that can do a
> typed import of symbols.
Not really. It's a useful distinction if one of the languages makes
the distinction. Let's take the case of Tcl and Python. Remember that
functions and variables overlap in Python but don't overlap in Tcl.
If you import a function from Tcl into Python, it should be a
variable. That's fine. add_sub can take care of that. That's the
intention. It'd work the same if you used add_var, but that doesn't
matter. The important thing is that Tcl doesn't need to know anything
about Python.
Importing from Python to Tcl is a different matter. Python's function
may be variables, but Python still knows that they're functions. So
Python can use add_sub to add the function to Tcl's namespace. This is
both useful and necessary because function and variable names don't
overlap in Tcl.
> What am I missing?
I hope this clears things up a bit.
Roger Browne
> And it doesn't answer my question at all, sorry. Which HLLs are able to
> divide their symbols into above categories?
Ah, maybe I see what you're getting at.
At compile-time, a HLL knows whether it is compiling a sub or a
variable. But at runtime, the HLL may or may not know whether a name is
associated with a sub or a variable.
But I see two cases where the HLL will be able to successfully use
'add_sub' and 'add_var':
1. If the HLL supports reflection/introspection (the ability of a
running program to learn about its own types), then it should certainly
have enough information to be able to use 'add_sub'/'add_var'.
2. The HLL may choose to store some kind of symbol table within its
private namespace (for the express purpose of supporting
'add_sub'/'add_var'). Amber will do this, because Amber is a minority
language and will be critically dependent on interoperability to be
viable. I can understand that the "big players" might not be bothered,
though.
Any HLL that doesn't know which names are subs can just call 'add_var'
for everything and let the receiving HLL cope as best it can.
> Remaining for me is the distiction between a variable and a function
> symbol:
>
> - python: no (all names are just names)
This is an orthogonal issue, no? Python can't have a variable and a sub
both named "foo", but that's unrelated to whether it knows if some
particular "foo" is a variable or a sub.
Regards,
Roger Browne
Leopold Toetsch
On Dec 5, 2005, at 5:55, Matt Diephouse wrote:
> Leopold Toetsch <l...@toetsch.at> wrote:
> Of course, now that I think about it more, it's possible that nothing
> else will be adding namespaces for Python.
Or: only python itself can create Python namespaces.
> In which case I'd advocate
> having Parrot keep track of namespaces in a separate hash.
That's more an implementation detail than anything else, but yes, it
sounds reasonable. We have to look, how classes (and PMCs) fit into the
scheme.
>
>> Remaining for me is the distiction between a variable and a function
>> symbol:
>>
>> - python: no (all names are just names)
>
> Correct. Variables, functions, and namespaces all collide.
>
>> - tcl: afaik no (all names are strings)
>
> Incorrect. You can have a variable, a function, and namespace with the
> same name.
>
>> - perl5: sometimes (via sigil, but $ref_tosub)
>> - perl6: maybe (sigil is part of the symbol name, but $ref)
>
> Functions, variables, and namespaces _are_ separate here.
Namespaces are always distinct, let's keep them aside. All the opcodes
that deal with namespaces define the interface for a 'namespace'. E.g.
store_global "Foo", "bar", $P0
defines "Foo" being a namespace, or C<get_namespace> returns one, and
so on.
> Don't let
> the fact you can make a variable reference a sub throw you off. They
> are separate entries in the symbol table and separate entities.
How so?
our $a;
sub foo { sub {1} }
$a = foo();
Now you (the compiler) wants to store the 'scalar' $a into an
appropriate namespace. The compiler doesn't know that '$a' is a code
reference, therefore 'add_sub' isn't used.
>
>> To make use of this distinction (var, function) we'd need a pair of
>> such languages, one that can do a typed export, and one that can do a
>> typed import of symbols.
>
> Not really. It's a useful distinction if one of the languages makes
> the distinction. Let's take the case of Tcl and Python. Remember that
> functions and variables overlap in Python but don't overlap in Tcl.
>
> If you import a function from Tcl into Python, it should be a
> variable. That's fine. add_sub can take care of that. That's the
> intention. It'd work the same if you used add_var, but that doesn't
> matter. The important thing is that Tcl doesn't need to know anything
> about Python.
CPython bytecode or any other implementation can just issue a
'LOAD_NAME' or 'LOAD_GLOBAL' opcode for fetching a name. The Python
import can just ask for some 'tcl.module.name'. Now tcl has the names
split into 3 categories. Which implies that the tcl 'name-exporter' has
to look into all 3 categories and deliver a matching name (and fail, if
there are dpulicates, but only when exporting to Python, and not when
exporting internally).
This means that, while tcl internally has of course the split of the
names, it's not really useful for the export to other HLLs.
> Importing from Python to Tcl is a different matter. Python's function
> may be variables, but Python still knows that they're functions.
No, CPython doesn't know that:
a = 1
a()
...
2 6 LOAD_NAME 0 (a)
9 CALL_FUNCTION 0
That's absoletly the same bytecode as produced by the 2nd line of:
class a: pass
a()
> So
> Python can use add_sub to add the function to Tcl's namespace.
Nope. The tcl 'name-importer' has to check the type of the names to
DTRT.
> This is
> both useful and necessary because function and variable names don't
> overlap in Tcl.
It might be necessary but can't and doesn't work like this.
This all doesn't really preclude parrot support for split namespaces,
but I really see not much use for it, as most languages (that need) it,
know how do deal with it (e.g. Perl6: store the sigil with the name).
leo
Chip Salzenberg
> On Dec 5, 2005, at 5:55, Matt Diephouse wrote:
> >>- perl5: sometimes (via sigil, but $ref_tosub)
> >>- perl6: maybe (sigil is part of the symbol name, but $ref)
> >
> >fact you can make a variable reference a sub throw you off. They are
> >separate entries in the symbol table and separate entities.
>
> How so?
>
> our $a;
> sub foo { sub {1} }
> $a = foo();
$a is a variable here, not a subroutine. Full stop.
The fact that $a is holding a value that, when properly dereferenced, points
to a subroutine, is of no import. (No pun intended. :-)) The fact that
Perl 5 makes it somewhat easy, and Perl 6 makes it dead easy, to do that
dereferencing operation, is also of no import.
> This all doesn't really preclude parrot support for split namespaces,
> but I really see not much use for it, as most languages (that need) it,
> know how do deal with it (e.g. Perl6: store the sigil with the name).
The point is to support exporting from a unified-namespace language to a
split-namespace language, and vice versa.
Are there any issues with the current spec that prevent its implementation?
--
Chip Salzenberg <ch...@pobox.com>