Import, export, and syntax extensions

[Bob Nystrom]

With Magpie right now, there's some limitations and problems with how
importing and exporting from modules works that I'm working to
address. Here's the super summary for the current system:

Top-level methods, variables, and classes declared with public names
(i.e. not starting with "_") are automatically exported from the
module.

There are a couple of import forms:

import some.module

This imports everything from some/module.mag and defines it in the
importing module, in the scope where the "import" appears.

import some.module name

This imports just "name" from some/module.mag. If there is a variable,
method, and/or syntax extension named "name" it will be imported.

import some.module name = other

This imports just "name" from some/module.mag, but defines it in the
importing module with the name "other".

import some.module = blah

This imports everything from some/module.mag, but prefixes each name
with "blah.". So if some/module.mag has "foo", you'll import
"blah.foo".

Note that "import" is just another kind of expression, which means it
can appear anywhere. This means you can do scoped imports like so:

do
import io
val file = open("blah.txt")
end

open("other.txt") // ERROR: open isn't in scope any more

Problems
--------

This works OK, but has a couple of problems:

1. There's no way to re-export an imported name. Let's say you have:

// a.mag
def foo()
...
end

var bar = 123

defineInfix("someop", ...) // Syntax still in-progress...

// b.mag
import a

// c.mag
import b

Now let's say that "b" wants to re-export the names it imports from
"a" so that "c" can see them. There's no way to do that right now. You
can *kind* of do it for methods by just defining a new specialization
of the imported method. And you can *kind* of do it for variables by
importing it under a different name and then defining a new variable
with the original name in the importing module.

For syntax extensions, you're just totally hosed.

2. The syntax for renaming is non-obvious. When you do:

import some.module foo = bar

Is "foo" the new name, or "bar"? "as" is much clearer but is tricky
because it isn't (currently) a reserved word.

3. There's no way to import everything from a module *except* some
name. This can be useful if one or two names in a module collides and
you just want to rename those but import everything else as-is.
Scheme's module system allows this.

Syntax Extensions
-----------------

Somewhat related to this issue is syntax extensions. Right now, you
can extend the grammar using defineSyntax(). You can call that
anywhere, but it only takes effect after the current top-level
expression is finished executing.

This lets you extend the syntax and immediately use that extension
within the same source file (neat!) but also means source files have
to be incrementally interpreted (in other words, we start evaluating
code in a module before we even finish *parsing* the entire module).
That last bit doesn't play very nice with compilation.

It would be really nice if you could parse an entire source file in
one go, including any syntax extensions.

I have an idea on how to make that work, but I want to get some feedback on it.

Imported Syntax
---------------

The idea is that a syntax extension can't be used in the same file
where it's defined. When you define a syntax extension, it gets
exported from that module, but doesn't actually take effect in that
module.

If you want to use some syntax extension, you have to define it in
another module and import it. That way, once we've done all of the
imports for a module, it's grammar is fixed and we can parse (and
eventually compile) the whole thing after that.

There's still one missing piece, though. Right now, *imports* can
appear anywhere too. If those imports can change the grammar, then we
still have the original problem where the grammar can change in the
middle of the file.

To fix that, we could restrict imports to only appear at the beginning
of a module. This is a much more static solution, similar to Java, C#,
etc. Usability-wise, this doesn't hurt much: imports almost always
appear at the beginning of the top-level of a module anyway.

Everything is an Expression
---------------------------

The challenge is that it moves away from everything being an
expression. With this, "import" would either still be an expression
but then some later step would complain if you used it outside of its
designated area, or it would be some other non-expression special
form.

Leaving it an expression is good because it means it still plays nice
with quotations. You can still do things like:

{ import blah }

To generate an AST object that represents an import expression. But it
means we need some later step to handle imports appearing where they
don't belong.

Making it a special form addresses that, but then the grammar for the
language becomes more complex: a source file's top-level production
becomes some special thing like:

Program ::=
ImportDirective* Expression*

instead of just being a list of expressions like it currently is.

The Way Forward
---------------

So what I'm currently thinking is:

1. import expressions will still be expressions, but it will be a
(static) error to use one anywhere but at the top of the program. This
keeps the grammar simple and plays nice with quotations. Really, it's
no worse than having something like this:

do
return "blah"
unreachable(code)
end

That "unreachable(code)" line is obviously wrong but still OK as far
as the grammar is concerned, and it doesn't seem to be the end of the
world.

2. The import syntax will change to allow exporting, renaming, and
excluding. This will evolve over time, but I'm currently thinking:

import foo.bar

Import foo/bar.mag, same as before.

import foo.bar as blah

Import foo/bar.mag, but prefix everything with "blah.".

import foo.bar with
exclude blah
bang as bong
end

Import foo/bar.mag, but not "blah". Import "bang" from it but renamed to "bong".

import foo.bar with only
blah
bang
end

Just import "blah" and "bang" from foo/bar.mag, and nothing else.

import foo.bar with only blah

Shorthand for the above if you only want to import a single name.

import foo.bar with
export bang
end

Import everything from foo/bar.mag, including "bang", and then
re-export "bang" from the importing module. In general, "export" and
"as" can be combined and used anywhere a name can appear, so this is
valid too:

import foo.bar with only export bang as bong

Taking a rough stab at a grammar, something like:

Import ::=
| "import" Identifier ("as" Identifier)? Qualifier?

Qualifier ::=
| "with" BlockOrSingle
| "with" "only" BlockOrSingle

BlockOrSingle ::=
| NamedImport
| "\n" NamedImport ("\n" NamedImport)* "end"

NamedImport ::=
| "exclude" Identifier
| "export"? Identifier ("as" Identifier)?

An "exclude" can only appear inside a "with" body, and not in a
single-line or "with only" import (since it would be useless).

Thoughts?

- bob

[Elliott Hird]

I dislike the use of = for renaming, and also its overloading to both
renaming of imported names and renaming of the imported module itself.

I parse "import ... = ..." as "(import ...) = ...", which is
problematic.

I suggest instead:

import some.module
import some.module <- foo, bar, baz
import some.module <- foo, bar, baz (quux) (renaming baz to
quux)
import some.module (blah)

the parens evoking "a doxiflimux dupliplector (dosiflector) is a kind
of omptidium. A dosiflector ..."

So for instance

import math (m) <- sin, cos, tangent (tan)

Also, I dislike the "if it exists, it will be imported" wording.
Requesting a name that does not exist should be an error.

[Elliott Hird]

Oh, and I think the "with only" stuff is unnecessary; if you just say

import foo.bar <- blah

then the only side-effect is having foo.bar in your namespace, and
reusing that would be a Bad Idea /anyway/ in case you ever want to
import the module for real. Or do you mean that it would import it as
"foo.bar.blah", but e.g. "foo.bar.quux" would not exist even if it is
in foo.bar? I think this is a bad idea too: a qualified module should
be "the same everywhere".

As far as re-exporting goes, I think it should be a separate
statement, rather than shoe-horned into the semantically unrelated
import statement.

As for exclude, I am not sure of its use. If it simply does not import
a name into the current namespace, then it is a nop unless you also
list the same name in the import list (and why would you?). If it
excludes from the qualified namespace, then see my previous opinion
that a qualified module should always have the same contents.

But if you have an

import foo.bar <- *

construct to import everything in a specific namespace -- which I
think is a good idea, for things like internal utility modules:

import myprog.utils <- *

then perhaps a hiding facility is useful (though I would consider it a
symptom of bad naming or organisation).

Then, how about

import myprog.utils <- * - dontWant

?

[Bob Nystrom]

On Thu, Jun 30, 2011 at 8:19 PM, Elliott Hird
<penguino...@googlemail.com> wrote:
> I dislike the use of = for renaming, and also its overloading to both
> renaming of imported names and renaming of the imported module itself.
>
> I parse "import ... = ..." as "(import ...) = ...", which is
> problematic.

Agreed, it's confusingly ambiguous. A keyword is better.

> import some.module <- foo, bar, baz

I'm not sure any punctuation will be clear here. I think punctuation
works well when we already know what it means (+, *, etc.) but
otherwise I think readable works are better. In general, Magpie tends
towards keywords over symbols.

For your specific suggestion, doesn't the arrow point the wrong way? I
think of it as importing *from* some.module *into* foo, bar, and baz,
which implies the arrow would go the other way.

Also, I worry that a comma-separated list doesn't scale well if you're
importing a long list of specific names (and/or renaming them). You
could wrap it across multiple lines like:

import some.module <-
foo,
bar,

baz,

But that looks out of place with the rest of the syntax to me.

> import some.module <- foo, bar, baz (quux)       (renaming baz to quux)

I like that that's terse, but it has the same problem to me that "="
has: it isn't apparent what it means. You just have to know "the thing
in parens is the rename", there's no way to infer it. Whereas, I like
to think it's intuitable what's going on here:

import some.module with
baz as quux
end

> Also, I dislike the "if it exists, it will be imported" wording.
> Requesting a name that does not exist should be an error.

Correct, I was unclear there. If you do a specific named import, it
should successfully import *something* with that name. the "if it
exists" is because a named import can refer to one or more of the
following: a variable, a multimethod, a prefix keyword, or an infix
keyword. When you import by name, you can't actually specify which of
those you want: you will import any of those that exist with that
name.

You're right that it should be an error to do

import some.module with
foo
end

And have it not find anything named "foo" to import.

- bob

[Bob Nystrom]

On Thu, Jun 30, 2011 at 8:23 PM, Elliott Hird
<penguino...@googlemail.com> wrote:
> Oh, and I think the "with only" stuff is unnecessary; if you just say

There are a few use cases:

1. I want to import everything this module exports.
2. I want to import everything this module exports, but this one thing
in there collides with something else I'm defining, so I want to
exclude or rename it.
3. I want to import everything that this module exports, and re-export
a couple of its methods from my module.
4. I just want to import a specific couple of things from this module.

1. is what Magpie assumes you'll be doing most of the time. It's
simple, terse, and (from what I've found) rarely causes name
collisions. Because 1. is the assumed default, when you do name
imports, it assumes you're doing so for use cases 2 or 3. So if you
do:

import math with
abs as absolute
end

It will still import everything else math exports, it's just renaming
"abs". So "with only" (which isn't implemented yet) lets you
distinguish that case from case 4.

>
> import foo.bar <- blah
>
> then the only side-effect is having foo.bar in your namespace,

That doesn't import "foo.bar" into the importing module's scope.
"foo.bar" just refers to a module, which isn't (currently) a
first-class concept in Magpie. What that would do is:

1. Ensure that the foo.bar module has been loaded and evaluated.
2. Define a new variable in the current module named "blah".
3. Assign it the value of the variable of the same name in foo.bar's
top-level scope.

If you were to import it with a qualified name, like:

import foo.bar < foo.bar.blah

That does *not* mean that you'll get some "foo" variable in the
top-level of your module and it has a "bar" property which has a
"blah" property. That's how Python works, but Magpie is distinctly
different there. Keep in mind that "." isn't even used for method
calls. It's just an identifier character. So if you import with a
qualified name, it means you'd have a variable whose name is
"foo.bar.blah".

> Or do you mean that it would import it as
> "foo.bar.blah", but e.g. "foo.bar.quux" would not exist even if it is
> in foo.bar?

If I understand you, that's correct. If you just want "foo.bar.blah",
that's all you get.

> I think this is a bad idea too: a qualified module should
> be "the same everywhere".

Magpie doesn't actually give you any reference to the module. It just
binds names to things you've imported from it. Magpie isn't an
"objects are bags of named properties" language (anymore, at least!)

> As far as re-exporting goes, I think it should be a separate
> statement, rather than shoe-horned into the semantically unrelated
> import statement.

I considered that. Something like:

import math
export abs // I want this module to expose "abs" as if it defined it

My main concern is that by not tying it to "import" I fear it leads
people to think they need to do that for all exported identifiers:

def foo()
/// Define some method in this module.
...
end

export foo // Don't forget to export it!

But that last line is wrong. Methods defined at the top-level without
private names are automatically exported. The *only* thing you need to
explicitly export is something that you've imported (because imports
are *not* automatically re-exported). So from that, it made sense to
me to tie exporting to importing.

> As for exclude, I am not sure of its use. If it simply does not import
> a name into the current namespace, then it is a nop unless you also
> list the same name in the import list (and why would you?).

I may have answered this already, but just to be clear. My assumption
is that users will very rarely use fully qualified names. Short names
are easier to read and fully-qualified names are really only helpful
in the case of collisions. So exclude exists (or will, when/if I
implement it) for cases where you want to import an entire module
unqualified (which is the common case) but it happens to have one or
two things that collide that you don't even need. So just exclude them
and keep on truckin'.

>  import foo.bar <- *
>
> construct to import everything in a specific namespace -- which I
> think is a good idea, for things like internal utility modules:

That's what import foo.bar does: import everything the module exports
into this module, without qualifier. I generally despise qualified
names. When I see stuff like Math.abs(-3), I kind of want to cry.

> then perhaps a hiding facility is useful (though I would consider it a
> symptom of bad naming or organisation).

It's probably a symptom of bad naming if you have to exclude to hide a
collision in your own module. But it's entirely possible to be
importing from A and B, two modules who know nothing of each other,
and have a collision between them. Imagine you're parsing XML and
tossing it into some graph library. Both could easily define "Node"
and neither is wrong for doing so. But if your module tries to use
them both, it'll need to disambiguate.

>  import myprog.utils <- * - dontWant

Is that excluding "dontWant" or importing both "-" and "dontWant"?
Operators are valid identifiers too! :)

- bob

[Elliott Hird]

On Jul 1, 7:37 am, Bob Nystrom <munificent...@gmail.com> wrote:
> Agreed, it's confusingly ambiguous. A keyword is better.

I don't think this conclusion follows from the evidence; something /
different/ is better, certainly, but this might not be a keyword.

> > import some.module <- foo, bar, baz
>
> I'm not sure any punctuation will be clear here. I think punctuation
> works well when we already know what it means (+, *, etc.) but
> otherwise I think readable works are better. In general, Magpie tends
> towards keywords over symbols.

I admit that I am a fan of judiciously-used punctuation; not to the
levels of J (ironically, a language I love), but a lot of code in e.g.
Lua tends to be unreadable for me, drowned under a sea of cluttering
and alike-looking keywords.

> For your specific suggestion, doesn't the arrow point the wrong way? I
> think of it as importing *from* some.module *into* foo, bar, and baz,
> which implies the arrow would go the other way.

Foo, bar, and baz /come from/ some.module. Think of a module as a
present; we're saying that foo, bar, and baz are packed into this
present. The other way around suggests that some.module is packed into
foo, bar and baz.

> Also, I worry that a comma-separated list doesn't scale well if you're
> importing a long list of specific names (and/or renaming them). You
> could wrap it across multiple lines like:
>
>   import some.module <-
>       foo,
>       bar,
>       baz,
>
> But that looks out of place with the rest of the syntax to me.

import some.module <-
foo

bar
baz
end

?

> > import some.module <- foo, bar, baz (quux)       (renaming baz to quux)
>
> I like that that's terse, but it has the same problem to me that "="
> has: it isn't apparent what it means. You just have to know "the thing
> in parens is the rename", there's no way to infer it. Whereas, I like
> to think it's intuitable what's going on here:
>
>   import some.module with
>     baz as quux
>   end

Well, certainly. But every language has a learning curve -- recall
Raskin, "intuitive equals familiar". In this case, I think you'll be
typing and reading import statements often enough that a syntax that
gets out of the way and lets you see what is actually going on quickly
as an experienced user is worth the ten minute learning curve. After
all, imports are /vital/ for understanding the piece of code that
follows; they tell you where it is executing, what environment it is
in.

> Correct, I was unclear there. If you do a specific named import, it
> should successfully import *something* with that name. the "if it
> exists" is because a named import can refer to one or more of the
> following: a variable, a multimethod, a prefix keyword, or an infix
> keyword. When you import by name, you can't actually specify which of
> those you want: you will import any of those that exist with that
> name.

Right. (Perhaps you should be able to? That could have weird semantics-
changing bugs.)

[Elliott Hird]

On Jul 1, 8:00 am, Bob Nystrom <munificent...@gmail.com> wrote:
> 1. I want to import everything this module exports.

I cover this later.

> 2. I want to import everything this module exports, but this one thing
> in there collides with something else I'm defining, so I want to
> exclude or rename it.

Ditto. For renaming, how about

import myprog.utils <- * dontWant (dw)

? Yeah, this is ugly and weird. I don't know about this one; I'll have
to think about it.

> 3. I want to import everything that this module exports, and re-export
> a couple of its methods from my module.

Yeah, I'm pretty firmly of the view that what a module exports should
not be tied into its import statements.

> 4. I just want to import a specific couple of things from this module.
>
> 1. is what Magpie assumes you'll be doing most of the time. It's
> simple, terse, and (from what I've found) rarely causes name
> collisions. Because 1. is the assumed default, when you do name
> imports, it assumes you're doing so for use cases 2 or 3. So if you
> do:

This irks me as a fan of namespace separation and makes me recall the
horror of namespaceless C, but then I suppose if the convention is to
define namespaces inside the modules anyway (a la Ruby), that would be
fine. But if that is the convention, then what is the module system
for?

>   import math with
>       abs as absolute
>   end
>
> It will still import everything else math exports, it's just renaming
> "abs". So "with only" (which isn't implemented yet) lets you
> distinguish that case from case 4.

Well, I am working from the POV of importing everything being a
separate statement, yeah.

> >  import foo.bar <- blah
>
> > then the only side-effect is having foo.bar in your namespace,
>
> That doesn't import "foo.bar" into the importing module's scope.
> "foo.bar" just refers to a module, which isn't (currently) a
> first-class concept in Magpie. What that would do is:

Oh, well, IM-very-strong-O, modules as first-class concepts are vital!
Otherwise, you will just end up reinventing first-class modules with
e.g. classes. (I am a great fan of the ML module system and I think
you might be too -- it allows you to parameterise a library or program
on the implementations of various data structures you use, or have a
"collection utilities" module that takes a collection module and
transforms it into a utilities module for that collection. It reminds
me of dependency injection, but done right.)

> 1. Ensure that the foo.bar module has been loaded and evaluated.
> 2. Define a new variable in the current module named "blah".
> 3. Assign it the value of the variable of the same name in foo.bar's
> top-level scope.

Right. First-class modules would likely change this story a lot.

> If you were to import it with a qualified name, like:
>
> import foo.bar < foo.bar.blah
>
> That does *not* mean that you'll get some "foo" variable in the
> top-level of your module and it has a "bar" property which has a
> "blah" property. That's how Python works, but Magpie is distinctly
> different there. Keep in mind that "." isn't even used for method
> calls. It's just an identifier character. So if you import with a
> qualified name, it means you'd have a variable whose name is
> "foo.bar.blah".

Right.

> > Or do you mean that it would import it as
> > "foo.bar.blah", but e.g. "foo.bar.quux" would not exist even if it is
> > in foo.bar?
>
> If I understand you, that's correct. If you just want "foo.bar.blah",
> that's all you get.

You don't; my question only makes sense from the POV of first-class
modules.

> > I think this is a bad idea too: a qualified module should
> > be "the same everywhere".
>
> Magpie doesn't actually give you any reference to the module. It just
> binds names to things you've imported from it. Magpie isn't an
> "objects are bags of named properties" language (anymore, at least!)

See above :) BTW, with an ML module system, modules have signatures --
modules : signatures :: values : types. This means that you could have
all the powerful concepts that this (IMO best-breed) first-class
module system offers while keeping the kind of "optional static
typing" that Magpie has even at the module layer.

> > As far as re-exporting goes, I think it should be a separate
> > statement, rather than shoe-horned into the semantically unrelated
> > import statement.
>
> I considered that. Something like:
>
>   import math
>   export abs // I want this module to expose "abs" as if it defined it

I would have it part of some "module header", but then I would also
have regular exports this way. IMO every library should list
everything it exports to avoid accidental exports. Internal program
modules can just "exports *" without worrying.

> My main concern is that by not tying it to "import" I fear it leads
> people to think they need to do that for all exported identifiers:
>
>   def foo()
>       /// Define some method in this module.
>       ...
>   end
>
>   export foo // Don't forget to export it!
>
> But that last line is wrong. Methods defined at the top-level without
> private names are automatically exported. The *only* thing you need to
> explicitly export is something that you've imported (because imports
> are *not* automatically re-exported). So from that, it made sense to
> me to tie exporting to importing.

Yeah. But then, I question the designing of such things around really
easy to clean up misconceptions.

With my ideal system, though, you actually would have to do that (but
different; or just say *), so the problem is gone :)

> > As for exclude, I am not sure of its use. If it simply does not import
> > a name into the current namespace, then it is a nop unless you also
> > list the same name in the import list (and why would you?).
>
> I may have answered this already, but just to be clear. My assumption
> is that users will very rarely use fully qualified names. Short names
> are easier to read and fully-qualified names are really only helpful
> in the case of collisions. So exclude exists (or will, when/if I
> implement it) for cases where you want to import an entire module
> unqualified (which is the common case) but it happens to have one or
> two things that collide that you don't even need. So just exclude them
> and keep on truckin'.

[addressed previously]

>
> >  import foo.bar <- *
>
> > construct to import everything in a specific namespace -- which I
> > think is a good idea, for things like internal utility modules:
>
> That's what import foo.bar does: import everything the module exports
> into this module, without qualifier. I generally despise qualified
> names. When I see stuff like Math.abs(-3), I kind of want to cry.

That's just because abs is a vital name in any namespace. Would you
say the same for SDL.init() or GL.flush() or GTK.blahblahblah()
or ...?

> > then perhaps a hiding facility is useful (though I would consider it a
> > symptom of bad naming or organisation).
>
> It's probably a symptom of bad naming if you have to exclude to hide a
> collision in your own module. But it's entirely possible to be
> importing from A and B, two modules who know nothing of each other,
> and have a collision between them. Imagine you're parsing XML and
> tossing it into some graph library. Both could easily define "Node"
> and neither is wrong for doing so. But if your module tries to use
> them both, it'll need to disambiguate.

I was working on the assumption that the vast majority of all-imports
would be from internal utility modules.

> >  import myprog.utils <- * - dontWant
>
> Is that excluding "dontWant" or importing both "-" and "dontWant"?
> Operators are valid identifiers too! :)

I was assuming you'd have some surrounding of operator names like
Haskell or OCaml's (-). Actually, you should probably give syntax
identifier names too, so that you can choose not to support some
provided syntax. (Yes, I am arguing for first-class syntactic
extensions. Why are you looking at me funny?)

But anyway, the "* -" syntax is ugly.

[Bob Nystrom]

(Sorry for the delay, I'm in the middle of moving right now...)

On Fri, Jul 1, 2011 at 1:30 AM, Elliott Hird
<penguino...@googlemail.com> wrote:
> I admit that I am a fan of judiciously-used punctuation; not to the
> levels of J (ironically, a language I love), but a lot of code in e.g.
> Lua tends to be unreadable for me, drowned under a sea of cluttering
> and alike-looking keywords.

I'm OK with languages that are more punctuation heavy (though I find
Haskell a bit more opaque than I'd like) but for better or worse, that
isn't the direction Magpie has taken. I agree that a sea of words can
look overly homogenous, but I find syntax highlighting helps
dramatically there.

> Foo, bar, and baz /come from/ some.module. Think of a module as a
> present; we're saying that foo, bar, and baz are packed into this
> present.

That's true, but its the module itself that defines that, and not the
import expression, right?

> The other way around suggests that some.module is packed into
> foo, bar and baz.

I would read "import some.module -> foo, bar, baz" as "take stuff from
some.module and put them into 'foo', 'bar, and 'baz' here" which is
what it does.

> import some.module <-
>  foo
>  bar
>  baz
> end

Not bad. Now replace "<-" with "with" since "with" is already a
reserved word and is used to start block elsewhere. :)

> Well, certainly. But every language has a learning curve

True, and in Magpie's case multimethods are already a pretty steep
bump coming from other languages.

> -- recall Raskin, "intuitive equals familiar".

Correct. Part of the reason I lean towards keywords is because readers
will at least be familiar with the *English* meaning of the word,
which gives them a little guidance. Most punctuation doesn't give any
clue at all.

> In this case, I think you'll be
> typing and reading import statements often enough that a syntax that
> gets out of the way and lets you see what is actually going on quickly
> as an experienced user is worth the ten minute learning curve.

We won't know for certain until we've built up a decent corpus of
Magpie code, but my belief is that qualified imports will actually be
rare. I'm operating under the assumption that you'll use simple
"import foo" imports 90% of the time.

>> Correct, I was unclear there. If you do a specific named import, it
>> should successfully import *something* with that name. the "if it
>> exists" is because a named import can refer to one or more of the
>> following: a variable, a multimethod, a prefix keyword, or an infix
>> keyword. When you import by name, you can't actually specify which of
>> those you want: you will import any of those that exist with that
>> name.
>
> Right. (Perhaps you should be able to? That could have weird semantics-
> changing bugs.)

That's a possibility, but I'd like to avoid that complexity until we
know we need it. My rule is: When in Doubt, Leave it Out! :)

- bob

[Mike Hershberg]

For what its worth I agree with Bob on this one. Magpie already leans in favor of keywords and the <- syntax just seems out of place. I'd rather err on the side of consistency than try to optimize typing early by introducing syntax shortcuts that don't mesh with the rest of the language design.

-Mike-

[Bob Nystrom]

On Fri, Jul 1, 2011 at 1:39 AM, Elliott Hird
<penguino...@googlemail.com> wrote:
>> 3. I want to import everything that this module exports, and re-export
>> a couple of its methods from my module.
>
> Yeah, I'm pretty firmly of the view that what a module exports should
> not be tied into its import statements.

A module's exports aren't tied to its import expressions. Any
top-level variable or method with a public name is exported. The only
thing tied to imports is which *imported* names are *re-exported*.

> This irks me as a fan of namespace separation and makes me recall the
> horror of namespaceless C

Keep in mind that Magpie has no global scope. Each module gets its own
top-level scope and only has access to what it imports. At
import-time, each module has control over the names it brings in, so
there's no reason to speculatively make your names longer and "less
ambiguous" just in case some later module may want to import it: it's
each modules responsibility to control the set of names it uses and
disambiguate them. C is a horror because it doesn't support renaming.

>, but then I suppose if the convention is to
> define namespaces inside the modules anyway (a la Ruby), that would be
> fine.

That's definitely not the convention. :)

The convention is:

1. Use names that are short, clear, and unambiguous (to a human).
2. Try to export more methods than types.

I believe that that (combined with multimethods) will eliminate the
majority of name collisions. I may be wrong, but I'd rather err on the
side of terseness and have to move towards longer names then err on
the side of boilerplate and later find out it wasn't necessary but is
too entrenched to ever remove.

> Oh, well, IM-very-strong-O, modules as first-class concepts are vital!

There very likely will be a concept of a first-class module, most
likely through the reflection system. But that doesn't mean modules
will just be bags of properties like they are in, say, Python.

> Otherwise, you will just end up reinventing first-class modules with
> e.g. classes.

What problems would that solve?

> (I am a great fan of the ML module system and I think
> you might be too

Everything I've heard about it sounds cool, but, alas, I haven't had
time to learn it as well as I should yet.

>> 1. Ensure that the foo.bar module has been loaded and evaluated.
>> 2. Define a new variable in the current module named "blah".
>> 3. Assign it the value of the variable of the same name in foo.bar's
>> top-level scope.
>
> Right. First-class modules would likely change this story a lot.

To what?

> See above :) BTW, with an ML module system, modules have signatures --
> modules : signatures :: values : types. This means that you could have
> all the powerful concepts that this (IMO best-breed) first-class
> module system offers while keeping the kind of "optional static
> typing" that Magpie has even at the module layer.

Well, it's worth noting that Magpie doesn't have static types at the
moment. I removed those (temporarily, I hope!) when I added
multimethods. When/if I get an optional type system working again, it
will likely be purely at the library level like Typed Racket.

> I would have it part of some "module header", but then I would also
> have regular exports this way. IMO every library should list
> everything it exports to avoid accidental exports.

That's appealing in principle, but painful in practice:

export Address
export street
export city
export state
export zip

defclass Address
street is String
city is String
state is String
zip is Int
end

I'm trying to make the language lightweight to use and a big part of
that is leveraging convention over explicit repetition. So instead of
having to list everything twice, there's a simple rule: if it has a
public name, it's exported. Otherwise, I fear you'd end up with 50
lines of import/header for every 100 line module.

> That's just because abs is a vital name in any namespace. Would you
> say the same for SDL.init() or GL.flush() or GTK.blahblahblah()

Well, a couple of points:

1. Those are all perfectly valid methods in Magpie, so those examples
would be fine as-is. (Though "init" is strange because that's the name
of the method used to initialize new instances. Seeing it used outside
of that context looks a bit like having a function named "constructor"
in C++.)

2. None of those take arguments, which means they're probably acting
on global state. I don't really dig global state, so I'd be happier in
many cases if it was flush(glBuffer) instead of just some
argument-less method. Once you add an argument, now multimethods will
help you avoid name collisions.

3. If you have a module that uses OpenGL and makes hundreds of calls,
do you really want to have to explicitly prefix each one with "GL."?
Why would it be the OpenGL module making that decision, and not the
module *using* it?

> I was working on the assumption that the vast majority of all-imports
> would be from internal utility modules.

We have different assumptions there. :)

> I was assuming you'd have some surrounding of operator names like
> Haskell or OCaml's (-).

Outside of lexing, Magpie doesn't distinguish between operator names
and other names. They're just names.

> Actually, you should probably give syntax identifier names too

They do have names. Syntax extensions are each associated with a
keyword that can be used to identify it. (Prefix and infix syntax
extensions are in their own namespaces.)

> so that you can choose not to support some
> provided syntax. (Yes, I am arguing for first-class syntactic
> extensions. Why are you looking at me funny?)

Like most things, you should eventually be able to poke at them
through reflection, but I wouldn't expect them to be as "naturally"
first-class as something like a class. Why would you want them to be?

- bob

[Mike Austin]

I'm sorry, I didn't read the entire thread.. just want to metnion that the language Dylan has one of the nicest module systems, something along the lines of:

import math { abs => absolute, sin => mysine }

I'm not sure if you like the curlies, but it would need some separation.  I like => better than -> because the = is usually aligned vertically better (Even here, the - is not in the center of >).

http://www.mactech.com/articles/develop/issue_21/21strassman.html

Mike

[Bob Nystrom]

On Thu, Oct 27, 2011 at 5:39 PM, Mike Austin <mike.aus...@gmail.com> wrote:
> I'm sorry, I didn't read the entire thread.. just want to metnion that the
> language Dylan has one of the nicest module systems, something along the
> lines of:
>
> import math { abs => absolute, sin => mysine }

Why does that not surprise me? Everything I read about Dylan seems to
be super nice.

>
> I'm not sure if you like the curlies, but it would need some separation.  I
> like => better than -> because the = is usually aligned vertically better
> (Even here, the - is not in the center of >).

Especially with renames, I really like a keyword like "as". I tried
using "=" before like:

import foo with bar = bang

and it's never obvious which side is the source name and which side is
the destination name. "import foo with bar as bang" is pretty clear,
though, I think.

>
> http://www.mactech.com/articles/develop/issue_21/21strassman.html

Thanks for this. Info on Dylan is hard to come by. :)

- bob

[Guillaume Laforge]

I'm quite used to the syntax using "as"... as in Groovy, it's pretty explicit:

import java.lang.Math as M

import static java.lang.Math.cos as cosine

The arrow is quite nice too though.

Guillaume

--
Guillaume Laforge
Groovy Project Manager
SpringSource, a division of VMware

Blog: http://glaforge.appspot.com/

Twitter: @glaforge

Google+: http://gplus.to/glaforge

[Mike Austin]

Does Groovy allow you to rename multiple bindings on the same line?  Something like:

import java.lang.Math { sin as sine, cos as cosine }

Mike

[Guillaume Laforge]

No, not on the same line, you have to use one per line.

Personally, I'm not too keen on putting everything on the same line, as that's a bit too much of information for one single line.

Guillaume

[Mike Austin]

Usually I am a fan of liberal vertical space, but it seems it violates the Don't Repeat Yourself rule:

import static java.lang.Math.cos as cosine

import static java.lang.Math.sin as sine

The "all on same line" could also be spread across multiple lines:

import static java.lang.Math {
    sine as sine, cos as cosine
}

This is basically what Bob had originally, which I think works fine:

import foo.bar with
    exclude blah
    bang as bong
end

Mike

[Guillaume Laforge]

True, DRY is important, and the proposed syntax originally suggested works fine too.

I'd especially reformat each alias on its own line:

import static java.lang.Math {

    sin as sine,

    cos as cosine

}

The little things that are a bit more painful IMHO with this alternative (rather than a dedicated line per import / alias) is that when you want to update your imports and aliases, you might have more work, for example, you might forget a comma at the end when copy / pasting a new line / alias, you might have to remove a whole {} block to remove aliases and that is harder than just removing a line and removing potentially bits from the end, etc.

From an IDE / editor perspective, there's a bit more work to do to update your aliases manually.

Guillaume

[Mike Austin]

If the comma was only used to separate bindings on the same line (call it a style), you could write:

import static java.lang.Math {

    sin as sine

    cos as cosine
    pow as power

}

This takes more vertical space, but is easier on my eyes than:

import static java.lang.Math.cos as cosine

import static java.lang.Math.sin as sine

import static java.lang.Math.pow as power

If you just wanted to rename one binding, the latter would work just as expected.

Mike

[Guillaume Laforge]

Yup, with comma or semicolon for seperation for same line imports / aliases, that sounds better to me.