Could (or should) we add sugar for submodule paths?

[Alexis King]

Exposing submodules is becoming more prevalent in Racket, and I think that may
be a good thing. The downside is that the syntax for requiring a submodule
outside of its defining module is a bit clunky (the use of `submod` is
necessary). The same would be true for `lib`, but Racket already has sugar for
`lib` in the form of symbolic module paths, which is obviously the most common
way to specify modules.

Since the grammar for these paths already constrains the characters they may
contain, it would be completely feasible to add a little more sugar to make
requiring submodules easier. To make this more concrete, something like
(require foo/bar:baz) could be equivalent to (require (submod foo/bar baz)).

I, myself, have two main thoughts about this, having already considered it
somewhat.

1. I think this would elevate submodules to a more commonly-used feature.
Right now, aside from main and test submodules, they rarely pop up, mostly
because there’s little need most of the time, and using them is verbose.
However, since modules themselves are intrinsically tied to the filesystem,
sometimes it’s nice to not need to create a full module.

2. There is a downside to this (aside from the obvious potential downside of
cluttering the module path with more syntax): it makes it much easier to
expose submodules as part of a public API, but still forces the consumer
of a module to be aware of the difference between a module and a submodule.

This means that a user needs to remember if a module path is foo/bar or
foo:bar, depending on whether or not the author used a submodule, and it
also means there is not a seamless path for moving a submodule to a full
module or vice-versa.

The above caveat is real, but I still think it could be advantageous all the
same. Thoughts? Questions? Concerns?

Alexis

[Greg Hendershott]

> Exposing submodules is becoming more prevalent in Racket,

That's interesting, I hadn't noticed this trend yet, what are some examples?


I feel like the downside you mentioned is significant. I guess instead
I wish it were possible for this to be totally up to the implementor
-- and not require (pun) the user to know about the "physical"
(sub)module location or to use special syntax.


Hmm, what if `(require foo/bar)` should mean any of (in order):

foo/bar.{rkt,ss}
foo/bar/main.{rkt,ss}
(submod foo bar) ;new

Admittedly adding the last meaning could cause collisions in existing
code: "test" is a likely collision, as it's likely to be both a
submodule name and a subdirectory name. However, it's probably
unlikely for the user of a collection to (require foo/test)? And the
collection itself would be using rel-strings with submod? And a tool
like raco test would also do the explicit (require (submod "foo.rkt"
test))? So maybe it's OK??

[Anthony Carrico]

I wonder how much trouble it would cause if there was a unified syntax
which considered a directory as a (standardized) module.

In:

foo/bar/bah

"bar" might be a directory "pseudo module" or a real module. I guess it
would be possible to define two submodules with the same name (once in a
file, and once in the filesystem). A rule would be needed to make one of
these primary, and the other shadowed.

When I first started using submodules, I intuitively assumed this
unified syntax, and had to refer to the manual to figure out the real one.

--
Anthony Carrico

[Matthias Felleisen]

An alternative would be to have named provide specifications. That way
we could tell one client to use API R and another client to use API TR to
access functions from our module. If we do this right, we can connect submodules
to such provides and nobody will ever know that they are requiring a submodule.

-- Matthias

> --
> You received this message because you are subscribed to the Google Groups "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/5D260BC4-FA07-4668-919D-92A6B3DC9DB4%40gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Jonathan Schuster]

On Tue, Jul 7, 2015 at 12:24 PM, Anthony Carrico <acar...@memebeam.org> wrote:

I wonder how much trouble it would cause if there was a unified syntax
which considered a directory as a (standardized) module.

In:

foo/bar/bah

"bar" might be a directory "pseudo module" or a real module. I guess it
would be possible to define two submodules with the same name (once in a
file, and once in the filesystem). A rule would be needed to make one of
these primary, and the other shadowed.

For what it's worth, Rust does something like this for its module system. See the "Multiple file crates" section here: https://doc.rust-lang.org/stable/book/crates-and-modules.html

[Jay McCarthy]

I've wanted to do this AND Anthony's idea of having
submodules/directories/files/etc all appear in a unified hierarchy

Jay

> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/2AD9E5E4-DB2C-4D9E-BE69-69C07A43BE4C%40ccs.neu.edu.

> For more options, visit https://groups.google.com/d/optout.

--
Jay McCarthy
http://jeapostrophe.github.io

"Wherefore, be not weary in well-doing,
for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
- D&C 64:33

[Eli Barzilay]

On Tue, Jul 7, 2015 at 12:24 PM, Anthony Carrico <acar...@memebeam.org> wrote:

> I wonder how much trouble it would cause if there was a unified syntax
> which considered a directory as a (standardized) module.
>
> In:
>
> foo/bar/bah
>
> "bar" might be a directory "pseudo module" or a real module. I guess it
> would be possible to define two submodules with the same name (once in a
> file, and once in the filesystem). A rule would be needed to make one of
> these primary, and the other shadowed.

FWIW, there was a variant of this, ages ago, in the code that generated
the web pages. (This was in the svn days, before it moved out of the
internal repository.) The idea was that a "foo/bar/baz.html" page could
be generated by a "foo/bar/baz.ss" file, or as a definition of the "baz"
page in "foo/bar/index.ss" or in "foo/bar.ss" , or as a "baz"
sub-definition in a "bar" group definition in a "foo/index.ss" or a
"foo.ss". The HTML production part is misleading: this was very similar
to a sub-module thing, since you could "import" values defined in these
pages -- where the path that is specified is relative to whatever file
is currently being rendered. So some `(import-from "../blah" whatever)`
could get a binding from a different part of the same file, or a
different file following the same search rules.

It was the extremely flexible with these search paths, you could easily
move code from a monolithic file to smaller ones to whatever degree you
want.

And it ended up as an unimaginable maintenance nightmare, constantly
using grep to find code to modify, and the theoretical ability to move
code was never used since it was very easy to break other code with such
moves.

--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!

[Jack Firth]

I would find something like this very useful, but similar to what Eli mentioned it would strongly encourage exposing subcollections and submodules which leads to inflexibility. Additionally, having directories, files, and submodules shadow each other in some defined order could lead to tricky to track down bugs. It might be better for an ambiguous require to be an error, or at least a warning.

[Matthew Flatt]

At Sun, 5 Jul 2015 22:27:32 -0700, Alexis King wrote:
> The downside is that the syntax for requiring a submodule
> outside of its defining module is a bit clunky (the use of `submod` is

> necessary). [...]

>
> Since the grammar for these paths already constrains the characters they may
> contain, it would be completely feasible to add a little more sugar to make
> requiring submodules easier. To make this more concrete, something like
> (require foo/bar:baz) could be equivalent to (require (submod foo/bar baz)).

Yes, that seems feasible to me.

It would be nice if a shorthand could also cover relative-path
references, too, but I don't see a way to do that (at least, not
without creating further confusion between relative-path references and
filesystem-path references).

At Tue, 7 Jul 2015 12:24:49 -0400, Anthony Carrico wrote:
> I wonder how much trouble it would cause if there was a unified syntax
> which considered a directory as a (standardized) module.

That would create lots of trouble, unfortunately.

Search paths are painful in general. It may seem that a search
shouldn't be that bad --- implemented once and for all in the
module-name resolver, say. It turns, though, out that various tools
need to implement the same search with slightly different constraints,
such as the module searches in `raco make` or `raco exe` or conflict
detection in `raco pkg`. Dealing with various search dimensions is one
of the least-fun parts of building those tools, so I'm especially
reluctant to add more.


In contrast, module-path parsing and normalization is relatively easy
to centralize. If a `foo/bar:baz` notation seems worthwhile to others,
I'd be willing to give it a try.