Alternate design for module declarations

[Evan Czaplicki]

Today I was reviewing elm-format with Aaron. I mainly write packages, so I am particularly sensitive to specifics of the module X exposing (..) format.

At some point, Aaron asked why we even need that line. The module name must match the file name, and the exposed values must appear in the doc comment. It is all redundant! It seems obvious in retrospect, and I am surprised I have never heard this suggestion!

I was pretty intrigued, so we thought it through and end up with the design described here:

Design for dropping module declarations

Timeline: This cannot be changed in 0.19, but it could be part of a batch of work that makes documentation nicer for companies with large closed-source applications. The current tools are designed for packages only, so writing professional docs internally does not have great support. So there is no immediate plan here, but it could be part of a release that emphasizes tooling.

Goals: I just wanted to write it down so I can come back to it later. If there is any high quality feedback that may be worthwhile to share.

Note: High quality feedback adds new information. It may look like this:

1. "Coming from experience with A and B languages, this surprises me because X"
2. "Language C does M, which suggests that X can be improved in the idea."
3. "I have often struggled with X in concrete scenario S, and this improves that."

In other words, there is informational content beyond your immediate emotional reaction to the idea.

[N H]

This is similar to how exports are done in ES6, TypeScript etc. You can read about that here: https://developer.mozilla.org/en/docs/web/javascript/reference/statements/export

The main difference is that you can use an ES6 `export` anywhere, rather than just in the head of the document. I don't think having them anywhere you like is a good idea. But what if `exposing X` came before the module documentation in the file instead, as the module line current does? 

--
You received this message because you are subscribed to the Google Groups "elm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/CAF7GuPFFNQoWhgkaFM9XO77CRDdbjgfKvnLYLpkQEgSAU9jfSQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Max Goldstein]

Coming from a lot of languages, I am very surprised that comments could have semantic value, even if only at the level of packages and not code execution. This is the first "negative" bullet point. The closest thing I've seen to this is a JS library (can't remember which one) that parses myFunction.toString() which includes comments to extract non-JS code. It was weird.

Regarding the extra idea, that would be useful for elm-test fuzzers and my (no longer maintained) turtle graphics library. However, I have found that whenever you want to hide and re-export a type, the package is big enough that you have private modules anyway for all the helper functions.

Removing the module declaration makes it much harder syntactically to indicate different styles of exposing, concretely the two existing and one proposed style for union types, and more speculatively something we haven't thought of and will want to add later.

[Francesco Orsenigo]

There is something that might be worth clarifying.

The way I understood things so far is that you expose a type, so that you can document it and the user can reference it, but you do not expose its constructors, so that the user can't see what's inside and can't create new instances.

For example, say that a module MyThing exposes the Model type.
I don't want the user to see the internals of Model but I want the user to create something that contains it:

type alias ParentModel =
  { myThing : MyThing.Model
  ...

  }

Also, I want to document MyThing.Model, to explain the user what it is and how to use it.

How would this work with the "Extra Idea" proposal?

Further, if I keep a type completely hidden, can I still expose functions that use it in its annotation?
Using the example above, can I expose `init : Model` and still not expose `Model` at all?

[Evan Czaplicki]

Max, you can think of "the module documentation comment" as "the exposing syntax", so if we modify the syntax of the proposal a bit, it could be:

module

Any sort of markdown goes here. We can expose things:

@exposing this, that

Then we have to have some syntactic marker for the

end of this region. Perhaps it could be like this:

end

I don't like the particulars of this at all, but my point is that "but it is a comment!" is a syntactic concern. It does not need to be if we have a suitable alternative syntax.

(I don't we should focus on the "extra idea" in this thread. Let's drop it for now and possibly consider it again later in a separate thread.)

--

You received this message because you are subscribed to the Google Groups "elm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/36d882c6-cea8-4f83-b03a-caeee81c918d%40googlegroups.com.

[Richard Feldman]

This proposal seems like a good time to revisit whether "expose everything in this module" should be an option. (I suspect it shouldn't.)

I tried pretending module Foo exposing (..) didn't exist in the SPA example and found it forced me to think more about what I wanted to hide and what I wanted to expose. It's weird that JavaScript (even ES6) doesn't have module Foo exposing (..) and we do, especially considering JS doesn't even have union types. As far as I've heard, people don't even miss that feature enough to request it in the JS community.

[Evan Czaplicki]

Richard, that is interesting! We have considered this question for import declarations quite extensively, but I cannot say I have thought about module exposing as clearly. I think it just implicitly got the same options as imports.

Thinking of it separately, I think it is quite reasonable to require module exports to be explicit. With beginners, they will be in just one module and main will work no matter what. And once you go to two modules, it seems quite healthy to encourage people to be explicit.

Summary: Your idea is quite strong. It can be considered separately from the original post though, so let's not pursue it further in this thread.

On Sat, Aug 5, 2017 at 9:26 PM, Richard Feldman <richard....@gmail.com> wrote:

This proposal seems like a good time to revisit whether "expose everything in this module" should be an option. (I suspect it shouldn't.)

I tried pretending module Foo exposing (..) didn't exist in the SPA example and found it forced me to think more about what I wanted to hide and what I wanted to expose. It's weird that JavaScript (even ES6) doesn't have module Foo exposing (..) and we do, especially considering JS doesn't even have union types. As far as I've heard, people don't even miss that feature enough to request it in the JS community.

--

You received this message because you are subscribed to the Google Groups "elm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/326b2365-4a71-488c-b668-da8531a00c04%40googlegroups.com.

[Peter Damoc]

Elixir and Python expose everything and have syntax for declaring private stuff (defp, underscores).  

When writing modules, I used to expose everything and even when I switched to being explicit in my exporting, I discovered that I list almost everything in the module (maybe a couple of helper functions get left out of the explicit exporting).  
I have frequently encountered compile errors caused by the fact that I forgot to expose a function. 

When writing libraries, the needs might be a little bit different. In this scenario there might be a lot of small, private helpers with just a few public items that make up the API. 

I have also sometimes forgotten to namespace the modules properly. (e.g. accidentally writing module User exposing (User, decoder, encode) instead of module Data.User exposing (User, decoder, encode) )  this might help with that. 

--

You received this message because you are subscribed to the Google Groups "elm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/CAF7GuPFFNQoWhgkaFM9XO77CRDdbjgfKvnLYLpkQEgSAU9jfSQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

There is NO FATE, we are the creators.
blog: http://damoc.ro/

[Tiziano Santoro]

I'd like to offer the point of view of Rust on this, which I have come to appreciate more and more. This means having symbols private by default, and having to manually annotate symbols that are meant to be public (Rust uses the keyword `pub` for that, before the symbol declaration). In Rust, enums (i.e. sum types) are either public or private, and all the constructors of an enum follow the same visibility of the constructor itself; I suppose elm could choose to instead require manually annotating individual constructors in addition to the enum type itself.

see https://doc.rust-lang.org/book/second-edition/ch07-02-controlling-visibility-with-pub.html#making-a-function-public for a general introduction, but I can pull out more interesting examples if desired.

re: using the comment syntax for controlling visibility: I personally think it is a bad idea and also a slippery slope; essentially it seems to me that it is being considered here because you do not wish to make breaking changes to the language (e.g. adding an explicit visibility modifier); adding this complexity to the comments rather than the language seems appealing because it allows existing code to keep kind of functioning, except it does not really (e.g. current private modules with no documentation would still break and require authors to add comments to fix them). So all in all, we may be better making this a first-class language concept, rather than trying to halfway hide it as comments.

[Tim Steenvoorden]

Hi all,

TLDR: instead of removing module headers and using module documentation as an export specification, I’m in favour of a module header specifying exported symbols and a source code structure that can also serve as documentation.

I think people have some excellent remarks on how to declare exports and ways to handle comments. At the core of the discussion is the fact that the module header and the module documentation duplicate information about exported symbols. To solve this, we could remove the module header and use the module documentation as an export list. I agree with Max though, that using comments for such an important feature as exporting symbols from a module can be dangerous. It mixes documentation and the programming language itself in a weird way. I’d like to advocate to restructure the module documentation instead and direct an Elm source file to a playground or notebook, mixing documentation and code in an well defined way.

The idea of *playgrounds* in [Swift](https://developer.apple.com/library/content/documentation/Xcode/Reference/xcode_markup_formatting_ref/index.html#//apple_ref/doc/uid/TP40016497-CH2-SW1), *notebooks* in [Mathematica](http://reference.wolfram.com/language/tutorial/NotebooksAsDocuments.html) or good old *literate programming* in, for example, [Haskell](https://www.haskell.org/onlinereport/literate.html) is mainly about structuring your source file in such a way it is understandable when reading top to bottom. Exactly the same as Evan tells us to structure our documentation of functions and types in the [module documentation](http://package.elm-lang.org/help/documentation-format) of an Elm source file.

Often, I find myself trying to keep the order of types and functions that I specified in the module documentation, in sync with the order of functions in my source file. This also means replicating the section headers I introduced in the module documentation, to clearly separate different groups of symbols. I end up writing a source file like this:

```elm

module Test exposing (f, g, h)

{-|

General info.

# Section 1

Intro to these functions.

@doc f, g

# Section 2

@doc h

-}

-- Section 1 --

{-| Doc for f -}

f : …

{-| Doc for g -}

g : …

-- Section 2 --

{-| Doc for h -}

h : …

```

I’m wondering why we can’t unify the module documentation and the source itself, breaking up the file into sections that group similar things together. We could thus writing something like:

```elm

module Test exposing (f, g, h)

{-|

General info.

-}

{-| # Section 1

Intro to these functions.

-}

{-| Doc for f -}

f : …

{-| Doc for g -}

g : …

{-| # Section 2

-}

{-| Doc for h -}

h : …

```

This way the source file reflects the documentation structure and vice versa. Some pros:

* There is a explicit separation between comments and the programming language, but they are clearly intertwined in the source file.

* Information about which symbols to expose is not duplicated any more. The compiler should only check if every exposed symbol has an associated doc comment.

* Structure of the module documentation and your source file is not duplicated any more. Moving a symbol from one section to another will be mirrored in the generated documentation.

* The concept of a module is still apparent and will help people to signal the importance of using modules to hide type information.

* You can read a module top to bottom as you would do with the rendered documentation, using your editor to quickly jumpt to sections.

* Generated documentation on a website would have the same structure as your source file, but with rendered headings, bullet lists, highlighted code examples, and so on, but without the code.

* Markdown can be used everywhere: module documentation, section documentation, symbol documentation.

One neg:

* If you want to move or rename a file, you’ll still need to change the module name. (See below for some thoughts on this).

So, instead of removing the module header and using the module documentation as an export specification, I’m in favour of a module header specifying exported symbols and a source code structure that can also serve as documentation, akin literate programming. 

At some point in the feature, we could extend this idea and make a playground / notebook based development environment, rendering the inline documentation together with the functions. This is what Apple currently does with its Swift Playgrounds and Mathematica already did for a long time. It would be a wonderful way to teach people Elm with step-by-step instructions!

## Other thoughts

### Removing the module header

This proposal could still be combined with the idea to remove the model header and thus the duplication of module name and file name. Therefore we should introduce a keyword like `export` (the opposite of `import`) or `expose` (resampling `exposing` in an import declaration) and annotate symbols with it as shown below.

```elm

{-|

General info.

-}

{-| # Section 1

Intro to these functions.

-}

{-| Doc for f -}

export

f : …

{-| Doc for g -}

export

g : …

{-| # Section 2

-}

{-| Doc for h -}

export

h : …

```

I think in the end this boils down to taste:

* Do you prefer a module header which tells the reader a module is declared, giving a summary of all exported symbols and overcome the duplication of module name and file name?

* Or do you prefer to start programming right away, assuming the file name as the module name and annotated every exported symbol with an extra keyword?

(Off course you could leave out the keyword and automatically export symbols containing associated documentation, but that would be quite subtile!)

### Local exports

A short note about internal exports (or the “third option”) Evan mentions in the Gist. Combining the above proposal of notebooks, removing the module header, and the idea of internal exports will need export modification keywords for types. These could be something like this:

* `export` ⇒ export only the type name, currently `TypeName` in the export list

* `export public` ⇒ export the type name and all its constructors, currently `TypeName(..)` in the export list

* `export internal` ⇒ export all the constructors internally in the package but only the type name to package users, proposed as `TypeName(..local..)` in Evan’s Gist (maybe use only parens as in `TypeName()`, or is this to subtile?)

This is similar to how Idris uses export modifiers, see http://docs.idris-lang.org/en/latest/tutorial/modules.html#meaning-for-data-types. Though Idris still uses an optional module header containing the keyword `module` and the module name, which doesn’t need to match the file name, see http://docs.idris-lang.org/en/latest/tutorial/modules.html.

### Doc comment symbol

I’m not sure any more about the `|` in  `{-| -}`. I think the origin of it lies in Haskell (Haddock actually), which uses `{-|` to start a comment about a symbol *after* the comment and `{-^` for the symbol *before*. This distinction isn’t used in Elm. Maybe `{--` and `--}` are clearer and less intrusive? Bonus point: you can use multiple dashes to visually separate sections!

```elm

module Test exposing (f, g, h)

{--

General info.

--}

{------------------------------------------

# Section 1

Intro to these functions.

--}

{-- Doc for f --}

f : …

{-- Doc for g --}

g : …

{----------------------------------------

# Section 2

--}

{-- Doc for h --}

h : …

```

Cheers,

Tim

--

You received this message because you are subscribed to the Google Groups "elm-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/c3ebb063-eefe-49da-ab2b-9093b035ebaa%40googlegroups.com.

[Paul Dijou]

I think that your point is valid even for libs Peter. In all the ones I've written, I always expose way more than half the functions (like 90% of them). If there is too much "private" code inside one of the exposed modules (in the elm-package.json), I will generally move it to another file anyway (Internals.elm or LowLevel.elm, you name it). So I also think it would make more sense to tag types and functions we want to hide rather than the one we want to expose.

Will the compiler help me prevent exposing a private method by mistake? Yes, of course. If the function does not have a documentation comment, it will fail to publish / compile.

About "port" modules, you could determine that by checking if the module actually have a "port" function (either Cmd or Sub). You could do the same for effect modules, like tagging the onEffects function with a reserved keyword or something like that.

Next is how to prevent duplicating the exposed function names inside the @docs. One easy way is to remove this @docs entirely. We just need to allow documentation comment with markdown anywhere inside the file so that we can put titles and stuff anywhere. But that would mean that the generated documentation is deeply linked to the code you wrote, you could not reorder the docs. I'm fine with that since, most of the time, my code is written in the same way I would like to document it. Here is a sample: https://gist.github.com/pauldijou/bbe1d0b59674a03a6828b0b6dabaf9bc

Otherwise, I like the idea of removing the module declaration and I would not put any meaning inside the comments.

[Evan Czaplicki]

I don't get why instead of assessing the proposal, we are seeing lots of counter-proposals with totally different designs. In my opinion, folks are saying:

• Can it be like C and Java?
• How about literate programming?

In other words, why isn't Elm like X? The reason it has the current design is that, as someone who writes a lot of packages, I find it very valuable to have all the documentation in one block. I can read through it, mess with the order, and make sure everything flows while looking at a small block of text. In my experience, this is extremely helpful in creating a coherent API that reads nicely in the docs. This is the core design choice on module docs, and I am not planning to change it. The form of this choice is the topic I wanted to discuss.

--

You received this message because you are subscribed to the Google Groups "elm-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/517205c1-f859-4984-abec-530cce56a3d7%40googlegroups.com.

[N H]

The issues that I see with the proposal have already been identified by you - port and effect modules become strange, comments have unexpected powers that is counter-intutive.

Here's my additional concerns:

- Local tooling (testing, benchmarking, debuggers):

Imagine you are writing tests. A standard workflow is to write tests and use `exposing (..)` in order to export everything for testing. It is rare (with elm-test right now) that you will ever want a module comment in your test file. If @exposing worked still with (..), that would be great.

- Possible edge cases:

 What if the same name is exposed multiple times? Is that a compile error? I would think so. How about if there is @exposing(..) and @exposing(someThing). I'd also expect that to be a compile error. 

- Beginner viewpoints

As far as I know, this would make Elm the only popular language to ever use comments for exports. This would be a huge deviation from almost every language out there. For that reason, I would be very interested to get the insight of a newcomer to the language. Right now, on Slack, we get almost no questions about exposing(..) syntax. The questions we do get are all about type constructors. I would guess with this change, we might get more questions. After all, every Elm file right now starts with `module Main exposing (..)`, but it doesn't necessarily have a module comment. But that is just my guess.


Overall, I'm fond of the goals of this change. I'm a little more cautious on putting more power into the docs, but I would have to see how newcomers assess this syntax. 

Hope this is useful.

To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/CAF7GuPECnQgmq5cZ120rTp8gQnQhn6t8UsPfJtM4rjJv1_ERAg%40mail.gmail.com.

[Robin Heggelund Hansen]

I really like the idea of reducing duplication by removing "module x exposes y", but I have to agree with what others have said that it feels weird mixing comments and code like this.

BUT(!) that doesn't mean it shouldn't be done. This feels wierd because few, if any, other languages does it this way. After a brief transition period, I'm sure people will like this better than what we have now.

The only thing I would like to suggest, is that the keyword for specifying what kind of module it is, is also specified as part of the package comment. Like:

{-|

@port module

This module contains...

@exposes func1, func2...

-}

Feels more natural to me that everything describing a package is part of the package description.

[Evan Czaplicki]

I want to reemphasize this comment from earlier in this thread where I say that this proposal DOES NOT REQUIRE using the doc comment syntax. I gave an example of that in the comment, and I will do another version here:

port module

Any sort of markdown goes here. We can expose things:

@exposing this, that

Then we have to have some syntactic marker for the

end of this region. Perhaps it could be like this:

end

{-| normal docs -}

this = 42

{-| more normal docs -}

that = 3.14159

I am not fond of having an end keyword, but my point is that it does not need to be "a comment" that has real meaning. There can be any sort of variation on that particular syntax, and I was hoping there may be some way to improve it further than the alternates I have described here.

--

You received this message because you are subscribed to the Google Groups "elm-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/cf2f4111-f974-4e0d-b01c-a3614f57b588%40googlegroups.com.

[Robin Heggelund Hansen]

Ahh, true. Sorry, I forgot about that comment.

Hmm. I do like that better. Perhaps instead of an end keyword one could just use import statements as a natural stopping point? In the case of no imports, the keyword could quite literally be "no import" or "import nothing" or even "import ()".

Perhaps having no imports and no description/exposes means the same as "module Main exposing (..)" for quick prototyping?

To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+u...@googlegroups.com.

[Mads Flensted-Urech]

Would it be an option to use indention to delimmit the block ? And could that allow you to remove the leading'@' from exposing?

module

  Any sort of markdown goes here.

exposing this

  Header section for `this`

exposing this

  Header section for `that`

{-| normal docs -}

this = 42

{-| more normal docs -}

that = 3.14159

To unsubscribe from this group and stop receiving emails from it, send an email to elm-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/a4d97d32-2cb5-4bce-baae-ba497800c3e9%40googlegroups.com.

[Jacek Panasiuk]

Idea - different syntax for the third option

• Maybe - don't expose constructors
• Maybe(.) - expose within the package
• Maybe(..) - expose everything

Rationale
Intuitively (.) "exposes less" than (..)
Also makes a bit of sense in command line terms: (.) "exposes things here", (..) "exposes things outside"

[Tim Steenvoorden]

Hi all,

Indentation would be nice to get rid of the `@` indeed. The `@` is a bit strange when compared to the rest of the syntax of the language. However, indentation is always tricky to parse.

Another option would be to just use good old line comment markers, to get an interspersed export list (using a bar-form here`--|`, could obviously be any other (normal) comment marker):

module

--| Any sort of markdown goes here.

--|

--| # Section 1

--| 

--| Header section for `this`

expose this

--| # Section 2

--| 

--| Header section for `that`

expose that, ...

import List

--| normal docs
this : Int
this = 42

--| more normal docs

that : Float

that = 3.14159

It would be something similar to how Haskell (Haddock) does this. They use section markers so you can structure your exported functions. Below an example from the [docs](http://haskell-haddock.readthedocs.io/en/latest/markup.html#documentation-structure-examples):

module Image
  ( -- * Image importers
    --
    -- | There is a "smart" importer, 'readImage', that determines
    -- the image format from the file extension, and several
    -- "dumb" format-specific importers that decode the file at
    -- the specified type.
    readImage
  , readPngImage
  , readGifImage
  , ...
    -- * Image exporters
    -- ...
  ) where

import Image.Types ( Image )

-- | Read an image, guessing the format from the file name.
readImage :: FilePath -> IO Image
readImage = ...

-- | Read a GIF.
readGifImage :: FilePath -> IO Image
readGifImage = ...

-- | Read a PNG.
readPngImage :: FilePath -> IO Image
readPngImage = ...

Thanks for explaining the reason about the current design of the module docs Evan. It is a good point to have small portion of text explaining the module without the need to get through the code and all detailed documentation, as would be the case with a more literate style code file.

I'd like to point out I think all replys in this thread till now take one or more bullets from the “high quality feedback" list to share experiences from other languages. I appreciate it to see these different approaches / struggles from other languages in this discussion!

Cheers,

Tim

To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/CAJngfSn89jUM4XadfpKSXH2nzWTj%2BqyT3M-%3DjrFQfZn8DS-Biw%40mail.gmail.com.

[Alex Barry]

Maybe to tag onto this idea:

module
  |> document "Some doc string"
  |> document "# Section"
  |> exposeAndDocument someFn "something about someFn"

The downside is that it will definitely read oddly, and it probably goes too far in the opposite direction of comments having too much power where now we're using code to express something that comments are better at doing.

To view this discussion on the web visit https://groups.google.com/d/msgid/elm-dev/a4d97d32-2cb5-4bce-baae-ba497800c3e9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gage Peterson]

Although this change would be pretty mechanical (one that could easily be automatically upgraded by Elm format, assuming that people want to use it or know about it) I do feel that it's a large mental shift for small gains. Although the top is technically not "a comment" (it's parsed by the compiler) it's essentially switching syntaxes to an extended markdown _with exports_ from everything up to the "end" which I think is pretty alien coming from other languages. You are essentially paying a learning cost for something that removes duplication In the export syntax but gains an additional documentation syntax. I don't feel that is worth it.

I personally love having a short, concise list of things that are exported especially for modules that don't have documentation (maybe a bad reason).

I think this encourages good modularization that are self contained and have most of the code staying a private implementation detail. I don't mind that the compiler had to check two places and bother me about getting them out of sync. It reminds me to write up good docs at the end.

I actually think the module name at the top of the file is useful to many people that don't use editors that show the file name very prominently, especially beginners. Perhaps that's in them though.

I think the only change I would propose is not having the module name be implicitly "Main" but rather making people always have to match the file name with a more friendly error message. Eg: "Woops! You need to have the module defined at the top, how about...".