Reimplementation of modules names

[José Valim]

A couple of days ago, someone opened a ticket on Elixir issues tracker saying it doesn't work on Windows:

https://github.com/elixir-lang/elixir/issues/178

I have since then verified the bug and I am working to address it.

Unfortunately, we will have to change current semantics. Fortunately, we have found it now and not too late.

This e-mail is to document the issue and the solution I have in mind before implementing it. Feedback is welcome.

## Understanding the root cause of the issue

In order for hot code swapping and code loading to work "thread safe", Erlang is responsible to load all modules byte code.

This is done by Erlang code server.

The Erlang code server has a series of paths it searches to find a module and it maps the module name to the filename.

That said, if you have an Erlang module named lists, you need to have a lists.beam bytecode file.

Since Elixir's goal is to stay as close to Erlang as possible when it comes to semantics, we have decided to follow the same approach.

That said, a module in Elixir named Foo::Bar, is written to ::Foo::Bar.beam bytecode file.

The issue arises because Windows does not accept ":" as a valid filename and therefore cannot compile any file.

We could fix this issue by providing a custom code server, but that would be steering away from Erlang and can introduce issues in new Erlang versions.

Another option is to make Foo::Bar generate a module name "Foo__Bar" (replacing Foo__Bar by underscores).

This would work fine but it would generate an impedance in the code since Foo::Bar no longer returns the atom ::Foo::Bar.

## A viable solution

One possible solution to the problem is to change modules lookup from "Foo::Bar" to "Foo.Bar".

This has one downside: it overloads the meaning of the operator "." but it would work fine in all other aspects.

Since we can address the overloading at compile time, it also shouldn't affect runtime performance.

This approach colides with Erlang packages: http://www.erlang.se/publications/packages.html

As consequence files won't be compiled to a flat structure anymore, but a file hierarchy.

For example, today Foo::Bar is compiled to ebin/::Foo::Bar.beam.

With this proposal, Foo.Bar will be compiled to ebin/__MAIN__/Foo/Bar.beam. __MAIN__ will be Elixir's root namespace.

This change should not affect anything in practice (it doesn't affect how your code is run, just Elixir's compilation internals).

## Summing up

I am proposing to change Elixir's constants from Foo::Bar to Foo.Bar to fix a Windows issue.

Except that Foo::Bar is going to be deprecated, it should not affect any real code.

If you see downsides in this approach or if you can think of better solutions, please make yourself heard. Feedback is welcome.

Thanks,

José Valim

www.plataformatec.com.br

Founder and Lead Developer

[Gustavo Dutra]

As I've already said in the IRC channel, there are pros and cons in both of them.

The "::" syntax for modules/namespaces are present in many languages, like perl and ruby, maybe it turns to be more intutiive. But, in the other hand, I don't see any problem using a "dot" to have "::" behavior. Other languages, like python, already do it. And honestly, code legibility is not affected at all. For instance, 'Foo.Bar.method' is not hard to konw that Foo is a module, Bar is also a module and method is a method, since it is not upper-cased.

So, for the ones who follows good practices and write modules which starts with a capital letter, it will not have a big effect. 'defmodule Foo.bar do ... end' looks fine. I've changed some of my elixir code to see what it'd look like, and I got a even better looking code, IMHO.

In addition, about the atom, I rather go with Foo.Bar than Foo__Bar.

For me, the change for a dot is a viable solution.

[Iaroslav Khrypko]

Hello,

That was me posting this bug.

Another possible solution would be to leave '::' and modify code_server:try_load_module method, so it replaces '::' with '__' (or any other allowed symbols) when tries to load the module.

Pros:

Current structure will be left as it used to be.

Cons:

In order to use Elixir you will have to modify erlang VM files. (which is inevitable sooner or later from my point of view)

Четвер, 22 березня 2012 р., 10:39:35 UTC+2 користувач José Valim написав:

[Alexei Sholik]

What's the true purpose of __MAIN__? I thought Elixir already avoids
conflicts with Erlang modules by using capitalized identifiers for
modules.

So far it has been easy to call Erlang functions in Elixir, you simply
transform module_name:function to :module_name.function. It doesn't
work as easily the other way around, in Erlang code you have to use
'__MAIN__.ModuleName':function.

Also, when defining an application configuration file for OTP, it has
to have a fully qualified name as well, e.g.
__MAIN__.WebsocketServer.app.

Is this really necessary to have such an inconvenient prefix for all
modules? Can't we do without it? If not, we could probably replace it
with something shorter like 'Ex'?

> *
> José Valimwww.plataformatec.com.br
> Founder and Lead Developer
> *

[José Valim]

What's the true purpose of __MAIN__? I thought Elixir already avoids
conflicts with Erlang modules by using capitalized identifiers for
modules.

Unfortunately that's not enough, because some filesystems are case insensitive. 

Also, when defining an application configuration file for OTP, it has
to have a fully qualified name as well, e.g.
__MAIN__.WebsocketServer.app.

Does the file needs to be at ebin/__MAIN__/WebsocketServer.app as well? 

Is this really necessary to have such an inconvenient prefix for all
modules? Can't we do without it? If not, we could probably replace it
with something shorter like 'Ex'?

It doesn't need to be long, but I want a direct mapping from the module name as you type in Elixir and the module name in Erlang. Today, it conceptually works because we can say all modules are defined inside __MAIN__. We could  say that instead that all modules are defined inside Ex, it may be shorter, but having the global module scope named Ex feels weird.

Finally, generating an app file and release files will hopefully be abstract in the future with the help of tools like mix, so this won't be (hopefully) a huge deal. :) In any case, I am certainly open for suggestions.

[Alexei Sholik]

On 5 April 2012 00:36, José Valim <jose....@gmail.com> wrote:

Also, when defining an application configuration file for OTP, it has
to have a fully qualified name as well, e.g.
__MAIN__.WebsocketServer.app.

Does the file needs to be at ebin/__MAIN__/WebsocketServer.app as well? 

I've created __MAIN__.WebsocketServer.app.src in my src directory. Then I build with rebar which compiles it into ebin/__MAIN__.WebsocketServer.app.

Is this really necessary to have such an inconvenient prefix for all
modules? Can't we do without it? If not, we could probably replace it
with something shorter like 'Ex'?

It doesn't need to be long, but I want a direct mapping from the module name as you type in Elixir and the module name in Erlang. Today, it conceptually works because we can say all modules are defined inside __MAIN__. We could  say that instead that all modules are defined inside Ex, it may be shorter, but having the global module scope named Ex feels weird.

Finally, generating an app file and release files will hopefully be abstract in the future with the help of tools like mix, so this won't be (hopefully) a huge deal. :) In any case, I am certainly open for suggestions.

I'm wouldn't be worried about __MAIN__ if I didn't have to type it every time I call a function from Elixir module. But as far as I know, Erlang doesn't have a `use` directive, so we can't automate this. Therefore, a shorter name would make it less painful to use Elixir modules in Erlang.

If Elixir gets momentum, libraries will start appearing that are written in Elixir from scratch. I'm sure it's not your goal to make it difficult to use those libraries in Erlang.

--
Best regards
Alexei Sholik

[Alexei Sholik]

Ah, it turns out I can actually put a module reference into a variable in Erlang or even define a macro. This makes the issue look less critical now :)