[erlang-questions] Request for comment: A proposal to introduce the concept of function domains

[Thomas Järvstrand]

Hi all,

I write this email to get the community's opinion on an idea to extend Erlang with the concept of function domains. The intention is for the end result to be submitted as an EEP.

The rationale is that encapsulation is a good thing and that code should not be allowed to depend on library functions that the library's author did not intend to expose to the outside world. Because of this I would like to introduce the idea of exporting functions into different domains.

There will be three predefined function domains: one will allow the function to be called from anywhere, one will allow the function to be called from within its own application and one to  disallow the function from being explicitly referenced anywhere outside its module (this is for behaviour callbacks, functions passed/returned as funs etc.)

To allow for extension in the future, compilation will allow any atom to be given as the domain name (warnings could be added for non-predefined domains), but xref will only be extended with processing of the predefined domains.

Suggestions for what to call the predefined domains are:

public, restricted, private

The rationale is that due to how the language works, a domain-declaration will only specify where we allow the function to be referenced with its fully qualified name we can't detect other any other references anyway.

external, internal, restricted
The rationale is that functions in the private domain are not really private at all since they can be called from anywhere (eg. the handle_call of a gen_server will be called from the gen_server-module). Private the becomes restricted, because that's what it really is and we use the duality of

external/internal for allowing calls from outside/inside the application. The main issue with this suggestion is that many people associate internal with non-exported functions.

I have two suggestions for how the domains should be declared in a module, they are both attributes that take two arguments: the domain and a list of <function>/<arity>:

The domain/2 attribute:

Pros: Backwards compatibilty

Cons: "Clutters" the attribute namespace. Requires information to be duplicated in the module (need both export and domain)

The export/2 attribute:

Pros: Avoids cluttering the attribute namespace and avoids duplicated information in the code.

Cons: Breaks backwards compatibility with earlier OTP releases for code written using the new attribute.

I would appreciate some input on this, both from the community and the OTP-team.

Regards
Thomas Järvstrand

[Michael Truog]

I think that function domains touches upon a related issue, which could hopefully be combined into one EEP.  The issue is a lack of module namespaces, which currently impacts application dependencies.  Ignoring any potential use by the developer of a namespace concept, and definitely ignoring the past packages implementation that has since been removed from Erlang, there exists an application dependency problem which limits the growth and flexibility of Erlang systems.  The problem is that if application A and application B both depend on C, but on different versions of C, it is difficult to include both A and B without modifying one of them.  The idea of function domains can help provide some access control of a module within the context of a single application, however, access control is a larger problem which impacts how applications can be combined.  To make Erlang more flexible for integrating with various open source dependencies, and various legacy code dependencies, it is often not realistic to build the release with only a single version of a specific application dependency.  I think having the version optionally be added to all of the module names (as a suffix) within an application, which is a problematic dependency, is a way of coping with the issue (it could be based on a flag within the application file of the dependency), but I am sure there are other solutions which may be better.  Either way, both emails deal with the concept of access control for Erlang source code.

- Michael

[Chris King]

On Thu, Apr 25, 2013 at 3:21 AM, Thomas Järvstrand
<tjarv...@gmail.com> wrote:
> I write this email to get the community's opinion on an idea to extend
> Erlang with the concept of function domains. The intention is for the end
> result to be submitted as an EEP.

"Function domain" already has a well-known mathematical definition:
http://en.wikipedia.org/wiki/Domain_of_a_function

I would instead call this "visibility" or "access level" or some such.
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Thomas Järvstrand]

Fair point. How about calling it visibility and using global, local and restricted as the predefined levels?

T

2013/4/25 Chris King <colan...@gmail.com>

[Lukas Larsson]

Hello Thomas,

I agree that there is a need for better encapsulating possibilities in Erlang. However in order to allow calls only within the current application you have to introduce the concept of an application into the language and not only in OTP. Personally I think having language support for a collection of modules (like packages in Java, or namespace in C++) is a necessity for the language to grow and mature. I do however not know what that would look like and what properties that would bring. Have you thought anything about this?

I also do not really understand the purpose of private. How does it differ from not-exported? Is it an exported function which can only be called from a specific behaviour module?

Lukas

[Robert Virding]

There is already an EEP which touches on this, http://www.erlang.org/eeps/eep-0005.html

Robert

---

[Thomas Järvstrand]

@Lukas:

There is actually no need need to introduce the concept of applications into the language itself, merely support for the notion of different types of exports. Interpreting the meaning of these exports can be left to xref, which already has support for applications.

The point of the private type of export is for functions that have to be externally callable, but that should never be explicitly referenced. Examples of this are behaviour callback functions (such as the handle_call etc. of a gen_server) and functions that are passed to or returned from functions calls (such as spawn). Ie, they are functions that may only be implicitly referenced (ie. in the calling code, either the module, the function name or both would have to be a variable). Basically this means that any calls to private functions that xref is able to detect are illegal.

@Robert

Yes, I'm aware of that EEP. It has similar aims and looks cosmetically similar but the suggestion to limit access on a module level would make things awfully messy IMHO. With EEP 5, you have to clutter your code with individual exports for every single caller/callee-pair inside your application. If you want to call a new function in your module, you would now have to edit two files instead of one. A more serious flaw though is that if you're writing a utility library that is to be used by others, you have no idea who will be calling your module, and so you cannot export your functions to them.

Thomas

2013/4/26 Robert Virding <robert....@erlang-solutions.com>

[Thomas Järvstrand]

Correction: private functions may be of course be referenced from inside their own module.

Thomas

2013/4/27 Thomas Järvstrand <tjarv...@gmail.com>

[Richard A. O'Keefe]

On 25/04/2013, at 7:21 PM, Thomas Järvstrand wrote:
> The rationale is that encapsulation is a good thing and that code should not be allowed to depend on library functions that the library's author did not intend to expose to the outside world. Because of this I would like to introduce the idea of exporting functions into different domains.

"Domain" here appears to mean "scope".

There has been a proposal for many years now,
inspired by Eiffel, to have

-export_to(Module, [F1/N1,...,Fk/Nk]).

You'll find it written up as EEP 5
http://www.erlang.org/eeps/eep-0005.html
but it is at least 10 years older than that.

The original proposal came with a variant that said
"these are callback functions for such-and-such a behaviour",
with the intention that it would be checked at compile
and/or load time that such a behaviour existed and wanted
those functions.

The Eiffel approach is organised around what is to be exported:
feature {<set of classes>}
<definitions of features>

The EEP5 approach is organised around the destination of exports:
-export_to(<single destination>, [<set of exports>]).

If EEP5 is extended slightly to allow

-export_to([<set of destinations>], [<set of exports>]).

and to allow [X] to be simplified to X when there is a single X,
then either way of slicing up the export matrix could be used,
whichever the programmer thought clearer.

> There will be three predefined function domains: one will allow the function to be called from anywhere,

I take it this has the same effect as -export does now.

> one will allow the function to be called from within its own application

The trouble is that "application" is an OTP concept, not an Erlang
concept, in that the Erlang compiler otherwise has no knowledge of
"applications".

Now, suppose we take the slightly extended EEP5. Can that do this job?

Well, suppose we set up a convention, that an application foo
has a foo_modules.hrl file containing

-define(FOO_MODULES, [<set of module names>]).

Then you will be able to do

-include('foo_modules.hrl').
-export_to(?FOO_MODULES, [<set of functions>]).

Some good things about this are
* no requirement for a run-time test 'are these two modules part
of the same application', which I am not sure is implementable
in principle
* you can export to *less* than a whole application but more than
a single module
* you can export to more than one application but less than the
entire universe

> and one to disallow the function from being explicitly referenced anywhere outside its module (this is for behaviour callbacks, functions passed/returned as funs etc.)

Ah, this is a tricky one. You're talking about allowing something to be
called elsewhere, but not explicitly. But that's rather a fragile concept.

m:f(X1, X2) % explicit
M = m, M:f(X1, X2) % not explicit
F = f, m:F(X1, X2) % not explicit
M = m, F = f, M:F(X1, X2) % not explicit

But if the compiler front end does constant propagation (and why shouldn't it?)
all of these would look identical to the back end.

Do we really have a problem here that needs solving?

In any case, doesn't this conflict with export_all?

> Suggestions for what to call the predefined domains are:
>
> public, restricted, private
> The rationale is that due to how the language works, a domain-declaration will only specify where we allow the function to be referenced with its fully qualified name we can't detect other any other references anyway.

But a private function under your scheme would in no sense be private;
it is after all a kind of _export_. What's more since you say that
"private" is for "behaviour callbacks ... etc" and since existing
behaviour modules are outside your application, "private" actually
makes a function available to the *same* modules that "public" does.

It seems to me that you are mixing up two very different things here:
- WHICH other modules may call a function
- HOW may they call it.

I think a coherent design needs to separate these issues.