OTP-9649 and further changes
Yurii Rashkovskii
Yurii Rashkovskii
Which is fine.
Thanks,
Yurii.
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Gleb Peregud
> Hi,
>
> I was wondering if anybody (especially from the OTP team) can shed
> some light on the following subject.
>
> In R15 we've got this:
>
> OTP-9649 Tuple funs (a two-element tuple with a module name and a
> function) are now officially deprecated and will be removed
> in R16. Use 'fun M:F/A' instead. To make you aware that your
> system uses tuple funs, the very first time a tuple fun is
> applied, a warning will be sent to the error logger.
>
> Which is fine.
>
> But I was wondering if there's any word out about the fate of tuple
> modules? The ones like {erlang}:element(1). Are they expected to be
> kept around? (I certainly hope they are :)
I believe you mean parametrized modules, since they are basically a
tuples in runtime, where first element is module name. Those are two
different concepts.
In general I think that parametrized modules are here to stay, since
so many projects are using it, including OTP code itself (mnesia).
Although it would be nice if they are actually moved into "official"
part of language.
Best,
Gleb
Yurii Rashkovskii
> I believe you mean parametrized modules, since they are basically a
> tuples in runtime, where first element is module name. Those are two
> different concepts.
Well, this is a "hack" used to enable parametrized modules. I,
however, use it outside of parametrized modules (just make functions
accept the value tagged with the module name as the last argument).
This is why I prefer calling them tuple modules so that I don't run
into anti-parametrized modules crowd when it is uncalled for (the
general consensus on IRC is that as long as your module is not
parametrized but explicit, you're fine).
>
> In general I think that parametrized modules are here to stay, since
> so many projects are using it, including OTP code itself (mnesia).
> Although it would be nice if they are actually moved into "official"
> part of language.
Exactly. That's why I would love to hear from the OTP team.
Yurii.
Gleb Peregud
> Well, this is a "hack" used to enable parametrized modules. I,
> however, use it outside of parametrized modules (just make functions
> accept the value tagged with the module name as the last argument).
Why do you need wrapping module name in a tuple? It works without it:
Eshell V5.9 (abort with ^G)
1> erlang:length([1,2,3]).
3
2> M = erlang.
erlang
3> M:length([1,2,3]).
3
Is there any case when wrapping is necessary, which I'm missing?
Best,
Gleb
Yurii Rashkovskii
>
> Eshell V5.9 (abort with ^G)
> 1> erlang:length([1,2,3]).
> 3
> 2> M = erlang.
> erlang
> 3> M:length([1,2,3]).
> 3
>
> Is there any case when wrapping is necessary, which I'm missing?
That was just a minimal (and useless) case of how tuple modules
behave. This is a far bigger deal when you have records (or just
tagged tuples) and modules named according to them. One of the real
use cases is misultin_req:
https://github.com/ostinelli/misultin/blob/master/src/misultin_req.erl
Yurii.
Barco You
Jon Watte
--
Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable.
Gleb Peregud
> Will "fun M:F/A" survive reloads and old code purge, and behave like the
> tuple function?
AFAIK, yes
Jon Watte
On Wed, Jan 18, 2012 at 18:40, Jon Watte <jwa...@gmail.com> wrote:AFAIK, yes
> Will "fun M:F/A" survive reloads and old code purge, and behave like the
> tuple function?
Valentin Micic
So, let me ask explicitly: is the following syntax going to be valid going forward (e.g. R16 onwards):
>
> Eshell V5.9 (abort with ^G)
> 1> erlang:length([1,2,3]).
> 3
> 2> M = erlang.
> erlang
> 3> M:length([1,2,3]).
> 3
...or would we be required to replace the above with:
67> F=fun erlang:length/1.
#Fun<erlang.length.1>
68> F([1,2,3]).
3
Kind regards,
V/
Gleb Peregud
Afaik this syntax is here to stay.
The syntax which lifetime is undefined yet is syntax mentioned by Yuri at the beginning of the conversation
Björn Gustavsson
> I have a separate question about this.
>
> {M, F} references survive code reload and old code purge. That is, you can
> hold on to these references "forever."
> fun() -> M:F() end does not -- once the code that defined the fun is
> purged, any reference will now generate an exception when called.
>
> Will "fun M:F/A" survive reloads and old code purge, and behave like the
> tuple function?
>
Yes. This behavior is documented at the end of section 7.17 in
the reference manual:
http://www.erlang.org/doc/reference_manual/expressions.html#id77989
/Björn
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
Björn Gustavsson
> Really sorry for rewinding this conversation back a bit, but I am getting a
> bit confused...
> So, let me ask explicitly: is the following syntax going to be valid going
> forward (e.g. R16 onwards):
>
>>
>> Eshell V5.9 (abort with ^G)
>> 1> erlang:length([1,2,3]).
>> 3
>> 2> M = erlang.
>> erlang
>> 3> M:length([1,2,3]).
>> 3
Yes. This will continue to work regardless of what
happens to parameterized modules. (This syntax
is much older than parameterized modules.)
/Björn
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
Björn Gustavsson
> Hi,
>
> I was wondering if anybody (especially from the OTP team) can shed
> some light on the following subject.
>
> In R15 we've got this:
>
> OTP-9649 Tuple funs (a two-element tuple with a module name and a
> function) are now officially deprecated and will be removed
> in R16. Use 'fun M:F/A' instead. To make you aware that your
> system uses tuple funs, the very first time a tuple fun is
> applied, a warning will be sent to the error logger.
>
> Which is fine.
>
> But I was wondering if there's any word out about the fate of tuple
> modules? The ones like {erlang}:element(1). Are they expected to be
> kept around? (I certainly hope they are :)
>
No, we don't expect to keep them.
As pointed out by others, "tuple modules" is just an implementation
detail of parameterized modules (which is an experimental feature).
We have not reached a decision yet, but in the future we expect
we will do one of two things:
* Remove parameterized modules completely from the compiler
and run-time system.
OR
* Implement parameterized modules properly by creating a new
data type, both to allow better type checking by Dialyzer, and
to possibly improve performance.
/Björn
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
Yurii Rashkovskii
Understandable. The only problem that I see with this decision is that
a considerable number of projects use this feature, even without using
parametrized modules directly (off the top of my head I can remember a
few but I am pretty sure there's much more).
And yes, I do understand that this feature was experimental. But
there's also reality of its use.
Yurii.
2012/1/23 Björn Gustavsson <bgust...@gmail.com>:
Yurii Rashkovskii
Is there any way tuple module calls can be separated from parametrized
modules and subsequently documented and become part of standard
Erlang?
Yurii
Björn Gustavsson
> Hi Björn,
>
> Understandable. The only problem that I see with this decision is that
> a considerable number of projects use this feature, even without using
> parametrized modules directly (off the top of my head I can remember a
> few but I am pretty sure there's much more).
>
To inform our decision, it would be very interesting to know:
1) Why are projects using "tuple modules"? Is is because
the syntax is easy to use or is because of performance or
some other reason?
2) Could the projects use parameterized modules instead?
3) Which are the projects?
Yurii Rashkovskii
>
> 1) Why are projects using "tuple modules"? Is is because
> the syntax is easy to use or is because of performance or
> some other reason?
I can't speak for everybody, but I can speak of our own use. In
different cases it is obviously a slightly different reason. In our
latest case, it is not an "ease of use" of the syntax, but a whole way
to establish a certain type of operations. This project (not yet
released as open source, but eventually will be) is a relational
database mapper akin to ActiveRecord.
It allows us to map records describing the structute of tables to be
mapped to modules that implement their functionality.
So, suppose we have db_User record:
-record(db_User, { id, email, password }).
and a respective db_User module:
-module(db_User).
-extends(db_model).
-include("models.hrl")
As you can see, this module exports nothing. However, because of
-extends attribute, it will redirect all those missed calls to
db_model module. With tuple calls, the magic happens around there.
When, for example, someone executes User:save(), it gets ultimately
converted to db_model:save(#db_User{}) which can take care of saving
the record. Moreover, db_model:save/1 attempts to call before_save /
after_save callbacks if they are defined in db_User. The way it does
it is fairly simple: it just calls User:callback_name() and if it is
not defined, that call again goes to the default implementation in
db_model.
Of course it is technically possible to implement all of this without
tuple calls (at an expense of worse syntax), but obviously I'd rather
keep it as is as it works fantastically.
>
> 2) Could the projects use parameterized modules instead?
Some could, I suppose. But a lot of developers (including myself) are
avoiding them because of their many problems, like obscuring functions
arities. Tuple calls don't do that, your module source has functions
the way they eventually will be available in a compiled form.
I suppose you're talking about refined parametrized modules that don't
use tuples and use their own type. In this case the above project will
suffer a damage as it is an important feature of the project to keep
database records as records so that they can be easily matched. Unless
this new type provides a clear alternative to the above, it'll force
us into rewriting the whole thing once tuple calls are removed.
>
> 3) Which are the projects?
I just started collecting the list at
https://github.com/yrashk/9649-consequences — obviously this list is
far from completion and doesn't account for non-opensource projects.
Yurii.
Björn Gustavsson
> Björn,
>
> Is there any way tuple module calls can be separated from parametrized
> modules and subsequently documented and become part of standard
> Erlang?
>
Perhaps.
The problem with the current implementation of "tuple modules"
is that the M in apply(M, F, A) and M:F(...) has a strange type, i.e. M can
be either an atom or a tuple. Unless we implement parameterized modules,
we want M to be only an atom. (There is a run-time cost in the
implementation of apply/3 to allow parameterized modules/"tuple modules",
and parameterized modules prevented an optimization I wanted to do in the
compiler for R15B.)
We might consider an alternative implementation of "tuple modules"
that does not use apply(M, F, A) or M:F(...), using either new BIFs or new
syntax. (Example: tuple_module_apply(M, F, A) could behave as apply/3
currently behaves, or alternatively only allow M to be a tuple. This is not
a serious suggestion, just an example to make it clear what I mean.)
I suggest that you write an EEP.
José Valim
Richard Carlsson
> Since the discussion steered towards "tuple modules", I am going to use
> the opportunity to ask.
>
> The fact that part of the standard library, like the dict module,
> receives the dict as last argument and the dict itself is a "tuple
> module" is an accident or a design decision? Does anyone actually write
> "Dict:find(Key)" instead of "dict:find(key, Dict)"?
This is just a coincidence. First, the dict module is much older than
the experimental implementation of parameterized modules. Second, the
data representation used by dict is explicitly documented to be
considered opaque and subject to possible changes. Nowhere is it said
that a dict is or will remain a callable object.
The reason it happens to work is that 1) the dict module uses a record
also called 'dict' for its hash table representation, and 2) when
performing a call to a parameterized module, the module object itself is
passed as an additional parameter, whose position (last) happens to
coincide with the expected position of the input dict in the API calls.
If the calling convention for parameterized modules would pass the
module object as the first argument (which it could do in a future
version of the compiler), or the dict module changed the record name
used for its tables, this would cease to work.
/Richard
José Valim
This is just a coincidence. First, the dict module is much older than the experimental implementation of parameterized modules. Second, the data representation used by dict is explicitly documented to be considered opaque and subject to possible changes. Nowhere is it said that a dict is or will remain a callable object.
Yurii Rashkovskii
I wanted to follow up on this. I am investigating this whiole issue
(as I really would love to keep tuple calls) and I would like to know
whether you have any numbers on how significant the speed impediment
of this feature is (the run-time cost incurred in apply/3) that you
mentioned earlier? Do you have any comparisons?
Yurii
Yurii Rashkovskii
I actually just did a totally unscientific test. I took the maint
branch, and ran 10,000,000 Mod:fn() calls where Mod was passed from
another function call and was just an atom. I ran it on maint, and on
my Mac Pro timer:tc clocked 446983 (that's 0.446 seconds).
After that, I brutally removed tuple call support from apply and
fixed_apply BIFs in beam_emu.c, making `if (is_not_atom(module))` go
straight to error label. I also removed `this` variable which was used
in those checks because there was no need for it anymore.
I re-ran the test, and timer:tc clocked 440680 and around.
So, the runtime cost difference was about 6000 microseconds, or 0.006
seconds, for 10 million calls. If I am not mistaking, this is a 1.35%
speedup on something as low cost as about 0.0446 microseconds per call
(or 0.0440 microseconds per call without support for tuple calls).
Is it this big of a difference to support the removal? What if we made
tuple calls a documented feature? I'd be glad to write an EEP to
support it.
You can find my test patch attached.
P.S. I also ran the same test for a tuple call (in the maint branch),
timer:tc clocked 545656 and around. So there is a penalty for using
tuple calls, but that's expectable and seems to be a reasonable
tradeoff for me.
P.P.S. Please let me know if something in this test is incorrect, I'd
be glad to improve it.
Yurii Rashkovskii
cycle on both versions of the runtime. It was about 70,000
microseconds on both. This means that while the speedup remains the
same, the cost of actual operations has to be dropped even lower, to
about 0.0370 microseconds per call (on that hardware)