Record Idioms

[Jeremy Huffman]

I noticed the Record syntax sugar can be (ab)used in a way I did not see documented. Consider this record:

defrecord Person, first_name: nil, last_name: nil do
  def full_name(person) do
    "#{person.first_name} #{person.last_name}"
  end
end

p = Person.new first_name: "Jeremy", last_name: "Huffman"

We could access the full_name function in a couple of ways either:

p.full_name

or:

I wouldn't advocate trying to use OO idioms in Elixir since it has consciously moved away from that, but a function such as below seems harmless enough and could be quite handy in places like template code. Would this be discouraged in Elixir?

[José Valim]

I personally discourage it. In general, records should be treated as data.

It may be convenient in some cases but I would use it with care and only in exceptional cases.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Saša Jurić]

May I as why?

I often (ab)use this feature to divide my code in a sort of immutable objects and it makes it easier me to write complex state mutations.

Here's the sketch:

defrecord State, ... do

  def complex_mutation(arg1, arg2, ..., this) do

    this.

      update_field1(...).

      field2(...)

      ...

  end

  ...

  def complex_mutation_n(..., this) do

    ...

  end

end

So from the outside, I can now say:

state.

  complex_mutation_1.

  complex_mutation_2.

  ...

For example, this is the snippet from my production code:

def apply_xml(message_id, xml, this) do

  this.
    apply_xml(xml).
    process_changes(message_id).
    store_model.
    clear_changes

end

I find this code not only easier to follow, but also to write. Sure, some performance penalty is introduced, but I find this to be irrelevant comparing to the actual code the functions are doing.

[Jeremy Huffman]

You can sort of chain it (really just nesting it) without doing this, but it ends up backwards right. So yes probably harder to follow. And you still have your subject last in your method definitions, which is also backwards if they aren't a record accessor. You might have something like:

def apply_xml(message_id, xml, state) do

  alias State, as: S

  S.clear_changes(

    S.store_model(

      S.process_changes message_id,

      S.apply_xml(xml, state)))

end

[José Valim]

You can sort of chain it (really just nesting it) without doing this, but it ends up backwards right. So yes probably harder to follow. And you still have your subject last in your method definitions, which is also backwards if they aren't a record accessor. You might have something like:

def apply_xml(message_id, xml, state) do

  alias State, as: S

  S.clear_changes(

    S.store_model(

      S.process_changes message_id,

      S.apply_xml(xml, state)))

end

 

And with the pipeline operator it becomes more elegant and easier to read:

  xml |>

    S.apply_xml(state) |>

    S.process_changes(message_id) |>

    S.store_model |>

    S.clear_changes

    

[Jeremy Huffman]

Oh nice, I'd missed that!

[Sasa Juric]

This won't work because process _changes takes the state as its last argument.

More importantly, for the same reason the |> doesn't work well with generated modifiers in records (update_).

On Feb 24, 2013, at 5:43 PM, José Valim wrote:

And with the pipeline operator it becomes more elegant and easier to read:

  xml |>

    S.apply_xml(state) |>

    S.process_changes(message_id) |>

    S.store_model |>

    S.clear_changes

    

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/aBzymSRUnFY/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

[José Valim]

This won't work because process _changes takes the state as its last argument.

More importantly, for the same reason the |> doesn't work well with generated modifiers in records (update_).

You would need to reorganize the functions in your module accordingly. You should always pass the record as the first argument for each function. It may not work when interfacing with Erlang but if you are working against Elixir or writing your own modules, it can design it accordingly.

[Sasa Juric]

I can't reorganize the functions which Record module generates, namely the my_record.field(new_value), my_record.update_field(…)

So |> is not always appropriate.

The problem is that |> appends previous expression as the first argument, while records rely on tuple modules, which appends the record as the last argument.

[Jeremy Huffman]

On Sun, Feb 24, 2013 at 1:52 PM, Sasa Juric <sasa....@gmail.com> wrote:

I can't reorganize the functions which Record module generates, namely the my_record.field(new_value), my_record.update_field(…)

Yes. One thing we could do there is make an extra accessor that takes the subject in first position and  hands it off to the updater:

def a_field (MyRecord[] = record, value)

  record.a_afield value

end 

It would be pretty easy I think to make a custom record macro that made these in addition to calling the standard function builders.

[Oren Ben-Kiki]

Actually why does "record.field(new_value)" take the record as the last instead of the first argument? If it were the 1st argument than "." and "|>" would be "more compatible", no? That is, why not just make record.field(new_value) == record |> Record.field(new_value), and similarly for update_field?

[Alex Rønne Petersen]

Completely agree. It baffles me why the 'self' argument is last in the
argument list...

> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[José Valim]

Unfortunately there is nothing we can do here, that's how tagged tuples operate in the Erlang VM.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Alex Rønne Petersen]

What exactly do you mean? Why does the record layout (in terms of
tuples) matter when invoking record functions?

[Oren Ben-Kiki]

It isn't about the record layout; it is that the generated functions take two arguments, the record and the new value of the field (or, the record and the function that computes a new value for the field). The order of these arguments is reversed from the convention Elixir uses everywhere else, specifically the one that makes "|>" useful. There doesn't seem to be a good reason for this, and there does seem to be a good reason to fix this to be compatible with the rest of the system - again, for example to allow saying `foo |> Record.bar(new_value)`.

Coming to think of it, this simple change may actually solve the "fast explicit record access" issue:

Today, one can write either `foo.bar` which may be optimized but in general is a polymorphic call, but also `foo |> Record.bar` which should always expand to the fast pattern-matching record access.

However, today one can't interchange between `foo.bar(new_value)` and `foo |> Record.bar(new_value)`, or between `foo.update_bar(&1 + 1)` and `foo |> Record.update_bar(&1 + 1)`. This is only because of the arbitrary choice of arguments to the generated field functions.

If this choice were reversed, we would be able to just say `foo |> Record.bar(new_value)` and `foo |> Record.update_bar(&1 + 1)`. This would neatly solve the issue of allowing explicit syntax to access the fast pattern-matching record access functions without introducing any new syntax and without having to worry about the `.` being mixed-up between module name space and field access.

Seems like a win-win, no?

[Yurii Rashkovskii]

This is not our choice. It is how it is implemented in Erlang:

https://github.com/erlang/otp/blob/maint/erts/emulator/beam/beam_emu.c#L6021-L6031 and then

https://github.com/erlang/otp/blob/maint/erts/emulator/beam/beam_emu.c#L6021-L6031 — notice what they do, if it was a tuple in the first chunk of the code, the module name (this = module) will become the last argument.

[Yurii Rashkovskii]

I apologize, the second link should have been https://github.com/erlang/otp/blob/maint/erts/emulator/beam/beam_emu.c#L6054-L6056

[Sasa Juric]

I see a couple of options:

1. The Record module might generate extra modifier clause as Jeremy described:

defrecord A, [:a] do
end

would expand to

defmodule A do
...
def a(new_value, A[])...
def a(A[], new_value)

end

The only downside I see is with recursive structures, i.e. when an instance of A contains a field which is also an instance of A. In this case, a call would always end up using the first clause.

2. A new operator might be introduced e.g. |>> or ||> which passes the last result as the last in list. I would actually find this useful in other contexts, e.g. when working with erlang libs which often take the context as the last argument.

3. Take control over the . operator at the elixir compiler level. Make it run some custom function when doing something.fun_name. This could be used to define Elixirs implementation of tuple modules to adhere to "context as the first argument" convention.

[Saša Jurić]

I still didn't get the answer to the question: Why do you discourage this?

On Sunday, February 24, 2013 4:07:28 AM UTC+1, José Valim wrote:

[Oren Ben-Kiki]

Why not just flip the order and change the way that "." works to be compatible with everything else (that is, add the "self" as the 1st argument)?

[Sasa Juric]

From what I gathered, the Records currently work as they do owing to the so called tuple modules, a still undocumented feature of Erlang:

Assuming that
a = {some_module, ...}.

Then an Erlang call
a:some_fun(1, 2, 3).

will in the runtime be transformed to:
some_module:some_fun(1, 2, 3, a)

In Elixir, when you instantiate a record:
a = MyRecord.new(...)

Then you get the tuple:
{MyRecord, ...}

Consequently, without any special work from the elixir compiler, when we call:
a.something(1, 2, 3)

we will actually call
MyRecord.something(1, 2, 3, a)

or to be more precise:
'Elixir-MyRecord':something(1, 2, 3, a)

So, to make the self work as the first argument, some special processing of the something.fun(...) must be done by the elixir compiler (this is a suggestion #3 in my previous post). Of course, this will probably introduce some performance penalty.

[Yurii Rashkovskii]

Oren,

See my other message on the thread. This is not our decision; this is how Erlang's apply primitive works.

[Oren Ben-Kiki]

Ah, I see. Sorry for being dense :-)

So Sasa's summary 3 options capture the issue pretty thoroughly.

Hmmm. There is a fourth option, but it is "somewhat drastic". What if Elixir has adopted the Erlang convention (also used by Haskell, BTW) that "the object" is always the last argument? That would make this a non-problem, except that it would break almost all the existing Elixir code out there...

I think the simplest option would be option (2), with a proper choice of the new operator. How about `>|` instead of `|>` (the `|` indicating where the object is added, at the start for `|>` and at the end for `>|`)?

[Sasa Juric]

Inline...

On Feb 25, 2013, at 12:04 PM, Oren Ben-Kiki wrote:

> Hmmm. There is a fourth option, but it is "somewhat drastic". What if Elixir has adopted the Erlang convention (also used by Haskell, BTW) that "the object" is always the last argument? That would make this a non-problem, except that it would break almost all the existing Elixir code out there...

I find Elixir's convention much more pleasant. It sucks to write:

Enum.each(fn() -> ... end, enumerable)

It gets progressively worse with folds. This is one of the more annoying characteristics of Erlang libs.

> I think the simplest option would be option (2), with a proper choice of the new operator. How about `>|` instead of `|>` (the `|` indicating where the object is added, at the start for `|>` and at the end for `>|`)?

I think this would be really neat. I already had a need for this on a couple of occasions. I would have implemented a macro, but from what I found Elixir doesn't allow to (re)define operators which is also something I'd like to see in the future.

[Oren Ben-Kiki]

I agree Elixir's choice of order comes out nicer in most cases :-)

I would be very careful with allowing generic definition of operators.

Regardless, I think it would be worthwhile to add `>|`. This will solve two issues - Invoking Erlang libraries, and explicit syntax for accessing the "fast" non-polymorphic record functions.

I'm opening an issue for this.

[Oren Ben-Kiki]

issue 855: https://github.com/elixir-lang/elixir/issues/855

[Jeremy Huffman]

>> Regardless, I think it would be worthwhile to add `>|`. This will solve two issues - Invoking Erlang libraries, and explicit syntax for accessing the "fast" non-polymorphic record functions.
>>
>> I'm opening an issue for this.

I like this as well, as you and Sasa point out it would have other uses beyond "solving" record pipelining. It is still a little suboptimal in my opinion; after all you could use the .dot operator to chain through parts of your pipeline that include record updates. The disadvantage of doing that is you are using an operator at the end of a statement that depends upon the parameter order of the function in the expression following it. If you rearrange your pipeline you must change this operator. This is why I didn't even suggest it as a workaround but how this would look is:

MOD.do_something(a_record, a_value) |>
MOD.something_else(another_value).
a_field_of_a_record(new_value)                |>
MOD.yet_another_thing(ya_value)

All the new operator buys is this:

MOD.do_something(a_record, a_value) |>
MOD.something_else(another_value)     >|
MOD.a_field_of_a_record  (new_value)   |>
MOD.yet_another_thing(ya_value)

Maybe looks a little cleaner, but does not edit any better - just try moving line two to follow line 3.

[Sasa Juric]

Personally, I find all of this inferior to the OO like technique:

  this.

    op_1(...).

    op_2(...).

    ...

Therefore I wonder why Jose advised against it, especially since he uses a similar approach himself, for example in the Dynamo project to manage the connection state.

Personally, after initial experiments in Elixir, I have settled with this OO like approach and after two years of pain, I have finally found it easy to handle complex state and complex mutations of it in a functional programming language.

[Oren Ben-Kiki]

The OO notation is for polymorphic dispatch. If I understand correctly, it works today. "Not encouraged" != "Prohibited". I expect that whether it is/not "good practice" will be discovered by experience (I have my personal opinion FWIW - I'd generally avoid it, Erlang likes Actors and not virtual functions; but that's just me, in my specific problem domain).

The OO notation is also 10x slower and less explicit. Having `>|` as an alternative non-polymorphic faster variant makes sense. It also solves the problem of calling Erlang library functions.

[Jeremy Huffman]

I would really hope that such a small community of a new programming language could achieve a consensus on programming idioms of this importance.  I actually sort of agree with Sasa, but I would not want to see a lot of libraries spring up, half of which use tuple modules all over the place and half of which do not.  Because of the parameter ordering problem you cannot as a user of that library blithely substitute one style of usage for the other.

The next obvious step from what Sasa is doing is to make "defobject" and "defm"(method) macros which can cart around arbitrary state in a HashDict (or something) for you.  You still have problems though with deep-nesting and likely other issues that led Jose away from OO style in .5. So I would like to hear more about that from him.

[José Valim]

Personally, I find all of this inferior to the OO like technique:

  this.

    op_1(...).

    op_2(...).

    ...

Therefore I wonder why Jose advised against it, especially since he uses a similar approach himself, for example in the Dynamo project to manage the connection state.

Personally, after initial experiments in Elixir, I have settled with this OO like approach and after two years of pain, I have finally found it easy to handle complex state and complex mutations of it in a functional programming language.

You summarized this right.

The problem in this approach is that it is bundling behaviour and state in the same bag and therefore it comes with all downsides from it (it is implicit, not as performant, etc). It also comes with extension problems, what happens when you are playing with someone else "record" as in "this.foo(bar).baz(bat)" and it does not have a function that you want? It doesn't feel natural to mix both styles "SomeModule.other(this.foo(bar)).baz(bat)" and then the "polymorphic guarantees" of records are gone since you can't extend/monkey patch it.

That said, the natural way to achieve polymorphism in Elixir is via protocols as they solve the extension problem. The goal of records is not to achieve polymorphism, they simply carry data and the dot notation happens to be how you access this data. They can be used as polymorphic entities but in rare situations. I will explain why I used them in the Dynamo connection:

1. It contains complex state and complex mutations (as you said)

2. It needs to be polymorphic (a connection is an adapter to different web servers) and the interface is huge

3. I was able to impose a limit on the API, the connection contains only low-level functions. Session, cookies and what-not related funtions still come from regular modules, so users likely won't feel a need to extend the connection with their own functions, they will provide their own module

So indeed the word here is discouraged, not forbidden. As Peter Van Roy said in Concepts, Techniques and Models of Programming: we should be using the simplest paradigm available to solve our problems, and bundling state and behaviour should be one of the last tools we'd reach for in our toolbox. Using this approach by default is going to put us in the places FP is trying to avoid us from getting into in the first place.

[Sasa Juric]

I've seen similar differences in approaches in Erlang community, so I don't expect anything different here.

I don't try to enforce the OO style I mentioned, I just say it works for me and helps me to make the code which is easier to maintain.

Regarding performance, a quick comparison of a dynamic call vs a functional one gave me a 2x difference. In absolutes it is about 0.04 microseconds per function call. Usually my functions execute longer, so I don't care about that micro performance loss. If this works in 95% of my code, I can live with the remaining 5% not using it due to performance reason.

Regardless, I found the >| operator still useful.

You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/aBzymSRUnFY/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

[Sasa Juric]

Most often I don't need polymorphism and runtime extensions.

I simply want to chain operations without creating intermediate variables. I found that this currently doesn't work with records, since data is passed as the last argument.

Additionally, I regard this style, a bit noisy:

  data |>

  Mod.op1(...) |>

  Mod.op2(...) |>

  ...

Since I constantly have to repeat Mod.

Also, now I have to have two modules per record: one for the actual record, and another one for my custom operations on that record, which I find a bit funny and excessive. If I want to get name, I will call person.name, but if I want to get full name I must call MyModule.full_name(person). I find this suboptimal.

So for me, it's not about polymorphism, it's about making a simple and concise code which is easier to read and maintain.

Regardless, if we could have the >| operator or some other solution to support chaining with records, I might end up writing pure FP code at least for simpler cases.

I should also remark that we didn't cover defrecordp in this discussion. Personally, I didn't use it so far, but I know it heavily relies on macros, so I don't know if it is possible to support chaining for them as well.