structs and @derive

[Dave Thomas]

There was some talk of adding @derive:

defmodule User do
  @derive [Enumerable, Access]
  defstruct %{name: "default", age: 0}
end

I can’t see this in the source—is it being done differently?

Also, given that the raw type of %User{} is just a map, I’m a little confused why Access isn’t being found anyway.

(guess what I’m writing about..)

Dave

[José Valim]

@derive is not yet implemented.

A struct does not implement any of the protocols that a map does. It is meant to be raw on purpose. We explain a bit why in the upcoming getting started guides:

https://github.com/elixir-lang/elixir-lang.github.com/blob/v0.13/getting_started/16.markdown

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[José Valim]

Ok, here is a proposal for @derive.

For those not familiar with structs, structs do not implement any of the protocols a map implements. In order to get the default functionality like Access and Enumerable protocols, you are supposed to implement the protocols yourself.

Of course implementing all protocols manually for all structs would be cumbersome, so the idea is that you can derive the protocol implementation for a struct from its map implementation. Taking Dave's example:

defmodule User do

@derive [Enumerable, Access]

defstruct name: "default", age: 0

end

The code above will define an implementation for both Enumerable and Access protocols for the User struct.

This approach works fine for the trivial cases but in some cases you may want a tailored implementation. For example, deriving a JSON implementation for a struct could automatically inline all fields on the protocol definition! In other words, the implementation of the JSON protocol for User could look like this:

def to_json(%User{name: name, age: 0}) do

"{name: " <> to_json(name) <> ", age:" <> to_json(age) <> "}"

end

We are basically saying: we know which fields this struct has so let me generate an implementation tailored for this struct and avoid wasting CPU cycles converting the keys to binaries. In order to support this, we just need to provide a callback. Here are the semantics I propose:

Considering the User struct above with @derive [JSON], we will

1. First check if the JSON.Map module exists (i.e. the implementation for maps)

2. If not, an error is raised

3. We will check if JSON.Map exports the __deriving__/2 function

4. If so, we will invoke it passing the environment (Macro.Env) and the struct fields as arguments. This callback must define an implementation of the JSON protocol for the User struct

5. If no __deriving__/2 function exists, we will define a JSON.User automatically that simply dispatches to JSON.Map

This makes it convenient for most protocols which don't need to worry with how derive works. However, it also makes it possible for custom implementations as shown above. Also, since __deriving__/2 will receive the environment as an argument, a developer should be able to customize how a custom implementation works via attributes:

defmodule User do

@json [skip: :age]

@derive [Enumerable, Access, JSON]

defstruct name: "default", age: 0

end

This allow us to reproduce the behaviour as seen in Go structs and other FP languages with tags (example here). I don't expect many protocols to use this feature but it will be definitely handy for the cases it is desired.

Is this reasonable?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Dave Thomas]

On Wed, Mar 19, 2014 at 5:53 PM, José Valim <jose....@plataformatec.com.br> wrote:

@derive is not yet implemented.

A struct does not implement any of the protocols that a map does. It is meant to be raw on purpose. We explain a bit why in the upcoming getting started guides:

I wonder if you should special-case is_map(some_struct) then? I understand the difference between the primitive type and the module​​, but maps and Map are pretty closely entwined, and having something with is a map but doesn’t support Access might be confusing.

[Dave Thomas]

defmodule User do

@json [skip: :age]

@derive [Enumerable, Access, JSON]

defstruct name: "default", age: 0

end

Would you consider​​

defmodule User do

@derive [Enumerable, Access, JSON: {except: :age}]

defstruct name: "default", age: 0

end

To me, it seems to be tidier to put the JSON parameters at the point you include them. 

[Josh Adams]

+1, I prefer dave's syntax readability-wise

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Josh Adams

[Alexei Sholik]

What if you wanted to support multiple formats for serialization? With Jose's approach one could establish a convention, for instance:

defmodule User do

@skip_fields [:age]

@derive [Enumerable, Access, JSON, XML, ProtoBuf]

defstruct name: "default", age: 0

end

With Dave's approach each protocol would require it's own unique set of options, packed into a single list.

Best regards
Alexei Sholik

[Dave Thomas]

On Wed, Mar 19, 2014 at 11:08 PM, Alexei Sholik <alcos...@gmail.com> wrote:

What if you wanted to support multiple formats for serialization? With Jose's approach one could establish a convention, for instance:

defmodule User do

@skip_fields [:age]

@derive [Enumerable, Access, JSON, XML, ProtoBuf]

defstruct name: "default", age: 0

end

So, given this, would

%User{...}[:age]

work or not? Does the @skip_fields apply to Access, or not? How can I tell?

And why skip, where every other API uses except: ?

If it was a common requirement to provide common parameters to a number of derived protocols, how about

@derive [Enumerable, Access]
@derive [[JSON, XML, Protobuf], except: age]

[Carsten Bormann]

I find the idea of leading people to manually generate JSON texts abominable, because people will be making exactly the kinds of mistakes the example code is making. Maybe there is a better example...

Sent from mobile

[José Valim]

Carsten, unless I am missing something, the point is to exactly generate it automatically.

On Thursday, March 20, 2014, Carsten Bormann <cabo...@gmail.com> wrote:

I find the idea of leading people to manually generate JSON texts abominable, because people will be making exactly the kinds of mistakes the example code is making. Maybe there is a better example...

Sent from mobile

On 20 Mar 2014, at 01:35, José Valim <jose....@plataformatec.com.br> wrote:

Ok, here is a proposal for @derive.

For those not familiar with structs, structs do not implement any of the protocols a map implements. In order to get the default functionality like Access and Enumerable protocols, you are supposed to implement the protocols yourself.

Of course implementing all protocols manually for all structs would be cumbersome, so the idea is that you can derive the protocol implementation for a struct from its map implementation. Taking Dave's example:

defmodule User do

@derive [Enumerable, Access]

defstruct name: "default", age: 0

end

The code above will define an implementation for both Enumerable and Access protocols for the User struct.

This approach works fine for the trivial cases but in some cases you may want a tailored implementation. For example, deriving a JSON implementation for a struct could automatically inline all fields on the protocol definition! In other words, the implementation of the JSON protocol for User could look like this:

def to_json(%User{name: name, age: 0}) do

"{name: " <> to_json(name) <> ", age:" <> to_json(age) <> "}"

end

We are basically saying: we know which fields this struct has so let me generate an implementation tailored for this struct and avoid wasting CPU cycles converting the keys to binaries. In order to support this, we just need to provide a callback. Here are the semantics I propose:

Considering the User struct above with @derive [JSON], we will

1. First check if the JSON.Map module exists (i.e. the implementation for maps)

2. If not, an error is raised

3. We will check if JSON.Map exports the __deriving__/2 function

4. If so, we will invoke it passing the environment (Macro.Env) and the struct fields as arguments. This callback must define an implementation of the JSON protocol for the User struct

5. If no __deriving__/2 function exists, we will define a JSON.User automatically that simply dispatches to JSON.Map

This makes it convenient for most protocols which don't need to worry with how derive works. However, it also makes it possible for custom implementations as shown above. Also, since __deriving__/2 will receive the environment as an argument, a developer should be able to customize how a custom implementation works via attributes:

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

[José Valim]

I prefer the attribute style because it gives you more flexibility than passing options. You could have for example default values for @json being inserted by a common macro shared in between modules (it would definitely be handy for ecto, for example).

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

[Carsten Bormann]

Maybe you can add "can you spot the error? That's

Why this should never be coded manually. "

Sent from mobile

[Dave Thomas]

On Mar 20, 2014 2:22 AM, "José Valim" <jose....@plataformatec.com.br> wrote:
>
> I prefer the attribute style because it gives you more flexibility than passing options. You could have for example default values for @json being inserted by a common macro shared in between modules (it would definitely be handy for ecto, for example).

Cool.

Will you be changing 'use' to work like that, too?

[Eric Meadows-Jönsson]

The __using__ macro has always been able to access module attributes defined in the module it is called from.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Eric Meadows-Jönsson

[Dave Thomas]

On Thu, Mar 20, 2014 at 9:33 AM, Eric Meadows-Jönsson <eric.meado...@gmail.com> wrote:

The __using__ macro has always been able to access module attributes defined in the module it is called from.

Of course. But that's not how you typically pass parameters to it... You put them on the 'use' line.

[José Valim]

I have started implementing this proposal and there is one particular point that is not clear. Should the __struct__ field be included on the derived implementation or not?

For example, if you do as originally proposed:

defmodule User do

  @derive [Enumerable, Access]

  defstruct name: "default", age: 0

end

Would you expect user[:__struct__] to return the struct value or nil (as if it was not available)? Would you expect Enum.to_list(user) to include a {:__struct__, User} tuple in the resulting list? Or should those be automatically removed?

Keep in mind that automatically removing __struct__ on deriving may be seen as inconsistent. For example, Enum.to_list(user) would be different than Map.to_list(user). However, this may be exactly the behaviour we want.

[Josh Adams]

I personally feel that hiding that implementation detail is likely to cause more confusion and make it seem more magical than it is.  I'm for returning the struct value in that case and in Enum.to_list.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Josh Adams