Proposal for required fields in struct expansion

[José Valim]

Hello everyone,

I would like to propose an extension to defstruct that will require certain fields to be given when expanding it. Here is an example:

defmodule User do

  defstruct [age: nil], # optional

            [name: nil] # required

end

 

With this feature, %User{} will fail as the :name field was not specified. %User{name: "foo"} or %User{name: nil} will both work as expected. The main use case is to make sure all important fields are set when building the data. For example, we can use such fields in the new date time types to enforce proper data representation.

Extra notes

1. The required fields are given as second argument to defstruct as the API must remain backwards compatibility

2. The fields are required only when building structs. Matching will always work without specifying any field, for example: %User{} = user

3. The Kernel.struct/2 function, used to build structs dynamically, won't check for required keys. Kernel.struct!/2 should be used if you want to check for required keys (and also check that no extra keys are given)

4. defexception will leverage the same functionality

Implementation

Implementation-wise, structs will now defined a __struct__/1 function, besides the regular __struct__/0 function. It has not been decided yet how such function will behave given it must work both for compile-time (%User{}) and runtime (struct! User, %{}) checks.

Feedback

Now it is your turn. :)

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[José Valim]

To clarify, both :age and :name fields will be present in the underlying User struct/map. The proposal is only about fields which must be enforced when building the structure.

We will likely need better names than optional/required.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Theron Boerner]

Why require a value for the required fields? I.e. :name vs [name: nil]. If it's required, the nil value seems verbose.

(On mobile, apologies for poor explanation)

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KYeJCq7u2iubGoNSnNS5y94o5Ju8_xs_n9PZ_Q1OTKRg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Hamilton]

I am not a fan of the proposed signature. Generally when we have multiple clauses of different arity, we try to make them purely extensions of the first. In the proposal, we go from (fields) to (optional_fields, required_fields). While one could say that it's really (optional_fields) to (optional_fields, required_fields), I think that's changing the semantics of the current signature, which to me is all the fields. I will submit, however, that this is a bit subjective and realistically the same in practice.

I would propose instead:

defstruct [age: nil, name: nil], required: [:name]

It not only maintains semantic backwards compatibility, but there isn't an implicit meaning behind the two different arguments. It's very clear that we have a list of arguments then an explicit list of required fields.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2B5b%2BxxcvMOL-n6XyJ4mcQcumTU5B0AhjaTuc7qk-0P1g%40mail.gmail.com.

[José Valim]

It is not required, it has the same rules as the regular fields.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAOtk34cFt09TnKccLerF7WJ2d66kUO2rcJqZ09NBKq2DB9qJ%2Bw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[Onorio Catenacci]

I'm +1 with Peter on this. I think having to explicitly call out required fields is more in keeping with existing syntax and maybe a little less error-prone.  

That said, I think the idea of allowing required fields is a good one.  

By the way--maybe because it's been a long day and I"m tired but I don't follow this point:

The Kernel.struct/2 function, used to build structs dynamically, won't check for required keys. Kernel.struct!/2 should be used if you want to check for required keys (and also check that no extra keys are given)

You mention Kernel.struct/2 in both cases?  Or are my eyes bad?  :)

--

Onorio

[Peter Hamilton]

Onorio: Your eyes are bad. it's struct vs stuct! (struct bang).

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/bede9a4c-8428-4447-bda1-c77eccd64893%40googlegroups.com.

[Onorio Catenacci]

Thanks Peter!  Figured I had to be missing something.

You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/rW8c3_xSVCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAOMhEnyNx3O%3D7QEk3Cw-Spvqaw6Ve1RpozVE6t_nsxQ8sxFU6Q%40mail.gmail.com.

[eksperimental]

José: great new feature. Definitely a needed one!
and +1 on Peter Hamilton's suggestion on how to define required fields.

An extra feature that will save us developers a lot of headeaches is
the ability to define a list of accepted values per field. As as the
current state, functions silently fail unexpected values are given

On Tue, 24 May 2016 23:35:34 +0000
Peter Hamilton <petergh...@gmail.com> wrote:

> I am not a fan of the proposed signature. Generally when we have
> multiple clauses of different arity, we try to make them purely
> extensions of the first. In the proposal, we go from (fields) to
> (optional_fields, required_fields). While one could say that it's
> really (optional_fields) to (optional_fields, required_fields), I
> think that's changing the semantics of the current signature, which
> to me is all the fields. I will submit, however, that this is a bit
> subjective and realistically the same in practice.
>
> I would propose instead:
>
> defstruct [age: nil, name: nil], required: [:name]
>
> It not only maintains semantic backwards compatibility, but there
> isn't an implicit meaning behind the two different arguments. It's
> very clear that we have a list of arguments then an explicit list of
> required fields.
>
> On Tue, May 24, 2016 at 4:26 PM José Valim
> <jose....@plataformatec.com.br> wrote:
>
> > To clarify, both :age and :name fields will be present in the
> > underlying User struct/map. The proposal is only about fields which
> > must be enforced when building the structure.
> >
> > We will likely need better names than optional/required.
> >

> > *José Valim*

> > www.plataformatec.com.br
> > Skype: jv.ptec
> > Founder and Director of R&D
> >
> > On Wed, May 25, 2016 at 1:17 AM, José Valim <
> > jose....@plataformatec.com.br> wrote:
> >
> >> Hello everyone,
> >>
> >> I would like to propose an extension to defstruct that will require
> >> certain fields to be given when expanding it. Here is an example:
> >>
> >> defmodule User do
> >>
> >> defstruct [age: nil], # optional
> >>
> >> [name: nil] # required
> >>
> >> end
> >>
> >>
> >> With this feature, %User{} will fail as the :name field was not
> >> specified. %User{name: "foo"} or %User{name: nil} will both work as
> >> expected. The main use case is to make sure all important fields
> >> are set when building the data. For example, we can use such
> >> fields in the new date time types to enforce proper data
> >> representation.
> >>

> >> *Extra notes*

> >>
> >> 1. The required fields are given as second argument to defstruct
> >> as the API must remain backwards compatibility
> >>
> >> 2. The fields are required only when building structs. Matching
> >> will always work without specifying any field, for example:
> >> %User{} = user
> >>
> >> 3. The Kernel.struct/2 function, used to build structs
> >> dynamically, won't check for required keys. Kernel.struct!/2
> >> should be used if you want to check for required keys (and also
> >> check that no extra keys are given)
> >>
> >> 4. defexception will leverage the same functionality
> >>

> >> *Implementation*

> >>
> >> Implementation-wise, structs will now defined a __struct__/1
> >> function, besides the regular __struct__/0 function. It has not
> >> been decided yet how such function will behave given it must work
> >> both for compile-time (%User{}) and runtime (struct! User, %{})
> >> checks.
> >>

> >> *Feedback*

> >>
> >> Now it is your turn. :)
> >>

> >> *José Valim*

> >> www.plataformatec.com.br
> >> Skype: jv.ptec
> >> Founder and Director of R&D
> >>
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "elixir-lang-core" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an email to elixir-lang-co...@googlegroups.com.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2B5b%2BxxcvMOL-n6XyJ4mcQcumTU5B0AhjaTuc7qk-0P1g%40mail.gmail.com

> > <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2B5b%2BxxcvMOL-n6XyJ4mcQcumTU5B0AhjaTuc7qk-0P1g%40mail.gmail.com?utm_medium=email&utm_source=footer>
> > .

[José Valim]

Given Peter's feedback, what if we define it as:

defstruct [:name, :age], enforce: [:name]

The idea of picking :enforce instead of :required is to avoid any possible confusion that some of those fields won't be effectively present in the struct.

Thoughts?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/20160525080713.279fe1dd.eksperimental%40autistici.org.

[Mike Evans]

+1 as it solves the immediate clarity issue nicely with an undeniably strong word.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4K5-pCjrDudw1MgXbGcwfhfn%3Da%2BpukmO00cgz1nsYg9Zg%40mail.gmail.com.

[Parker Selbert]

The most recent proposal, using `enforce: [:name]` is extremely clear, and nicely backward compatible. Overall I love the recent focus on enforcing the presence of attributes and data integrity.

 

— Parker

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/6C3AEDA3-34F1-4CED-8B4E-B1769C83D812%40silljays.com.

[José Valim]

Thanks Mike and Parker for the feedback so far.

Eric has proposed another approach, via module attributes:

@enforce [:name]

defstruct [:name, :age]

One of the benefits of using module attributes is that it matches nicely the already supported @derive attribute:

@derive Poison.Encoder

@enforce [:name]

defstruct [:name, :age]

Furthermore, the module attribute composes better. For example, if we want to support @enforce in defexception (which is based defstruct), we don't need to change anything, it just works:

@enforce [:message]

defexception [:message]

The same could work with Ecto:

@enforce [:name]
schema "users" do
  ...
end

The only downside is that a module attribute can be silently defined. For example, someone may set @enforce [:foo, :bar] in their module for other reasons and now they will conflict. However, in such cases we can easily check and guarantee all fields given to @enforce are also defined in the struct.

Can anyone think of other pros-and-cons here?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/1464186821.454905.618394761.3E78A7BC%40webmail.messagingengine.com.

[Chris McCord]

I'm onboard with the module attribute approach as it aligns with what we already have. The potential existing `@enforce` conflicts would be extremely minimal so +1 to eric's proposal. 

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Ji7JxRFN7fw72VNwqD%2BaqPZ3MQy7EJi-DcUztG-S1KKA%40mail.gmail.com.

[Saša Jurić]

I actually like passing options more. In fact, I wonder why is derive an attribute, and not an option?

Something like, 

    defstruct [:name, :age], derive: Poison.Encoder, enforce: [:name]

seems more idiomatic to me. 

[Myron Marston]

I like this feature a lot, but dislike the term enforce. IMO, it muddies things and is much less clear than require. In my experience, the term required is generally used to indicate fields that must be present. For example, JSON schema uses that term. enforce gives me the sense that some kind of validation policy is applied but does not describe what the validation policy is. Does it enforce the type of values allowed for the named fields? Does it enforce the fields are present? Does it enforce the named fields are not present? Does enforce: [:percent] validate that percent is between 1 and 100?

Obviously, given there is no further information provided with the struct definition, it would be reasonable to assume that it means that the presence of the named fields is enforced, but I fail to see how enforce is preferable to require.

Regardless of which word is chosen, I think it would be easy to write the past-tense form of the word (e.g. enforced or required instead of enforce or require). With the keyword argument approach, defstruct could provide an nice error when you use the wrong form. With the module attribute approach, it wouldn’t really make sense to detect that, since module attributes can be user-defined and any attribute is valid.

So I would vote in favor of:

defstruct [:name, :age], require: [:name]

…and also ask that it provides a nice error if you use required: or any other invalid option.

Myron

[José Valim]

In fact, I wonder why is derive an attribute, and not an option?

For all the reasons said above. :)

[Peter Hamilton]

I'm pretty happy with any of these approaches. They all cover my main concern which was to not split the fields into two lists.

On Wed, May 25, 2016 at 8:43 AM José Valim <jose....@plataformatec.com.br> wrote:

In fact, I wonder why is derive an attribute, and not an option?

For all the reasons said above. :)

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4LLqwY2ErB7yNQJXweq1pVU8gxwCg3mkcym_RczFsNDCA%40mail.gmail.com.

[José Valim]

I like this feature a lot, but dislike the term enforce. IMO, it muddies things and is much less clear than require. In my experience, the term required is generally used to indicate fields that must be present. For example, JSON schema uses that term. enforce gives me the sense that some kind of validation policy is applied but does not describe what the validation policy is. Does it enforce the type of values allowed for the named fields? Does it enforce the fields are present? Does it enforce the named fields are not present? Does enforce: [:percent] validate that percent is between 1 and 100?

That's a very good point. I believe :require is better than :required (thanks for that) but I still believe the require/required connotation can be confusing. Also require may be a bit overloaded in Elixir (given the require special form and Code.require_file). Can you think of other options? :)

[Chris McGrath]

On 25 May 2016, at 16:58, José Valim <jose....@plataformatec.com.br> wrote:

That's a very good point. I believe :require is better than :required (thanks for that) but I still believe the require/required connotation can be confusing. Also require may be a bit overloaded in Elixir (given the require special form and Code.require_file). Can you think of other options? :)

I had a quick look in the OS X Thesaurus and these stood out for me.

compulsory

essential

need

Chris

[Paul Schoenfelder]

Given the context I'm not sure `require` is all that confusing personally. It seems familiar and very clear. +1 from me.

Paul

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/C87C5A29-BB52-454D-8CAF-6534F34310B7%40chrismcg.com.

[Geovane Fedrecheski]

Hi everyone,

Adding my 2 cents, I actually see `required` as a better option than `require`. 

Reasoning: I see `defstruct` as kind of a declaration. So the thinking goes like that: I'm declaring a structure who has fields x, y and z; the required fields are x and z. Also, `required` in this context gains a different semantic compared to `require` (as in `require_file`).

On a side idea, brainstorming a bit, something like `not_empty`, or `not_null` came to my mind.

quarta-feira, 25 de Maio de 2016 às 12:58:53 UTC-3, José Valim escreveu:

I like this feature a lot, but dislike the term enforce. IMO, it muddies things and is much less clear than require. In my experience, the term required is generally used to indicate fields that must be present. For example, JSON schema uses that term. enforce gives me the sense that some kind of validation policy is applied but does not describe what the validation policy is. Does it enforce the type of values allowed for the named fields? Does it enforce the fields are present? Does it enforce the named fields are not present? Does enforce: [:percent] validate that percent is between 1 and 100?

That's a very good point. I believe :require is better than :required (thanks for that) but I still believe the require/required connotation can be confusing. Also require may be a bit overloaded in Elixir (given the require special form and Code.require_file). Can you think of other options? :)

quarta-feira, 25 de Maio de 2016 às 12:58:53 UTC-3, José Valim escreveu:

I like this feature a lot, but dislike the term enforce. IMO, it muddies things and is much less clear than require. In my experience, the term required is generally used to indicate fields that must be present. For example, JSON schema uses that term. enforce gives me the sense that some kind of validation policy is applied but does not describe what the validation policy is. Does it enforce the type of values allowed for the named fields? Does it enforce the fields are present? Does it enforce the named fields are not present? Does enforce: [:percent] validate that percent is between 1 and 100?

That's a very good point. I believe :require is better than :required (thanks for that) but I still believe the require/required connotation can be confusing. Also require may be a bit overloaded in Elixir (given the require special form and Code.require_file). Can you think of other options? :)

[Michał Muskała]

I think for the reasons given, especially ability to easily integrate
it with exceptions, ecto or any other code that internally defines
structs.

As for the name, I don't have a strong opinion. I can see how required
can be confusing in the sense that nil is a valid value - it can be
reasonably expected that required does not allow for nil.

> https://groups.google.com/d/msgid/elixir-lang-core/CAK%3D%2B-TsErYwW3z9L7zeFtmEmqEkgaz-VYUeskt%2BKDSK%2BuZDtbg%40mail.gmail.com.

[Saša Jurić]

Which all reasons? I see only one possible reason, which is “composability”.

If that’s the case, I’m not convinced it’s a good enough reason.

The approach with attributes seems clunky to me. A struct def is spread over three separate expressions, which must appear in some particular order. First the attributes of the struct appear, and only then the struct. That reads funny.

Also, it’s a stretch, but it’s possible that a sloppy programmer separates these expressions, say by inserting a multiline comment, a typespec, or even something completely unrelated. Then it becomes harder to spot the complete specification of the struct.

Another issue is that this “composability” is based on setting a piece of info into some context. That’s seems to me more like a mutable approach, and it suffers from same pitfalls. Say I want to move the struct to another module, I have to pay attention to move all three expressions (which might be separated by some other not necessarily struct related stuff). Or say in my macro which defines a struct, I want to prevent some option (e.g. require). I need to work explicitly to opt-out of that (and I need to be aware of that fact in the first place).

Finally, this approach might set a bad example. Other macro authors might use the same technique where plain arguments would suffice.

In contrast, having these macros accept valid options as arguments allows us to specify the struct in a single expression (which IMO reads nicer). It makes the interface more explicit about what’s accepted, and it forces the authors of custom macros, such as defexception/defstruct to opt-in to those extra features. And it sets a nice example :-)

I’m not saying that attribute approach is necessarily bad. But given that you mention only two examples (defexception and schema), and the extra code that would be required with the argument approach is small, I’m not sure this is a good trade off in this case.

[Michał Muskała]

Well, I haven't done a good job writing. I meant to say that I prefer
module attributes.

[Paul Schoenfelder]

The one thing I don't like about using options with defstruct is that I tend to define my structs with the keyword list syntax, defining default values - if I want to do that *and* define required fields, I now have to wrap the definition of the fields+defaults in a list, i.e.: defstruct [foo: "bar",           baz: :qux,           bar: nil], required: [:foo, :baz] Granted, that's not a big deal, but I like the following much better: @required [:foo, :baz] defstruct foo: "bar",           baz: :qux,           bar: nil That said, I don't disagree with your reasoning, but I think the ship has already sailed with regards to precedent (i.e. `derive`). Paul

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/0A95E06E-8689-4D3C-954E-17046EEBDB8E%40gmail.com.

[Ciarán Walsh]

Why not simply `required_keys:` then? It makes the meaning explicit, and removes confusion with allowing nil values.

[Saša Jurić]

That’s a good point about optional args!

I don’t think the ship has sailed though. We could soft deprecate @derive (it still works, but it’s not documented), and introduce options. So the old code would not be backward incompatible until 2.0.

[José Valim]

Also, it’s a stretch, but it’s possible that a sloppy programmer separates these expressions, say by inserting a multiline comment, a typespec, or even something completely unrelated. Then it becomes harder to spot the complete specification of the struct.

Sloppy programmers will break all kinds of code. I generally really dislike the sloppy programmer argument. Do we really want to guide our decisions based on programmers that are sloppy?

 

Say I want to move the struct to another module, I have to pay attention to move all three expressions (which might be separated by some other not necessarily struct related stuff). Or say in my macro which defines a struct, I want to prevent some option (e.g. require).

If you are moving code around, you always need to pay attention. :) What if you are defining the struct like this:

if some_condition? do

  ... some code ...

  defstruct

else

  defstruct

end

If you get the first defstruct and ignore the rest... it will be wrong.

In contrast, having these macros accept valid options as arguments allows us to specify the struct in a single expression (which IMO reads nicer). It makes the interface more explicit about what’s accepted

This is, however, a good argument. If someone makes a typo or provide an invalid option, it is easy to check with options, but not possible with module attributes.

[Saša Jurić]

On 25 May 2016, at 18:46, José Valim <jose....@plataformatec.com.br> wrote:

Also, it’s a stretch, but it’s possible that a sloppy programmer separates these expressions, say by inserting a multiline comment, a typespec, or even something completely unrelated. Then it becomes harder to spot the complete specification of the struct.

Sloppy programmers will break all kinds of code. I generally really dislike the sloppy programmer argument. Do we really want to guide our decisions based on programmers that are sloppy?

All other things being equal or very similar, I think the interface which prevents sloppiness is better. 

 

Say I want to move the struct to another module, I have to pay attention to move all three expressions (which might be separated by some other not necessarily struct related stuff). Or say in my macro which defines a struct, I want to prevent some option (e.g. require).

If you are moving code around, you always need to pay attention. :) What if you are defining the struct like this:

if some_condition? do

  ... some code ...

  defstruct

else

  defstruct

end

If you get the first defstruct and ignore the rest... it will be wrong.

It will break on compile, hinting me that I didn’t move the complete code. Yes, I need to pay attention, but compiler has my back.

I understand what you’re saying, and I agree my point is moot, but I still think that a single “atomic” expression is better than three which semantically boil down to one. 

The former approach means I have to know only Elixir syntax to understand what constitutes a single expression.

The latter approach means I have to know the semantics of @derive, @require and defstruct/defexception/schema, to know that these things tie together. 

[José Valim]

All other things being equal or very similar, I think the interface which prevents sloppiness is better. 

Except they are not equal and similar. One provides composability exactly because it is less restrictive. The other provides better feedback for being more restrictive with checking of options and less composable. We can make both assessments without bringing the sloppy programmer into the discussion.

 

It will break on compile, hinting me that I didn’t move the complete code. Yes, I need to pay attention, but compiler has my back.

Not necessarily?

    if some_option? do

      # ...

      defstruct foo: nil

    else 

      defstruct foo: nil, bar: :baz

    end

  

The latter approach means I have to know the semantics of @derive, @require and defstruct/defexception/schema, to know that these things tie together. 

In both cases you need to understand the semantics of derive and require. For module attributes though, you do need to know how these things tie in together. It is the trade-off between restrictive and composable.

[Glauber Campinho]

I'm quite new to Elixir, maybe I'm talking bullshit, but this looks to me almost like a guard clause for the struct. Can't make the expressions on guard clauses or match available somehow to each field?

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BicfA9Z%3DesQ%3Dw08XmtD_LvQKF2f4%3DEYx_N96fhL8FB5g%40mail.gmail.com.

[Saša Jurić]

> In both cases you need to understand the semantics of derive and require. For module attributes though, you do need to know how these things tie in together. It is the trade-off between restrictive and composable.

I don’t see that trade-off here.

defstruct which takes parameters is composable. Any macro that uses defstruct can take the same parameters (or some subset), and pass them to defstruct, and it will work. It can also take some different parameters, and derive from them the ones that needs to be passed to defstruct.

The “composability” of module attribute is really an equivalent of a global var. One expression sets something into the context, and another, seemingly unrelated expression depends on it. Therefore, it works out of the box for anyone who happens to use the same expression. But I’d say it’s more like a shortcut, rather than something I’d choose as the first option. I’m not saying it’s always a bad approach, but I’d expect some strong reasons for choosing it. I don’t see such reasons here.

A trade-off which IMO does exist is opt-in vs opt-out. With params, each macro writer needs to explicitly allow the supported options. With module attributes, these are implicitly available without writing a single line of code.
It’s worth pointing out that this trade-off revolves around writers of macros which use structs. For users of macros, I think the difference between two approaches is mostly aesthetic.

Since I don’t expect macros written on top of defstruct are such common case, a more complicated, less idiomatic interface backed with attributes is IMO not justified here.
More often than writing such macros, I define structs, and having the standard interface (macro which takes params) is IMO better suited for that.

[Paul Smith]

I like Ciarán Walsh's idea of `required_keys`. I don't have a strong preference for an attribute or an option to `destruct`

I think this proposal overall would be a great addition!

[José Valim]

The “composability” of module attribute is really an equivalent of a global var. One expression sets something into the context, and another, seemingly unrelated expression depends on it.

I don't think this is a far comparison. First, they are not global, they belong to a module. Second, a macro can only fetch its latest value. Macros cannot depend on it as a reference in a way global variable works in most languages. Both of those are big reasons why global variables are frowned upon and none of these are present in module attributes.

Let me be clear: I am not yet advocating for one approach or the other in this thread, but we need to be fair in our comparisons if we want to have a fair discussion.

While there is an implicit association between the module attribute and the upcoming construct (which is probably what you tried to hint to above), that's an extremely common construct in the Elixir module body. It is the reason why we can write code like this:

@doc "Delegates to Foo.bar"

@spec bar() :: Foo.bar

defdelegate bar(), to: Foo

It is also why I can @doc and @spec the function defined by defstruct.

Constructs like imports and aliases would also perfectly fit your definition of "one expression sets something into the context, and another, seemingly unrelated expression depends on it". Aliases, in particular, wouldn't give any compile time guarantee. And those aren't global variables either.

A trade-off which IMO does exist is opt-in vs opt-out. With params, each macro writer needs to explicitly allow the supported options. With module attributes, these are implicitly available without writing a single line of code.
It’s worth pointing out that this trade-off revolves around writers of macros which use structs. For users of macros, I think the difference between two approaches is mostly aesthetic.

That's a good way to see it as well.

[Bryan Joseph]

I like the idea. I've run into situations where it could have been useful. I may be way off, but is there possibly something to be borrowed from the recently announced clojure.spec? http://clojure.org/about/spec

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/a9bd4321-4e3f-49a7-84af-95951e5fdb29%40googlegroups.com.

[Saša Jurić]

On 25 May 2016, at 20:23, José Valim <jose....@plataformatec.com.br> wrote:

Constructs like imports and aliases would also perfectly fit your definition of "one expression sets something into the context, and another, seemingly unrelated expression depends on it". Aliases, in particular, wouldn't give any compile time guarantee. And those aren't global variables either.

As I said in the previous post: "I’m not saying it’s always a bad approach, but I’d expect some strong reasons for choosing it.”.

I can see some reasons of the chosen approach for the cases you mention, while the only benefit in the context of defstruct is IMO not strong enough to justify the approach. In fact, I’m not even sure if it is a benefit, for the reasons I mentioned earlier.

[Petrică Clement Chiriac]

We want to enforce to pass :name 

%User{} will fail 

%User{name: "foo"} or %User{name: nil} are fine

We can solve this with one function: new/1, create/1

Is not about key: we have :name key anyway

Is not about value: nil or any term is valid.

I just note some remarks:

 "To have one value" --- This is wrong. In all cases we have one value, nil is valid value.

 "ability to define a list of accepted values per field" --- We do not have types 

 "enforcing the presence of attributes and data integrity" -- We do not do that 

 "in the sense that nil is a valid value - it can be reasonably expected that required does not allow for nil."  -- In not about nil

I see a lot of confusion. 

all problem is related to call one function with right parameters according arity and guards

What about to specify one function that create this struct 

defmodule User do

              @defstruct_new create  

  defstruct [age: nil, name: nil]

              def create(%{name: name, age: age}) when is_integer(age) do

                 # init struct

              end 

end

 

What I like here, in this way Date struct can be created only with int values :)

 

--

Petrică Clement Chiriac

On Wednesday, May 25, 2016 at 2:18:10 AM UTC+3, José Valim wrote:

Hello everyone,

I would like to propose an extension to defstruct that will require certain fields to be given when expanding it. Here is an example:

defmodule User do

  defstruct [age: nil], # optional

            [name: nil] # required

end

 

With this feature, %User{} will fail as the :name field was not specified. %User{name: "foo"} or %User{name: nil} will both work as expected. The main use case is to make sure all important fields are set when building the data. For example, we can use such fields in the new date time types to enforce proper data representation.

Extra notes

1. The required fields are given as second argument to defstruct as the API must remain backwards compatibility

2. The fields are required only when building structs. Matching will always work without specifying any field, for example: %User{} = user

3. The Kernel.struct/2 function, used to build structs dynamically, won't check for required keys. Kernel.struct!/2 should be used if you want to check for required keys (and also check that no extra keys are given)

4. defexception will leverage the same functionality

Implementation

Implementation-wise, structs will now defined a __struct__/1 function, besides the regular __struct__/0 function. It has not been decided yet how such function will behave given it must work both for compile-time (%User{}) and runtime (struct! User, %{}) checks.

Feedback

Now it is your turn. :)

[José Valim]

Hi Petrica,

This is a good proposal however it has a very strong downside: %User{} today gives a compile-time guarantee that the keys will be checked. Your code is effectively moving those to runtime. This is a big departure and a big change.

So let's step back: what is the current proposal (the one you replied to) about? It is a compile-time mechanism to ensure that enforced keys are given. It is not for the developers consuming an API but for the API writers themselves. For example, someone could even use it temporarily when adding a new field to a struct, then enforcing the field temporarily to catch all compile-time errors and then finally removing the enforcement. The current proposal is not a form of validation nor it aims to provide a guarantee. And most of all: it is about a *compile-time* guarantee.

Your proposal goes in the opposite direction: it is about a runtime validation since we cannot really check age is an integer at compile time.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Onorio Catenacci]

I think we all agree that this is a good idea; the dispute seems to be over the syntax we should adopt.  

I guess that like Sasa I'm not a big fan of using attributes to accomplish this. I'd much rather see us use a keyword list for this. I can't say--"Hey, here's this empirical proof that using attributes for this purpose is worse than using a keyword list"--it just feels as if using a keyword list is a bit more explicit in terms of the association with the struct.  But if the attribute is immediately before the struct definition I suppose that's fairly explicit too.

Is there any difference in documenting a keyword list (as opposed to documenting a new attribute)?  Is either one easier for new folks to follow in terms of finding it in the documentation?  Just the other day we were working with someone on Slack and he was mentioning about not being able to find :normal in the Phoenix docs (I finally found an example but only by using Google to search the site).  But I'm not sure that attributes are easier to search in terms of documentation.

For what it's worth, I don't know if that helps to maybe move the discussion away from personal preferences but that's what I'm trying to do.

--

Onorio

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/rW8c3_xSVCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4LAy%2BnyO0bv5tpuA0TYRYgHMtYLtryPYpe1s3smX0OM8g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Onorio Catenacci
 
http://onor.io

http://www.google.com/+OnorioCatenacci

[Petrică Clement Chiriac]

You are right!

My big concern is what developer thinks ...

Is not about keys, values, nil or types ... we can't force this as long we can change data later  

I think things become too complicated and we need very good reasons to do that... or to see more usecases

I am supporting Saša Juric: 

 "I’m not saying it’s always a bad approach, but I’d expect some strong reasons for choosing it.”.

--

Petrică Clement Chiriac  

[Michał Muskała]

2016-05-25 19:30 GMT+02:00 Saša Jurić <sasa....@gmail.com>:
> defstruct which takes parameters is composable. Any macro that uses defstruct can take the same parameters (or some subset), and pass them to defstruct, and it will work. It can also take some different parameters, and derive from them the ones that needs to be passed to defstruct.
>

The problem here is that library needs to explicitly support it, so
you can't use a new feature with old code without changing it.
Another, even bigger issue, is backward compatibility in libraries.
If we wanted to support that in Ecto, while supporting both Elixir 1.2
without this feature and 1.3 with this feature is to check elixir
version and add multiple ifs, in various places. This is not bearable
and I suspect most libraries will either break backwards compatibility
or simply ignore this feature.

> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to elixir-lang-co...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-core/CAP%3DvNq_DFyBTMboje%2BaVPQc5dWdybgZ1wp31WR2FeLAnPBXxwQ%40mail.gmail.com.

[Myron Marston]

First off, I like the required_keys as the name for the option.

As for module attribute vs keyword option: I favor the keyword option approach for the same reasons that Saša outlined. The module attribute option uses an imperative, global state approach that feels like the equivalent of this:

String.split_options(parts: 3)
String.split("1,2,3,4", ",")

…and yet I expect everyone is glad that the Elixir API instead puts parts as a keyword option on the split call:

String.split("1,2,3,4", ",", parts: 3)

If we favor options being passed as explicit arguments in general, it’s not clear to me it’s better to use the imperative global state approach here.

All that said: I understand the composability argument for the module attribute (although, that argument could also apply to anytime a new option is being introduced to any API — and yet we would normally favor passing new options as an explicit argument) and will be glad to see this feature in whatever form it takes.

Myron

​

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/rW8c3_xSVCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/78274dc5-4099-4036-b920-c51214ad2e66%40googlegroups.com.

[José Valim]

The module attribute option uses an imperative, global state approach that feels like the equivalent of this

Imperative but not global. It can be persisted later on and made global as read-only but during definition it can only be effectively changed by the current module or entities called in the same process by the current module using the module name as reference.

To make things more precise, the code example you pasted:

String.split_options(parts: 3)
String.split("1,2,3,4", ",")

As a module attribute would be written rather as:

String.split_options(__MODULE__, parts: 3)
String.split(__MODULE__, "1,2,3,4", ",")

I.e. you cannot change it globally, you can only change it for the current module in the process that is defining the module itself.

It is imperative though as it relies on side-effects. Although almost everything we do at the module body is a side-effect: defining functions, defining attributes, the module definition itself, etc.

[Saša Jurić]

On 25 May 2016, at 22:06, Michał Muskała <mic...@muskala.eu> wrote:

2016-05-25 19:30 GMT+02:00 Saša Jurić <sasa....@gmail.com>:

defstruct which takes parameters is composable. Any macro that uses defstruct can take the same parameters (or some subset), and pass them to defstruct, and it will work. It can also take some different parameters, and derive from them the ones that needs to be passed to defstruct.

The problem here is that library needs to explicitly support it, so
you can't use a new feature with old code without changing it.
Another, even bigger issue, is backward compatibility in libraries.
If we wanted to support that in Ecto, while supporting both Elixir 1.2
without this feature and 1.3 with this feature is to check elixir
version and add multiple ifs, in various places. This is not bearable
and I suspect most libraries will either break backwards compatibility
or simply ignore this feature.

Perhaps I’m missing something, but here’s how I see it.

As I said, @derive could be soft deprecated, meaning it still works in 1.x but it’s not documented. That means that all existing code works as before. Ecto which wants to support both Elixirs shouldn’t rely on the features from the newer version anyway.

If Ecto wants to work with the new require feature, it has to require Elixir >= 1.3. The require would be passed as an option.

If you don’t want to bump Ecto major, you could do the same trick for derive: soft deprecate @derive, and suggest moving to :derive.

To avoid confusion, defstruct could error if both @derive and :derive are provided.

Am I missing something?

Also, I’d like to touch on “most libraries will either break backwards compatibility or simply ignore this feature”. I’m not sure how many libraries are we talking about? There’s only one 3rd party lib mentioned in this thread. Another that comes to mind is my own FSM, which supports @derive by pure chance rather than my explicit choice :-) 

There might be others, but do you think it’s a frequent case to build custom macros on top of structs?

[Dmitry Belyaev]

Definition of a function in a module is side effect free as the order you define functions does not matter. The definition is atomic.

With module attributes we have more than one definitions: some amount of overwrites, and some amount of usages. What if after defstruct we have some other definition that also may be modified by an attribute with such name?

In case of exceptions and Ecto macros that could benefit from this or more additional arguments to defstruct. Why not to take all the unexpected additional arguments (for example in defexception) and pass them to defstruct explicitly? And defstruct will decide what to do with them.

As attributes are not parameters how to decide where they might be documented? At the function they are applied to? How about the example with defexception, would it mention that it is supported? If they are documented on its own then how to make sure they affect different definitions the same way?

In case of explicit arguments the answers to those questions are known.

--
Best wishes,
Dmitry Belyaev

[José Valim]

Definition of a function in a module is side effect free as the order you define functions does not matter. The definition is atomic.

There are many ways to observe the side-effects of defining a function in a module. There are many functions in Module that makes the side-effect observable as well as constructs like defoverridable that makes the function definition order dependent. The order between macros and function definitions also matter if you want to use the macro in the same module that defines them.

In case of exceptions and Ecto macros that could benefit from this or more additional arguments to defstruct. Why not to take all the unexpected additional arguments (for example in defexception) and pass them to defstruct explicitly? And defstruct will decide what to do with them.

Keep in mind this go directly against some other feature proposed for defstruct which was for it to explicitly check the options passed to it and raise for any unknown option. If we simply accept anything in defstruct, if you make a typo when giving an option, it will go undetected (as a module attribute would).

As attributes are not parameters how to decide where they might be documented?

You can't document an attribute, so it needs to be documented in every construct that uses it, in this case, defstruct, defexception and so on. So it is not really different from options in this aspect.

[Parker Selbert]

One I started thinking of defstruct as a macro that modifies the module itself it made more sense to use a module attribute.

At this point my primary concern is discoverability and documentation. Module attributes live in their own section of the docs, until you go seeking them out their interaction with other functions or macros is completely hidden. I would expect to see details of @enforce/@require in the docs for defstruct.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4LxVGykfbLz7mq%3DfiY06pfeKWP21h-RBwvgKhBFBh_6%3DQ%40mail.gmail.com.

[Onorio Catenacci]

I don't know if other practitioners of FP will agree with me or not but I'll say this: side-effects are not evil.  _Uncontrolled and accidental_ side-effects are evil.  If there were no side-effects anywhere we couldn't get anything accomplished.  Writing to the console is a side-effect.  The problem isn't side-effects; the problem is unintentional side-effects.

Sorry I know this is a tangent to this discussion but I sometimes start to think that there are people who simply assume any side-effect whatever is a bad thing and, as software developers, we should know better.

The question is not whether side-effects can be avoided or if they're bad or good; the question is are the side-effects explicit so that anyone developing with this mechanism will know there will be side-effects.

--

Onorio

 

--

You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/rW8c3_xSVCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/FE38208F-B0D5-4BBA-A4D2-1A06D68733CE%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Dmitry Belyaev]

If you want composability how about:

require_keys [:a, :b],
in: defstruct [:a, :b, :c]

require_keys [:message],
in: defexception ...

Also repeating mandatory keys with some default values seems quite annoying.

[Dmitry Belyaev]

On 27 May 2016 12:44:49 AM AEST, "José Valim" <jose....@plataformatec.com.br> wrote:

>> additional arguments to defstruct. Why not to take all the unexpected
>> additional arguments (for example in defexception) and pass them to
>> defstruct explicitly? And defstruct will decide what to do with them.
>>
>
>Keep in mind this go directly against some other feature proposed for
>defstruct which was for it to explicitly check the options passed to it
>and
>raise for any unknown option. If we simply accept anything in
>defstruct, if
>you make a typo when giving an option, it will go undetected (as a
>module
>attribute would).
>

I didn't say defstruct would silently ignore the unknown arguments. It should reject such definition.

>As attributes are not parameters how to decide where they might be
>> documented?
>>
>
>You can't document an attribute, so it needs to be documented in every
>construct that uses it, in this case, defstruct, defexception and so
>on. So
>it is not really different from options in this aspect.

Exactly. If the behaviour depends on the point of application and is documented there along with normal arguments, it's not any different than an argument. Why not to make it an argument then?