Tools for Manipulation Nested Data Structures

[Drew Olson]

When I spin up a new project in Elixir I almost always pull in https://github.com/jeremyjh/nested. This makes it super convenient to work with nested data structures.

Clojure has tools in the standard library for working with these types of structures (get-in, update-in, etc).

Is this something that could be moved into the standard library for Elixir? It seems generally useful.

Thoughts?

- Drew

[José Valim]

That's a very good question Drew!

We had discussions about nested data structures some time ago and the approach in nested was ruled out because of records. With records, simply doing put_in(opts, [:field, ...], value) would be ambiguous because for things like HashDict, you wouldn't know if :field is the record field or the value being accessed.

However, moving to maps and structs and getting rid of records eliminates this distinction. I actually expect user[:name] to work with most structs. So I believe that we can actually provide put_in and update_in functions without much problem.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Drew Olson]

Great news José. Another reason to be excited about moving to maps!

[Carsten Bormann]

On 15 Feb 2014, at 12:27, José Valim <jose....@plataformatec.com.br> wrote:

> moving to maps and structs and getting rid of records

Hmm, this choice of words makes me nervous.

Are you setting yourself up for a Python-3-type suicide event?

Elixir benefits a lot from the Erlang back catalog, and you can’t burn all bridges there.
At the speed at which the Erlang ship turns, there will be lots of records out there for a long time.
Also, by now, Elixir itself has amassed a codebase (8.6 million “lines changed on github"*), much of it counting on records.
(And there may be other good reasons for handing around tuples that look a lot like records.)

This is not a tirade against change. Elixir is still a fast-evolving language, and that is good.
You just have to help the community to evolve with the language.
Requiring everyone to change // into \\ is a small mechanical operation, aided by good compiler feedback.
You are unlikely to lose anyone in that transition.

Requiring everyone to translate their record-based implementations to map-based ones amounts to a rewrite.
People will weigh the need for this, and the timing for it, slowing upgrades.
Data point: Having required pervasive changes to move from Python 2 to Python 3 is what has been making Python 3 utterly irrelevant in large parts of the Python world… (This is slowly getting fixed there by adding more compatibility, but you can learn from that mistake.)
There should at least be an “import OldRecords” (with appropriate lexical scope) for a long time...
(You may already be meaning to do that, but you also have to say it.)

Grüße, Carsten

*) http://langpop.corger.nl/, according to which Elixir already is about 1/8th of the Erlang universe…

[Alex Rønne Petersen]

Elixir has not reached a point where the language is stable. It is not
1.x. That is why the maps change is being made and why this isn't
really comparable to Python 2 -> 3.

Also, I see no reason we can't consume Erlang records just because we
don't have native records. We just translate them to maps instead.

On Sat, Feb 15, 2014 at 5:20 PM, Carsten Bormann <cabo...@gmail.com> wrote:
> On 15 Feb 2014, at 12:27, José Valim <jose....@plataformatec.com.br> wrote:
>
>> moving to maps and structs and getting rid of records
>
> Hmm, this choice of words makes me nervous.
>
> Are you setting yourself up for a Python-3-type suicide event?
>
> Elixir benefits a lot from the Erlang back catalog, and you can't burn all bridges there.
> At the speed at which the Erlang ship turns, there will be lots of records out there for a long time.
> Also, by now, Elixir itself has amassed a codebase (8.6 million "lines changed on github"*), much of it counting on records.
> (And there may be other good reasons for handing around tuples that look a lot like records.)
>
> This is not a tirade against change. Elixir is still a fast-evolving language, and that is good.
> You just have to help the community to evolve with the language.
> Requiring everyone to change // into \\ is a small mechanical operation, aided by good compiler feedback.
> You are unlikely to lose anyone in that transition.
>
> Requiring everyone to translate their record-based implementations to map-based ones amounts to a rewrite.
> People will weigh the need for this, and the timing for it, slowing upgrades.

> Data point: Having required pervasive changes to move from Python 2 to Python 3 is what has been making Python 3 utterly irrelevant in large parts of the Python world... (This is slowly getting fixed there by adding more compatibility, but you can learn from that mistake.)

> There should at least be an "import OldRecords" (with appropriate lexical scope) for a long time...
> (You may already be meaning to do that, but you also have to say it.)
>
> Grüße, Carsten
>

> *) http://langpop.corger.nl/, according to which Elixir already is about 1/8th of the Erlang universe...

[Kirill Romanov]

суббота, 15 февраля 2014 г., 20:53:46 UTC+4 пользователь Alex Rønne Petersen написал:

Elixir has not reached a point where the language is stable. It is not
1.x. That is why the maps change is being made and why this isn't
really comparable to Python 2 -> 3.

Also, I see no reason we can't consume Erlang records just because we
don't have native records. We just translate them to maps instead.

Change is going to be painful why not admit it? I'm just hoping it will be slow and we'll have erlang-17 (and maps compatible) release before current records implementation will go away. I have a lot of code using constructions like somerecord.somefield(new_value). Also there are tons of libraries and today it's easy to write code like:

    case :emysql.whatever(...) do

       :ok_packet[] -> ...

    ...

What will it look like after records are gone? {:ok_packet, ..., ... }?

[Carsten Bormann]

On 15 Feb 2014, at 17:53, Alex Rønne Petersen <al...@alexrp.com> wrote:

> Elixir has not reached a point where the language is stable.

Bad excuse.

No language has ever reached this point.

Once you build a community, you have a fiduciary responsibility to it.

Even when you announce that the ride will be bumpy, you still have to care about the people who are on-board.

> It is not 1.x.

Bad excuse part 2. Really bad attitude.

That kind of thinking can be used to justify any kind of irresponsibility.

> That is why the maps change is being made and why this isn't
> really comparable to Python 2 -> 3.

It is very comparable. The parameters are different: There is a much larger expectation of stability from Python than from Elixir.
But that doesn’t mean all existing effort the community has ever put into Elixir code is disposable and worth nothing.

> Also, I see no reason we can't consume Erlang records just because we
> don't have native records. We just translate them to maps instead.

Great, now that sounds like a part of a transition strategy, and that’s all I’m asking for.
(Nobody wants this to slow down to Java speeds.)
Just don’t forget about those 8.6 million lines of Elixir on github, and don’t let your responsibility be blinded by “it’s not 1.x".

Grüße, Carsten

[Dave Thomas]

We are all pioneers here. We are all working to create something new, and something great.

Some, like José work by creating. Most of us work by consuming, by giving feedback, and by putting effort in when the decisions made involve change.

Let's say that differently. There are no "users" here. Everyone is a contributor.

José decided to migrate to maps. It was a decision that has taken a year, and a lot of discussion. I know from conversations with him just how much energy he's put into this. 

Folks on this list have also contributed with ideas, feedback, prototypes. And now we're contributing by changing the existing codebase to move to the new features. It's all part of the contract.

That is simply life on the frontier.  

Dave

[Alex Rønne Petersen]

On Sat, Feb 15, 2014 at 6:18 PM, Carsten Bormann <cabo...@gmail.com> wrote:
> On 15 Feb 2014, at 17:53, Alex Rønne Petersen <al...@alexrp.com> wrote:
>
>> Elixir has not reached a point where the language is stable.
>
> Bad excuse.
>
> No language has ever reached this point.

It is not an excuse. Elixir makes no guarantee about language
stability in 0.x as per Semantic Versioning 2.0.0. It *will* once we
go past 1.x, but you cannot reasonably expect stability in a 0.x
product. There is a clear roadmap to 1.x:
https://groups.google.com/forum/#!topic/elixir-lang-core/l84apId6eiA

Of course there'd be no sense in just changing things left and right
just because we can. Common sense applies.

>
> Once you build a community, you have a fiduciary responsibility to it.

Yes, absolutely. What I am pointing out is that while we can (and
should) make reasonable attempts to ease transitions, ultimately, you
cannot *expect* stability in 0.x. That's what I mean by this not being
comparable to Python 2 -> 3, because that's a past-1.x product where
stability is a requirement for continued use and adoption.

>
> Even when you announce that the ride will be bumpy, you still have to care about the people who are on-board.

Easing the transition is of course perfectly fine and reasonable.

>
>> It is not 1.x.
>
> Bad excuse part 2. Really bad attitude.

I'm not sure I got the right message across. As mentioned above, I
merely wanted to point out how stability works with regards to our
versioning system. I am not at all against easing the transition.

>
> That kind of thinking can be used to justify any kind of irresponsibility.

Yes, but of course we should employ common sense.

>
>> That is why the maps change is being made and why this isn't
>> really comparable to Python 2 -> 3.
>
> It is very comparable. The parameters are different: There is a much larger expectation of stability from Python than from Elixir.
> But that doesn't mean all existing effort the community has ever put into Elixir code is disposable and worth nothing.
>
>> Also, I see no reason we can't consume Erlang records just because we
>> don't have native records. We just translate them to maps instead.
>
> Great, now that sounds like a part of a transition strategy, and that's all I'm asking for.
> (Nobody wants this to slow down to Java speeds.)
> Just don't forget about those 8.6 million lines of Elixir on github, and don't let your responsibility be blinded by "it's not 1.x".
>
> Grüße, Carsten
>

[Dave Thomas]

> Once you build a community, you have a fiduciary responsibility to it.

Yes, absolutely.

No, absolutely.

Once you join a community, you have a duty to participate for the common good. That’s what it means to be part of a community.

Otherwise you're just a consumer. And if someone is just a consumer, what benefit does the community as a whole get from having that person around?

Dave

[José Valim]

I am afraid this discussion is being triggered by uncertainty. So I will repeat what has been said in the maps proposal and in the maps threads:

1. Elixir v0.13 will be released with support for both records and maps. The plan is that v0.13 will be backwards compatible with the latest Elixir v0.12.x;

2. We will have a period where both records and maps will be supported to aid the migration. This is the time for the community to start migrating their records to maps/structs. Then we will slowly start converting the usage of records in Elixir code to maps/structs;

3. We will still support Erlang records, i.e. the current functionality known as defrecordp will remain in the language. But their main role will be Erlang integration.

In other words, what will be removed are Elixir records (which are backed by modules).

 

> It is not 1.x.

Now that we have re-outlined the upgrade plan. Let's talk about this.

1.x is not a bad excuse nor a bad attitude. It is a contract. Everyone who uses Elixir must be aware that, before we reach 1.0, backwards incompatible changes **will** happen to the language. If that does not please someone, please wait ~6 months before writing Elixir code (the estimate for Elixir 1.0).

I am not trying to be rude. I am just pointing out that if someone is expecting stability, now is not the time. I also don't want to sound ungrateful, I am very grateful for everyone who has contributed and written Elixir code, which helped the language grow.

Once we reach 1.0, I am doing a commitment of properly following SemVer. I am not doing any commitment *before* that. Regardless of no commitment, I try to be really thoughtful regarding the language changes. There is plenty of discussion before hand, to avoid code churn, and I always deprecate features before removing, except on cases it is not possible or extremely hard to do.

  

Just don't forget about those 8.6 million lines of Elixir on github, and don't let your responsibility be blinded by "it's not 1.x".

This is actually really cool. I really had no idea there were 8.6 million lines of Elixir code change on Github!

[Carsten Bormann]

On 15 Feb 2014, at 18:45, José Valim <jose....@plataformatec.com.br> wrote:

> This is actually really cool. I really had no idea there were 8.6 million lines of Elixir code change on Github!

Well, the number is inflated by all the // changing into the \\… :-)

Seriously, this is a sizable community, and you (and everybody else that has contributed) can be proud of having built it.

Grüße, Carsten

[José Valim]

> This is actually really cool. I really had no idea there were 8.6 million lines of Elixir code change on Github!

Well, the number is inflated by all the // changing into the \\... :-)

We can also add to the count the changes from %r/regex/ to ~r/regex/ ;)

 

Seriously, this is a sizable community, and you (and everybody else that has contributed) can be proud of having built it.

:D