Suggestion: Map.deep_merge

219 views
Skip to first unread message

Tobias Pfeiffer

unread,
Sep 6, 2016, 11:20:23 AM9/6/16
to elixir-l...@googlegroups.com
Hi everyone,

in Elixir we work a lot with maps (or at least I do :D). And we know and
love Map.merge. However given a map like %{a: %{b: 1}} and we want to
merge it with a map like %{a: %{c: 2}} then the :a key will be totally
overridden with whatever is supplied in the other map.

However, for instance for configuration, what we might want is the
result to be %{a: %{b: 1, c: 2}} - e.g. if the value is a map on both
sides recursively merge that as well.

It is rather simple to implement in elixir with Map.merge/3 [1] but I
think it'd still be helpful to have it in elixir core (otherwise I'll
make a mini hex package :D). E.g. it is useful and it'll keep people
from wondering about it or writing their own a little flawed
implementations.

So, what do you think - Map.deep_merge in elixir-core - yay/nay? I'm
happy to draft a PR if it gets the sign of approval.

Tobi


[1] http://stackoverflow.com/a/38865647/1810896
--
http://www.pragtob.info/

OvermindDL1

unread,
Sep 6, 2016, 2:41:22 PM9/6/16
to elixir-lang-core
I had made my own a few times, though rare, it could be useful.  Unsure about it in core but in a library definitely.

Brian Cardarella

unread,
Sep 7, 2016, 5:02:34 AM9/7/16
to elixir-lang-core
This is assuming that all nested values are maps all the way down. What happens in the event of a Keyword list for example? Or a tuple? If the use-case is limiting itself to just maps of maps this works. But if the wider use-case should be supported it may make sense to consider a Kernel.deep_merge that has indifferent access support for the values to be merged.

Bruce Tate

unread,
Sep 9, 2016, 11:22:01 AM9/9/16
to elixir-l...@googlegroups.com
+1 from me. We have needed it. 

-bt

--
http://www.pragtob.info/

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/5eb1a5d2-01b3-9a27-6ff9-36836959cf36%40gmail.com.
For more options, visit https://groups.google.com/d/optout.



--
Bruce Tate
President, RapidRed, LLC
Phone: 512.772.4312
Fax: 512 857-0415

Author of Seven Languages in Seven Weeks, Deploying Rails Applications, From Java to Ruby, Rails: Up and Running, Beyond Java, 6 others.

OvermindDL1

unread,
Sep 9, 2016, 11:25:05 AM9/9/16
to elixir-lang-core
Just as I recently posted that I did not have much use for it, I could have just used it and ended up building one...

/me sighs

But still, I'm weary about how to implement it generically, needs to be worked on.

Also, this should be on https://elixirforum.com/ as it will get a lot more visibility and communication there.  These mailing lists are mostly dead since https://elixirforum.com/ took over.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

José Valim

unread,
Sep 9, 2016, 11:32:23 AM9/9/16
to elixir-l...@googlegroups.com
Also, this should be on https://elixirforum.com/ as it will get a lot more visibility and communication there.  These mailing lists are mostly dead since https://elixirforum.com/ took over.

This is the elixir-lang-core mailing list. This mailing **has not** been replaced by elixirforum. This list is meant to more "low profile" as it is meant for language proposals and discussions. It is not a general discussion place. Therefore this discussion is happening in the proper place. There is no guarantee me or any of the other members of the Elixir Core Team will see feature discussions in Elixir Forum.


José Valim
Skype: jv.ptec
Founder and Director of R&D

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/12c78eac-97a0-4a69-86fd-5416928180e6%40googlegroups.com.
Message has been deleted

OvermindDL1

unread,
Sep 9, 2016, 12:00:33 PM9/9/16
to elixir-lang-core
On Friday, September 9, 2016 at 9:25:05 AM UTC-6, OvermindDL1 wrote:
But still, I'm weary about how to implement it generically, needs to be worked on.

I looked through my big project to see a couple of areas where I deep-merge, in all cases I needed a resolver function (usually appending lists together or sorting something that was not a map or so forth).  Thus if a deep_merge is made, even in a library, it needs a version that can take a resolver function for non-map values.

Tobias Pfeiffer

unread,
Sep 9, 2016, 12:51:52 PM9/9/16
to elixir-l...@googlegroups.com, Brian Cardarella
I was thinking about this as well (well keyword lists, tuples didn't come to my mind and wouldn't know of a merge method there).

Having both Map.deep_merge and Kernel.deep_merge seems sensible to me (or maybe just the Kernel version?), although Kernel.deep_merge seems like more work and more potential edge cases popping up (due to its more general nature). With Map.deep_merge I'm also reasonably confident that it won't require a lot of maintenance as long as no one removes Map.merge/3 or is_map which is why I see it as an addition with little impact on future maintenance. I might be too naive though ;)
> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elixir-lang-co...@googlegroups.com
> <mailto:elixir-lang-co...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/c451058c-6a49-481e-9cb8-1541ce1b77a7%40googlegroups.com
> <https://groups.google.com/d/msgid/elixir-lang-core/c451058c-6a49-481e-9cb8-1541ce1b77a7%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
http://www.pragtob.info/

Renan Ranelli

unread,
Sep 9, 2016, 12:51:53 PM9/9/16
to elixir-l...@googlegroups.com
I have wrote a version for this in our project. It behaves exactly like Map.merge/2 but concating lists and merging inner maps:

```
  # @spec merge_cat(map, map) :: map
  def merge_cat(e, acc), do: Map.merge(acc, e, &combiner/3)
  defp combiner(_, l1, l2) when is_list(l1) and is_list(l2), do: l1 ++ l2
  defp combiner(_, m1, m2) when is_map(m1) and is_map(m2), do: merge_cat(m1, m2)
  defp combiner(_, _, v2), do: v2
```


--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/c451058c-6a49-481e-9cb8-1541ce1b77a7%40googlegroups.com.

Tobias Pfeiffer

unread,
Sep 10, 2016, 2:47:09 AM9/10/16
to elixir-l...@googlegroups.com
I think this is a good idea. For instance Renan's version (which I like)
concatenates lists which is a behaviour I'd really not want in my
current project. Question is also about keywords then though.

API might get to be more complex then. Maybe just Map.deep_merge and a
custom resolver function for every other value?

Tobi

> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elixir-lang-co...@googlegroups.com
> <mailto:elixir-lang-co...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/96b6870e-e868-40eb-9013-4cdfb669f391%40googlegroups.com
> <https://groups.google.com/d/msgid/elixir-lang-core/96b6870e-e868-40eb-9013-4cdfb669f391%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
http://www.pragtob.info/

Bruce Tate

unread,
Sep 13, 2016, 12:31:53 PM9/13/16
to elixir-l...@googlegroups.com
It seems like you could have a Map.deep_merge/2 which has an explicit behavior for conflicting keys (e.g. first wins or second wins), and a Map.deep_merge/3 which also accepts a function to merge individual keys. 

-bt

On Sat, Sep 10, 2016 at 1:47 AM, Tobias Pfeiffer <pra...@gmail.com> wrote:
On 09/09/2016 06:00 PM, OvermindDL1 wrote:
> On Friday, September 9, 2016 at 9:25:05 AM UTC-6, OvermindDL1 wrote:
> >
> I looked through my big project to see a couple of areas where I
> deep-merge, in all cases I needed a resolver function (usually appending
> lists together or sorting something that was not a map or so forth).
>  Thus if a deep_merge is made, even in a library, it needs a version
> that can take a resolver function for non-map values.

I think this is a good idea. For instance Renan's version (which I like)
concatenates lists which is a behaviour I'd really not want in my
current project. Question is also about keywords then though.

API might get to be more complex then. Maybe just Map.deep_merge and a
custom resolver function for every other value?

Tobi

> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/7458c350-bf3b-f88b-3214-7e928ea3c5c5%40gmail.com.

For more options, visit https://groups.google.com/d/optout.



--

Tobias Pfeiffer

unread,
Oct 19, 2016, 9:17:35 AM10/19/16
to elixir-l...@googlegroups.com
Hey y'all!

Sorry for the resurrection but due to circumstances I just got to finish
and clean up my implementation with docs et. al. now. I sent a PR to
elixir as most people here seemed to agree it was a good idea but the
discussion went more into the implementation details so I thought having
an implementation to discuss might be good:
https://github.com/elixir-lang/elixir/pull/5339

The PR implements Map.deep_merge/2 and Map.deep_merge/3 implementing
Bruce's idea (or well I believe so).

Cheers and thanks for all the feedback,
Tobi

On 09/12/2016 03:55 PM, Bruce Tate wrote:
> It seems like you could have a Map.deep_merge/2 which has an explicit
> behavior for conflicting keys (e.g. first wins or second wins), and a
> Map.deep_merge/3 which also accepts a function to merge individual keys.
>
> -bt
>
> On Sat, Sep 10, 2016 at 1:47 AM, Tobias Pfeiffer <pra...@gmail.com
> <mailto:pra...@gmail.com>> wrote:
>
> On 09/09/2016 06:00 PM, OvermindDL1 wrote:
> > On Friday, September 9, 2016 at 9:25:05 AM UTC-6, OvermindDL1 wrote:
> > >
> > I looked through my big project to see a couple of areas where I
> > deep-merge, in all cases I needed a resolver function (usually appending
> > lists together or sorting something that was not a map or so forth).
> > Thus if a deep_merge is made, even in a library, it needs a version
> > that can take a resolver function for non-map values.
>
> I think this is a good idea. For instance Renan's version (which I like)
> concatenates lists which is a behaviour I'd really not want in my
> current project. Question is also about keywords then though.
>
> API might get to be more complex then. Maybe just Map.deep_merge and a
> custom resolver function for every other value?
>
> Tobi
>
> > --
> > You received this message because you are subscribed to the Google
> > Groups "elixir-lang-core" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to elixir-lang-co...@googlegroups.com
> <mailto:elixir-lang-core%2Bunsu...@googlegroups.com>
> > <mailto:elixir-lang-co...@googlegroups.com
> <mailto:elixir-lang-core%2Bunsu...@googlegroups.com>>.
> <https://groups.google.com/d/msgid/elixir-lang-core/96b6870e-e868-40eb-9013-4cdfb669f391%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/elixir-lang-core/96b6870e-e868-40eb-9013-4cdfb669f391%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> > For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
>
> --
> http://www.pragtob.info/
>
> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to elixir-lang-co...@googlegroups.com
> <mailto:elixir-lang-core%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/7458c350-bf3b-f88b-3214-7e928ea3c5c5%40gmail.com
> <https://groups.google.com/d/msgid/elixir-lang-core/7458c350-bf3b-f88b-3214-7e928ea3c5c5%40gmail.com>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
>
>
> --
> Bruce Tate
> President, RapidRed, LLC
> Phone: 512.772.4312
> Fax: 512 857-0415
>
> Author of Seven Languages in Seven Weeks, Deploying Rails Applications,
> From Java to Ruby, Rails: Up and Running, Beyond Java, 6 others.
>
> --
> You received this message because you are subscribed to the Google
> Groups "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elixir-lang-co...@googlegroups.com
> <mailto:elixir-lang-co...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/CAMp4_igE40wDiRiPXtJGdVBCYEXXxVa%2Bs-%3D6hLv17fBZ%3D_%3DOug%40mail.gmail.com
> <https://groups.google.com/d/msgid/elixir-lang-core/CAMp4_igE40wDiRiPXtJGdVBCYEXXxVa%2Bs-%3D6hLv17fBZ%3D_%3DOug%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Tobias Pfeiffer

unread,
Nov 10, 2016, 2:53:27 PM11/10/16
to elixir-l...@googlegroups.com
One last resurrection, after the PR was rejected I started writing a
library for this functionality incorporating the feedback from the PR
discussion (or well so I believe). It merges maps and keyword lists and
has some other goodies, you can find it here:
https://github.com/PragTob/deep_merge

Tobi

José Valim

unread,
Nov 11, 2016, 4:51:28 AM11/11/16
to elixir-l...@googlegroups.com
Neat. You can also look into making the protocol derivable with a default implementation that only merges structs of the same kind. This way someone can easily get deep merging for structs by adding "@derive DeepMerge.Resolver".



José Valim
Skype: jv.ptec
Founder and Director of R&D


     To view this discussion on the web visit
     https://groups.google.com/d/msgid/elixir-lang-core/7458c350-bf3b-f88b-3214-7e928ea3c5c5%40gmail.com
     <https://groups.google.com/d/msgid/elixir-lang-core/7458c350-bf3b-f88b-3214-7e928ea3c5c5%40gmail.com>.
     For more options, visit https://groups.google.com/d/optout
     <https://groups.google.com/d/optout>.




--
Bruce Tate
President, RapidRed, LLC
Phone: 512.772.4312
Fax: 512 857-0415

Author of Seven Languages in Seven Weeks, Deploying Rails Applications,
 From Java to Ruby, Rails: Up and Running, Beyond Java, 6 others.


--
You received this message because you are subscribed to the Google
Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/e074c6c9-fac7-c394-7c82-d1090173cba6%40gmail.com.

Tobias Pfeiffer

unread,
Nov 11, 2016, 6:33:22 AM11/11/16
to elixir-l...@googlegroups.com
Great idea and... I didn't even know that was possible :D Still some learning to do ;) Thanks for the tip José - I'll open issue #1 :)
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KF60%3DfT5mW_mZzjmuReDCwGtVgxeZ--7_9wZmmADRY1g%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages