[Proposal] Surface Erlang's maps:map function

[jonar...@gmail.com]

A common task is to iterate over a map performing some operation thereby producing a new map. There are some ways to do this in Elixir presently, the simplest probably being for...into:

for {key, val} <- map, into: %{} do
  {key, val * 2}

end

Enum.reduce/3 is also an option. However, Erlang provides a simple function to replace the values of a map with maps:map function:

maps:map(fun(Key, Val) -> 2 * Val end, Map)

I think an equivalent of this in Elixir would be very useful either as Map.map/2 or Map.transform_values/2 like so:

Map.transform_values(map, fn {_key, val} -> val * 2 end)

I'm interested to hear if the community considers this a worthwhile addition!

[Jason Axelson]

Unless I'm missing something I think this is directly handled by `Map.new/2` since it can operate on any enumerable. So:

for {key, val} <- map, into: %{} do

  {key, val * 2}

end

is equivalent to:

Map.new(map, fn {key, val} -> {key, val * 2})

Which I think is clear and succinct (although I do think that `Map.new/2` isn't as widely known as it could be). The implementation is based on :maps.from_list/1:
https://github.com/elixir-lang/elixir/blob/v1.11.3/lib/elixir/lib/map.ex#L172

-Jason

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/d843c44b-e658-4d71-bb66-00c1e0a21ef7n%40googlegroups.com.

[Austin Ziegler]

It’s not implemented with maps:map/2, but Map.new/2 should do the trick.

`Map.new(map, fn {k, v} -> {k, v * 2} end)`

That said, `maps:map/2` is available:

`:maps.map(fn _k, v -> v * 2 end, %{x: 1, y: 2, z: 3})`

It might be worth exploring whether `Map.map` would be useful/efficient enough to add for piping purposes.

-a

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/d843c44b-e658-4d71-bb66-00c1e0a21ef7n%40googlegroups.com.

--

Austin Ziegler • halos...@gmail.com • aus...@halostatue.ca
http://www.halostatue.ca/ • http://twitter.com/halostatue

[jonar...@gmail.com]

This usage of Map.new/2 is news to me (pun somewhat intended) and pretty neat! However, I think that the naming alone may make it less discoverable and a bit harder to grok when first encountered. There also might be some performance gains from using Erlang's maps:map for this task instead of maps:from_list but I'm not sure. I'll do some profiling and find out.

[eksperimental]

Great finding.
Regardless the benchmarks, Map.new/2 should be optimized for maps using
:maps.map/2, because as of now we are doing: map |> Enum.to_list() |>
reduce list |> :lists.reverse() |> :maps.from_list()

On Tue, 12 Jan 2021 14:09:26 -0800 (PST)
"jonar...@gmail.com" <jonar...@gmail.com> wrote:

> This usage of Map.new/2 is news to me (pun somewhat intended) and
> pretty neat! However, I think that the naming alone may make it less
> discoverable and a bit harder to grok when first encountered. There

> also *might* be some performance gains from using Erlang's maps:map

> >> <https://groups.google.com/d/msgid/elixir-lang-core/d843c44b-e658-4d71-bb66-00c1e0a21ef7n%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> .

[eksperimental]

In fact, Map.new/2 can be simplified, (it passes all tests)

@spec new(Enumerable.t(), (term -> {key, value})) :: map
def new(enumerable, transform) when is_function(transform, 1) do
Enum.reduce(enumerable, %{}, fn term, map ->
{key, value} = transform.(term)
Map.put(map, key, value)
end)
end


It would be interesting to see how that benchmarks against :maps.map/2
when enumerable is a map.

[Austin Ziegler]

Building `Map.new/2` on `maps:map/2` would be incompatible, because the transformation function differs in arity (`/1` for `Map.new/2` and `/2` for `maps:map/2`).

It would be possible to build `Map.new/2` such that it can tell the difference between a `/1` or a `/2` when the first function is itself a map or struct…but I’m not sure that would be an improvement, and it would lead to potentially confusing documentation on when a `/2` could be passed to `Map.new/2` and when a `/1` could be passed to `Map.new/2`.

If something is to be done, I believe that the original proposal, surfacing `maps:map/2` as `Map.map/2` with the arguments flipped for pipeline use _might_ be the best choice. At the same time, I’m not _entirely_ sure that would be useful, as most pipelines transform maps into lists and `maps:map/2` borks on a list:

```elixir

iex(1)> :maps.map(fn k, v -> v * 2 end, [{:a, 1}, {:b, 2}])

warning: variable "k" is unused (if the variable is not meant to be used, prefix it with an underscore)

  iex:1

** (BadMapError) expected a map, got: [a: 1, b: 2]

    (stdlib 3.12.1) maps.erl:247: :maps.map(#Function<13.126501267/2 in :erl_eval.expr/5>, [a: 1, b: 2])

```

At this point, even though I am happy to have discovered `maps:map/2` from this discussion, that this would _not_ improve the usability of Elixir on map transformation.

A different question: how can we make these rich Erlang functions much more visible to Elixir users like myself? I don’t care that the arguments are “backwards” from the pipeline, because `maps:map/2` is _incredibly_ useful and will improve some code that I have in production. 

-a

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/5fff9633.1c69fb81.eec7e.37f2SMTPIN_ADDED_MISSING%40gmr-mx.google.com.

[José Valim]

Eksperimental, it is faster to build a list and call maps:from_list/1 than build the map as we go forward.

Also, please note maps:map/2 is implemented using maps:iterator/1 and maps:from_list/1: https://github.com/erlang/otp/blob/master/lib/stdlib/src/maps.erl#L374-L379 - so we can implement our own maps:map/2 that keeps our function arity.

All that said, if we want to make things faster, then we should evaluate changing Enumerable.Map to use iterators. But we would need extensive benchmarks to check if it is worth it.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAJ4ekQvW-UNQC3pORyb-D4hGCTPvupkT%2Bp53KSkTj8bknj%3Dqgg%40mail.gmail.com.

[Michał Muskała]

I remember benchmarking the iterators back when they were first introduced. My conclusion was that because they yield {key, value, next} tuples, while all elixir APIs expect {key, value} tuples, the packing & unpacking of those tuples becomes prohibitively expensive. This might no longer be an issue, so it could use some fresh benchmark, though I would be very surprised if things improved in any significant way.

 

Michał.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4LKSMDNM%3DbvXg68tpgQzqMxPEBaVzf96dishNuhutcCsg%40mail.gmail.com.

[eksperimental]

Thank you José and Michal for your input.
I will write a benchmark then and see if there is room for improvement.

> > https://groups.google.com/d/msgid/elixir-lang-core/CAJ4ekQvW-UNQC3pORyb-D4hGCTPvupkT%2Bp53KSkTj8bknj%3Dqgg%40mail.gmail.com
> > <https://groups.google.com/d/msgid/elixir-lang-core/CAJ4ekQvW-UNQC3pORyb-D4hGCTPvupkT%2Bp53KSkTj8bknj%3Dqgg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> > .
> >
>

[eksperimental]

The arity issue can be dealt internally.

def new(map, transform) when is_map(map) and is_function(transform, 1)
do
:maps.map( fn key, val ->
{_new_key, new_value} = transform.({key, val})
new_value
end,
:maps.iterator(map)
)
end

[José Valim]

But that will make things slower, which is counter to the point of using :maps.map (inside Map.new) in the first place. :)

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/6000506f.1c69fb81.6dc1c.59d9SMTPIN_ADDED_MISSING%40gmr-mx.google.com.

[jonar...@gmail.com]

As promised, I ran some benchmarks comparing maps:map, Map.new, and for, and was actually pretty surprised by the outcome:

Name               ips        average  deviation         median         99th %
Map.new          72.78       13.74 ms    ±21.75%       13.64 ms       24.06 ms
maps:map         62.72       15.94 ms     ±5.54%       16.22 ms       18.57 ms
for              55.97       17.87 ms    ±13.60%       17.45 ms       27.70 ms

Comparison:
Map.new          72.78
maps:map         62.72 - 1.16x slower +2.20 ms
for              55.97 - 1.30x slower +4.13 ms

You can find the code for the benchmark here: https://gist.github.com/J3RN/51db9b64e51e83276a7bdb17bc720a92

I ran this benchmark with several different sizes of maps, and the results were consistent from 10 keys to 100,000 keys.

Where do we go from here? Given that Elixir's up-front transformation of a Map into a List proved faster than Erlang's iterators, I struggle to imagine how we could write a faster version than what we already have with Map.new. My original goal was to add a function to Elixir that would allow developers to do transformations on maps in a pipeline, and it appears that Map.new fits that niche.

However, Map.new but lacks discoverability (IMO) simply due to its name. When encountering the need for this function, I searched for function names such as Map.map, Map.transform, etc, and didn't find what I was looking for. Would a simple alias of, say, Map.transform to Map.new be a worthwhile solution to this problem?