capture &2 cannot be defined without &1

[Paul Butcher]

I just tried to merge two string -> int dictionaries, where I wanted to add the integers together whenever the same key appeared in both dictionaries. Here's what I tried first:

Dict.merge(totals, counts, &(&2 + &3))

This gives me the error:

capture &2 cannot be defined without &1

Of course, I can fix this easily enough by writing:

Dict.merge(totals, counts, fn(_k, v1, v2) -> v1 + v2 end)

But this seems rather more verbose than necessary - I'd be interested to understand the reason why the shorter version is disallowed?

Out of interest, the equivalent function in Clojure takes a binary function:

clojure.core/merge-with

([f & maps])

  Returns a map that consists of the rest of the maps conj-ed onto

  the first.  If a key occurs in more than one map, the mapping(s)

  from the latter (left-to-right) will be combined with the mapping in

  the result by calling (f val-in-result val-in-latter).

I'd be interested to know of use-cases where the ternary version is useful (i.e. when the result of the merge depends on the key as well as the two values) - I can't immediately think of any.

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

[Eric Meadows-Jönsson]

The reasoning is that there is no way to create an ampersand function with unused last arguments, example: fn x, y, _ -> x * y end, so there should be no way to create an ampersand function with unused arguments at all, for consistency.

Why we don't allow the shorter version is because we generally don't overload functions based on the arity of the given function in Elixir. That could be discussed though.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Eric Meadows-Jönsson

[TR NS]

On Sunday, September 1, 2013 12:52:13 PM UTC-4, Paul Butcher wrote:

I just tried to merge two string -> int dictionaries, where I wanted to add the integers together whenever the same key appeared in both dictionaries. Here's what I tried first:

Dict.merge(totals, counts, &(&2 + &3))

This gives me the error:

capture &2 cannot be defined without &1

That surprises me. What's the point of enumerating them then? ... Ok, it lets us change the order of application, right? So since captures support partial application. Why would that have to be limited to trailing arguments?

[Peter Minten]

It's not just limited that way. You also can't do
`&some_unary_func(some_param)` (i.e. you need at least one placeholder).

[Dave Martin]

To ask a related question, why is "&(...)" as a substitute for "fn -> ... end" (no arguments) disallowed?

[Eric Meadows-Jönsson]

We have &Enum.map/2 which expands to fn arg1, arg2 -> Enum.map(arg1, arg2) end. So if we allowed &(...) with no arguments &Enum.map/2 would become ambiguous. Is it the previous expansion or (fn -> Enum.map end) / 2

​

On Thu, Jun 26, 2014 at 11:21 PM, Dave Martin <xx...@dave.to> wrote:

To ask a related question, why is "&(...)" as a substitute for "fn -> ... end" (no arguments) disallowed?

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Eric Meadows-Jönsson

[Dave Martin]

Since arity isn't optional, I don't see where the ambiguity comes in?

iex(1)> &Enum.map
** (CompileError) iex:1: invalid args for &, expected an expression in the format of &Mod.fun/arity, &local/arity or a capture containing at least one argument as &1, got: Enum.map()
    (elixir) src/elixir_exp.erl:215: :elixir_exp.expand/2
iex(1)> &Enum.map/0
&Enum.map/0
iex(2)>

Even so, it appears & can capture arity/0 functions, its just unwilling to generate anonymous ones.

(or capture explicit arity/0 ones):
iex(12)> &Enum.map()
** (CompileError) iex:12: invalid args for &, expected an expression in the format of &Mod.fun/arity, &local/arity or a capture containing at least one argument as &1, got: Enum.map()
    (elixir) src/elixir_exp.erl:215: :elixir_exp.expand/2


Incidentally, I find the following contortion interesting (in that it isn't an error):

iex(6)> &(Enum.map) / 2
&Enum.map/2
iex(7)>

And that this isn't legal (yes, I see this is an erlang error):

iex(8)> func/2 = &Enum.map/2 (func should only be bound to or match an arity2 function)
** (ErlangError) erlang error: :illegal_pattern

[Eric Meadows-Jönsson]

Incidentally, I find the following contortion interesting (in that it isn’t an error):

iex(6)> &(Enum.map) / 2
&Enum.map/2

This is the very reason that we have an ambiguity. The parenthesis are optional for &, it is just an ordinary operator that happens to be a macro that expands to an anonymous function. & is not parsed as a function call. Let’s look at the AST for the expressions:

iex(1)> quote do: &Enum.map/2
{:&, [],
 [{:/, [],
   [{{:., [], [{:__aliases__, [], [:Enum]}, :map]}, [], []}, 2]}]}

iex(2)> quote do: &(Enum.map)/2
{:&, [],
 [{:/, [],
   [{{:., [], [{:__aliases__, [], [:Enum]}, :map]}, [], []}, 2]}]}

As you can see they are parsed the same way regardless of the parenthesis. Just like how 2+(2) is parsed to the same AST as 2+2.

The fact that we can capture functions of arity 0 has nothing to do with it, it is not a technical limitation. We chose to disallow it because the expressions would be ambiguous or confusing at best.

​

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Eric Meadows-Jönsson

[Dave Martin]

To play devil's advocate for a moment, it seems that for &, if expressions were required to be in parenthesis and captures to not be, and thus allowing an arity 0 &(...) would be preferable to saying that &   (   Enum.map)    /   (4) turns into &Enum.map/4, instead of a function that calls Enum.map/0 and gets divided by 4, which seems like the more natural interpretation.

I guess the trouble I'm having is that Enum.map/2 seems like one token to me, but (Enum.map)/2 seems like 3, and given the way that parses, & &1+3 should be equivalent to &(&1+3), but it isn't. I presume this is a precedence issue.

& &1+3 doesn't work
&(&1+3) does
g=&if true, do: IO.puts &1 works

iex(21)> f=&(&1).(5)
#Function<6.106461118/1 in :erl_eval.expr/5>
iex(22)> f.(&IO.puts/1)
5

After & &1+3 failing, I was expecting f after line 21 to be 5. Instead we get a function that always passes 5 to whatever function is its argument.

I'll confess that I find some of Elixir's syntax rules (or at least precedence of operations) confusing.
I would propose that the way & works should be, and before today, I thought it was (I have a proposal I like even better below):

&func/arity -> capture a function
&(expression) -> generate an anonymous function taking 0 to n arguments  (&1..&n)
&(func/arity) -> generate an anonymous function of arity 0, which returns func/arity
&(func/arity &1) -> syntax error
&(func)/number -> syntax error (functions can't be divided)
func/arity -> an individual token, the function with specified arity
func / 2 -> calls func, divides the result by 2.
&func -> syntax error, no arity specified
& &1 -> syntax error, &1 is not a function, no arity specified.

Or to put it another way: & is the capture operator, &( is the generate-an-anonymous-function operator. They're really two separate things. And I suspect the only reason & the capture operator is even needed at all is so that Enum.map/2 the function can be distinquished from Enum.map/2 the function call who's result is divided by 2, which could be fixed by indicating arity with some other symbol that wouldn't result in an ambiguity. Enum.map`2 or Enum.map~2 perhaps. Then & could just be the shortcut generate-an-anonymous-function-with-anon-arguments operator.

Enum.map [1,2,3], IO.Puts`1
1
2
3
[:ok, :ok, :ok]
Enum.map [1,2,3], &(&1+5)
[6, 7, 8]

Or, be like C (and some other languages) and take the largest possible match, thus always interpreting func/number as a function and func /number as a division, but I think just using a different symbol for arity would be better.

[Eric Meadows-Jönsson]

I think the mistake here is that you are seeing & as a syntax construct. Elixir has very few syntax constructs, & is in fact just an operator that follows all the general syntactic rules of Elixir, the only thing special about it would be its place in the precedence table (*). We cannot make special rules for & regarding parsing, that means no special whitespace handling or special handling of parenthesis.

In many other languages most things like if, def and defmodule are special keywords in the language, because of this they can make up special syntax rules for those keywords. In Elixir they are all macros and parsed just like all other function calls, i.e. no special rules.

Lets look at your proposals and why they might not work in a language such as Elixir.

&func/arity -> capture a function

Agree, this is how it works today.

&(expression) -> generate an anonymous function taking 0 to n arguments (&1..&n)

(func/arity) is an expression. Ambiguous?

&(func/arity) -> generate an anonymous function of arity 0, which returns func/arity

We can’t special handle parenthesis. (((expr))) should be parsed as expr and so (func/arity) should be parsed as func/arity.

&(func)/number -> syntax error (functions can’t be divided)

We should not add a syntax error here. For example someone creating a DSL may want to use this syntax, so why add this restriction?

func/arity -> an individual token, the function with specified arity

func / 2 -> calls func, divides the result by 2.

Lets not add special whitespace handing of the division operator. How do we determine (at compile time) that func is actually a function?

&func -> syntax error, no arity specified

Remember, no special parenthesis handling :). &(func) should parse as &func.

& &1 -> syntax error, &1 is not a function, no arity specified.

Again the parenthesis. & &1 should parse the same as & (&1) and &(&1).

I guess the trouble I’m having is that Enum.map/2 seems like one token to me, but (Enum.map)/2 seems like 3, and given the way that parses, & &1+3 should be equivalent to &(&1+3), but it isn’t. I presume this is a precedence issue.

AFAIK, I could very well be wrong, there is no way to construct such a precedence table.

As you see it is very hard to make the syntax work for perfectly for each use case while still keeping the language’s syntax rules general. The footnote below shows an example where it seems we were unable to keep it 100% general.

(*) Actually, this is not entirely true. There is some special handling to make & 1 / 2 parse as (&1) / 2, and & foo / 2 parse as &(foo/2). This, I agree, is a slight wart in the language. Should we have a different operator for binding a named function (&foo/2) and creating an anonymous function (&(&1 + &2))?

​

--

You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Eric Meadows-Jönsson

[José Valim]

(*) Actually, this is not entirely true. There is some special handling to make & 1 / 2 parse as (&1) / 2, and & foo / 2 parse as &(foo/2). This, I agree, is a slight wart in the language. Should we have a different operator for binding a named function (&foo/2) and creating an anonymous function (&(&1 + &2))?

This is a bug. As Dave said, & &1 + 1 could possibly be treated as &(&1 + 1). We should just make & have lower precedence than binary operators.

And we should *definitely* avoid whitespace sensitiveness. The few parts of the language that require it are often source of confusion.

[José Valim]

> I'll confess that I find some of Elixir's syntax rules (or at least precedence of operations) confusing.

Dave, if you can point out which other ones you find confusing, now would be the best time, as we plan to revisit the operator precedence table in the next release.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Eric Meadows-Jönsson]

To clarify the changes to & in the coming release. & is being changed to have a lower precedency, lower than all other operators except ->. We are keeping the special handling of &1, because all unary operators followed by a number binds strongly (+1, -1 and &1).

& &1 + 2 will parse as &((&1) + 2).

​

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Eric Meadows-Jönsson

[Dave Martin]

I had a very productive interlude with Eiffel at one point, and as a result, I've grown to love the ability to leave off parenthesis, and line-terminators, so with that in mind:

In addition to &, other things that have bothered me:

somefunc asdf |> someotherfunc gherg Which i think should turn into

somefunc(asdf) |> someotherfunc(gherg), but I think it actually turns into:

somefunc(asdf |> someotherfunc(gherg)) or possibly:

somefunc(asdf |> someotherfunc) gherg

I'm not entirely sure what's intuitive here for others, but for me its the first thing turning into the second. Especially if one is used to unix-style commands with pipes, where one typically does not put parentheses around things.

Another is

def func, do:

  needsParens(sadf)

def func do

  doesn'tNeedParens sadf

end

This usually only gets me when I'm converting multi-line functions to single-line, and forget to "fix" them.

I think I understand intellectually why this is the case.I was confused for a long time about why one version needs a comma before the do, and the other one doesn't. And I'm still not sure I completely understand. Is do/end its own macro, or are there two different versions of def? (If do/end is a macro, does it insert a comma in front of whatever it turns into?) And why doesn't one need a comma after do:? (I don't have a problem with this syntax, but it was confusing when I was first trying to learn the language). It makes it look like its doing special parsing, even if it isn't; which in turn makes it confusing when & can't do special parsing.

And I'm not sure why arity has to be indicated with what looks like a division (other than adoption from erlang), if its going to be the cause of some strange stuff.

[Eric Meadows-Jönsson]

somefunc asdf |> someotherfunc gherg Which i think should turn into
somefunc(asdf) |> someotherfunc(gherg), but I think it actually turns into
somefunc(asdf |> someotherfunc(gherg)) or possibly
somefunc(asdf |> someotherfunc) gherg

All binary operators work like this with the exception of ->. Without this behaviour we wouldn’t be able to have macros like assert. assert can inspect the given expression and with the knowledge from this give very good error messages. With your suggestion assert foo == bar would parse as (assert(foo)) == bar and make assert unable to see the ==.

Because of this it is recommended to always use parenthesis when piping. You should really always use parenthesis except for the cases where you know for sure there can be no ambiguity.

def func, do:
needsParens(sadf)

def func do
doesn’tNeedParens sadf
end

None of them need parenthesis, def func, do: needsParens sadf is perfectly valid syntax. If you would call needsParens with multiple arguments on the other hand, then you would need parenthesis. For example: def func, do: needsParens arg1, arg2. Since def is macro (not a keyword) and we are just calling the macro we cannot know if the last comma belongs to the def call or the needsParens call. Again, use your parenthesis :).

I think I understand intellectually why this is the case.I was confused for a long time about why one version needs a comma before the do, and the other one doesn’t. And I’m still not sure I completely understand. Is do/end its own macro, or are there two different versions of def? (If do/end is a macro, does it insert a comma in front of whatever it turns into?) And why doesn’t one need a comma after do:? (I don’t have a problem with this syntax, but it was confusing when I was first trying to learn the language). It makes it look like its doing special parsing, even if it isn’t; which in turn makes it confusing when & can’t do special parsing.

do ... end are not macros, they are keywords in the language used to enclose a block. A block is a list expressions that will be evaluated in order and return the last expression.

def foo() do
  a
  b
end

The code above is parsed like this: def(foo(), [do: __block__([a, b])]). __block__ is not a function, it is actually not even a macro, when the compiler sees __block__ it will emit code that evaluates all expressions in the given list and returns the last one. And as you can see do ... end becomes [do: ...]. [] is optional around keyword lists so you can also write it as do: .... Now you can see that the two ways to define functions really are the same.

def is just a macro where the first argument is the function head and second argument is the function body.

​

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Eric Meadows-Jönsson

[José Valim]

Thanks for the reply Dave.

When it comes to syntax, Elixir has a goal to have few rules, otherwise they would make the syntax itself and macros too complex. With this in mind, my comments are inlined below.

 

somefunc asdf |> someotherfunc gherg

All user operators in Elixir have higher precedence than function calls. So it becomes simpler to know that "foo 2 + bar" is parsed the same as "foo 2 |> bar". Not only that, because almost everything is a function/macro call, having |> with higher precedence could actually lead to some disasters, like:

    def foo, do: bar |> baz

Actually translating into:

    (def foo, do: bar) |> baz

Which leads back to your point made below about requiring more work when converting do/end to do:.

(Eric says -> is an operator but it is more of a parser/grammar implementation detail. The list of binary operators can be found here: https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/macro.ex#L31-L50).

 

Another is

def func, do:

  needsParens(sadf)

def func do

  doesn'tNeedParens sadf

end

do/end is just syntax sugar for the "do: (block)" keyword list.

def func do

  doesn'tNeedParens sadf

end 

Is the same as:

def func, do: (doesn'tNeedParens sadf)

In fact, none of the examples above need parentheses. The parentheses are only required if you have more than one argument in the inner function call:

some func, needParens sadf, ola

Because it is unclear if you mean one of:

some func, needParens(sadf, ola)

some func, needParens(sadf), ola

The decision to raise in this last case is actually debatable, we could very well make it work, but asking users to be explicit strikes me as a good thing as things could get pretty messy once you add more arguments:

if some, do: needParens sadf, ola, else: bar 

 

And I'm not sure why arity has to be indicated with what looks like a division (other than adoption from erlang), if its going to be the cause of some strange stuff.

We could have come up with a specific syntax but we didn't and now the use of &fun/arity is so pervasive that changing it would just be too hard. We had discussions about this in the past too and no good alternatives came up too.

What came up from that discussion though was a set of measures to reduce the ambiguity. We have removed the // operator from the language because of that.

Note though that the choice of / is not what makes things like "&something" forbidden in the language (which I assume is the origin of your comment). Even if we had picked another operator, the new operator would be bound to the same rules as /, so the same constraints would apply (exactly because we want to have few rules).

For example, it is common to think of a possible solution like: "you can just check if the right side of / is an integer", as in:

&hello/0

But we should remember that we could actually be inside a quote:

&unquote(fun)/unquote(arity)

And now the parser has no idea unquote(arity) is an integer.

Or what if you are injecting the whole fun/arity expression, as in:

&unquote(fun_arity)

That's why / has little to do with forbidding "&something". The syntax constructs should work when written in parts and then glued via quoted expressions.

[Dave Martin]

I oversimplified my def example, sorry about that. I meant to replace it with a better example.

And as you can see do ... end becomes [do: ...]

It appears do ... end actually becomes ", [do: ...]"

P.S. I was in a bit of a rush on the previous post, and its been a while, but I think Eiffel only lets you leave off parenthesis when its a 0 argument call, and my last post implied otherwise. (It does let you leave off statement terminators, unless putting more than one statement on a single line).

[Dave Martin]

I'll have to re-read the version of the materials that are present now (in case this already exists and I've missed it) but I'd like to suggest some sort of philosophy document, that explains the ideas/goals behind Elixir, and how and why and when it parses things the way it does; and how this is different than what I'll just refer to as more traditional languages.

I can see that I'm going to need to learn about macros as well. It is apparent that the way the language works is based on them, and even if one isn't going to use them, they need to be understood to understand why some things are the way they are. (ugh, sorry about the grammar there).

I was under the impression that macros were to be avoided unless you explicitly needed one for something, and even then think three times about it first. But I think understanding them and how elixir is built on them will be helpful in sorting out some of the things that seem strange. (I'm still having trouble for instance with &((Enum).map)   / 2 being the same as &Enum.map/2 which seems like (1 2) 3 being equivalent to 123 to me).

[Dave Martin]

Another one:

why is it:

for <<r::8, g::8, b::8 <- pixels >>, do: {r, g, b}

and not:

for <<r::8, g::8, b::8>> <- pixels, do: {r, g, b}

It makes it look like a special generator syntax had to be invented for binaries, instead of just saying generators need either an enum or a binary on their right-hand-side, and if its a binary, the minimum amount that fits into the left hand side will be taken. (which all seems to be the case, except binaries need this alternate syntax. It seems like the implementation would still need to detect this alternate syntax, so why can't it just do the equivalent of "when is_binary(arg)", instead of "when target looks like <<_ <- arg>>"

And it still works if pixels is really an enum, so "regular" binary matches won't break:

iex(17)> pixels = [<<1,2,3>>,<<2,3,4>>,<<3,4,5>>]

[<<1, 2, 3>>, <<2, 3, 4>>, <<3, 4, 5>>]

iex(18)> for <<r::8, g::8, b::8>> <- pixels, do: {r, g, b}

[{1, 2, 3}, {2, 3, 4}, {3, 4, 5}]

On Saturday, June 28, 2014 2:04:56 AM UTC-6, José Valim wrote:

[José Valim]

Not all patterns valid in an enum generator would be valid in binary ones. The one below works only for enums:

    <<r::8, _::binary>> <- enum

Also mixing the two syntaxes would make error reporting harder (because we don't really know what you mean) and optimizing trickier.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer