[Proposal] Allow guards to be interspersed anywhere in clause heads

[Christopher Keele]

This is a more formal write-up of the discussion started with José here. Interested in all feedback, potential use-cases, and syntactic edge-cases!

Proposal

Allow `when conditions...` to be placed anywhere within the patterns used in functions, cases, and other such match head constructs—in addition to the suffix-only version we support today.

I believe this can be done by the compiler today as a backwards-compatible enhancement with no change to the parser.

Synopsis

I'm proposing we allow guard clauses anywhere within match patterns, as well as following them, so that the AST for match heads containing guards can be composed easily.

To show rather than tell, assuming you have AST for the following:

X = x when x > 0
Y = y when y > 1

Both X and Y are valid, stand-alone match heads. However, they are not also valid stand-alone match patterns: there is no easy way to combine them to form a new 2-arity match. The naive approach would be:

XY = (x when x > 0, y when y > 1)

While this is not currently a valid match head, once can hoist any internal guards into a valid form like:

XY2 = (x, y) when x > 0 and y > 1

This proposal explores allowing XY as valid Elixir code and rewriting it to XY2 at compile time.

Terminology

To talk about this precisely I'm going to appropriate some ETS terminology to reference Elixir syntax constructs I don't have canonical names for. Let me know if you are aware of the correct terms for these!

- a match specification is defined as a match head -> body pair

- a match head is defined as a match pattern optionally followed by guard clauses

- a match pattern is defined as a comma delimited series of expressions allowed in match contexts

- a guard clause is defined as a when conditions expression formulated of functions allowed in guard contexts

Technically, what I'm proposing is to loosen the restriction that guard clauses must follow match patterns, by allowing match heads themselves to recursively be valid expressions in parameters lists. The compiler can extract all guard clauses found within a match head, leaving only a valid match pattern, and combine the extracted guards with any existing ones to create a new valid match head with equivalent semantics.

Parsing

Currently, both of these (and all other variations I can think of) are already valid syntax to the parser:

fn x when x > 0, y when y > 1 -> #... end
def name(x when x > 0, y when y > 1), do: #...

Only the compiler keeps you from running a program with these internal guards; it can be parsed into AST with the exact precedence we want without a hitch.

Grammar

The only ambiguity I can think of would be the following:

fn x when x > 0, y when y > 1 when y < 3 -> #... end

It is not clear where whether the intent was to create an 'or' multi-guard around the entire parameters list, or to guard both the last parameter and the parameters at large with two separate clauses. The parenthesis conventionally always used in defs resolves the ambiguity. I am open to ideas on how to handle this situation, though personally I envision a compile-time warning and treating it as a multi-guard as this is most consistent with the precedence of when. I don't see it coming up too often in generated code.

It is worth observing that while the following technically has the same ambiguity:

fn x when x > 0, y when y > 1 -> #... end

however you decide to treat the guard after the second parameter, the resulting guards post-rewrite will be semantically equivalent.

Rewriting

Since everything up to this point is already valid, I suspect the rewrite could be done in a single place in the compiler with no further changes to any other code.

The algorithm I have in mind is to simply walk the AST outside-in, removing each set of consecutive guards it finds, and exploding the permutation of each sets' multi-guards out to create new trailing clauses, then anding all terms together in each new guard clause. In the most common case that simply entails prefacing the trailing guards with any interior guard, all anded together.

This approach is intentionally naive about what variables are referenced in which guards where. Anything that produces a valid guard in the end can fly, even if generated code produces them in odd or unexpected places within the params list.

Simple extraction:

def name(%{foo: bar when is_integer(bar), fizz: buzz when is_integer(buzz)})
  when bar + buzz > 100

def name(%{foo: bar, fizz: buzz})
  when (is_integer(bar) and is_integer(buzz)) and (bar + buzz > 100)

In order to handle mutli-guards correctly we'd have to get a little n^m but I doubt even the most ambitious metaprogramming would ever need to generate code like that.

Matrix of multi-guards with oddly referenced variables:

def name(x, y when y > z when z > x, z)
  when is_integer(z)
  when is_string(z)

def name(x, y, z)
  when y > z and is_integer(z)
  when y > z and is_string(z)
  when z > x and is_integer(z)
  when z > x and is_string(z)

Thoughts

The purpose of this is to make function heads more composable in meta-programming. Initially it would be released with zero fanfare except perhaps a footnote in the guard docs. However, I can see this ability catching on with people coming from strongly-specified typed languages, allowing them to qualify type expectations in parameter lists inline with where variables are defined.

I don't think that would be too problematic since I find the style to be pretty readable and more importantly the code still easy to reason about. However, this is definitely enabling an alternate way to write pretty basic syntax, so if we were really against human beings rather than macros employing doing this, the rewrite process could emit warnings from code not marked as generated.

I have no idea how the formatter should handle this syntax. Perhaps its behaviour already suffices, since it's pretty good at deconstructing long function heads?

I'm interested if there are strong opinions about this one way or the other on the mailing list, as well as if there are any implications I've overlooked in my suggested implementation.

Thanks for reading!

Chris K

[Allen Madsen]

I believe `when` with `=` is visually ambiguous and works different from how people would expect it to.

The way I read the result of this AST:

def foo(%{x: x} when x > 1 = result), do: 1

{:def, [context: Elixir, import: Kernel],

 [

   {:foo, [context: Elixir],

    [

      {:when, [],

       [

         {:%{}, [], [x: {:x, [], Elixir}]},

         {:=, [],

          [

            {:>, [context: Elixir, import: Kernel], [{:x, [], Elixir}, 1]},

            {:result, [], Elixir}

          ]}

       ]}

    ]},

   [do: 1]

 ]}

It means:

def foo(%{x: x} when (x > 1 = result)), do: 1

Instead of:

def foo((%{x: x} when x > 1) = result), do: 1

Allen Madsen
http://www.allenmadsen.com

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/29c74e96-efc0-4254-bd34-eb1fc2c03968%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Christopher Keele]

That's a good point. I suspect the convention around manual usage would converge around:

def foo(%{x: x} = result when x > 1), do: 1

which parses as expected. This version feels more natural to me.

I don't think the other form would accidentally be introduced while injecting one snippet of AST, because the tree would never be constructed the wrong way. String-based expression interpolation might make that mistake, but if you're metaprogramming with literal strings of code you've already made a mistake.

I hadn't typed out an assignment case with inline guards, it does feel like it's doing too much in a short space with few grammar tokens to help understand it. That's a good argument for intentionally discouraging these constructs outside of metaprogramming, rather than adopting it as a common usage language feature.

[Allen Madsen]

If the goal were primarily macro usage, wouldn't this type of thing already be possible with a macro today?

Allen Madsen
http://www.allenmadsen.com

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/f2fafacb-3302-4c03-9b1b-ed766a896cc2%40googlegroups.com.

[Fernando Tapia Rico]

IIRC, macros do not allow to abstract both the pattern match and the guard within the same expression. For example, the Elixir codebase contains a helper function that abstracts the pattern match, the guard and the "read" function for dates and times separately. Here is the helper function https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/calendar/iso.ex#L39-L70; and here is an example of its usage https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/calendar/datetime.ex#L522-L531.

Besides, as described by José Valim in here, internal guards would allow to write macros like:

def my_fun(my_macro())

which would be expanded to:

def my_fun(my_var when my_var in [:foo, :bar])

and rewritten by the Elixir compiler to:

def my_fun(my_var) when my_var in [:foo, :bar]

However, I'm not sure how common are those use cases 🤔.

[Louis Pilfold]

Hey

Macros are more than capable of providing this functionality, you'd just need to define your own `def` like macro that generates the desired function.

```

MyLib.def my_fun(my_var when my_var in [:foo, :bar])

```

Cheers,

Louis

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/7fda093f-03b5-4d1d-9c16-73c9475ca6a7%40googlegroups.com.

[Christopher Keele]

 > If the goal were primarily macro usage, wouldn't this type of thing already be possible with a macro today?

> Macros are more than capable of providing this functionality, you'd just need to define your own `def` like macro that generates the desired function.

This is absolutely true, and expat does something similar today. It could be done easier and better in core, of course. I think the heart of the proposal is: given that this is straightforwardly possible today, and the implementation identical and completely deterministic no matter the application, is there a good reason not to expand the expressivity of guard clauses in the core of the language beyond that which erlang offers?

The macro-authors-over-common-coding argument is me simply saying, let's consider this from the perspective of increasing the language's extensibility first, since I suspect the discussion around whether or not it makes day-to-day coding more productive to be far more opinion-oriented.

I am not sold on the idea myself, which is why I dug into the implementation a bit so I could present it better. It absolutely could be done via macro, but I think it exists at an interesting low-hanging fruit, low-risk way to make the language itself more powerful today.

> IIRC, macros do not allow to abstract both the pattern match and the guard within the same expression.

You do a better job of summarizing how this could be used in core today than I did. I am not sure how common the use-case is either, but I do have one concrete example in expat and I wonder what might embraced in the future if this purely historical grammatical restriction were lifted.

[Louis Pilfold]

Hey

> is there a good reason not to expand the expressivity of guard clauses in the core of the language beyond that which erlang offers?

I've no strong opinion on the proposal, but I'd just like to point out that expressivity has a formal CS definition and implementing this proposal will not increase the expressive power of guard clauses as they will not be capable of any functionality they did not have before.

This is a matter of style and syntax.

Cheers,

Louis

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/9e9be43d-b772-480c-afe9-bb07a7e01119%40googlegroups.com.

[OvermindDL1]

This proposal would also allow an expansion of `defguard` to cover my previous recommended functionality to it as well.

[Christopher Keele]

Hey Louis!

 

I've no strong opinion on the proposal, but I'd just like to point out that expressivity has a formal CS definition and implementing this proposal will not increase the expressive power of guard clauses as they will not be capable of any functionality they did not have before.

 

That's a good point. Arguably it would add expressivity to match patterns—though I'm relying on the fact that that doesn't add new abilities to the language as a whole to perform the rewrite, so it is definitely syntactic/semantic sugar.

This proposal would also allow an expansion of `defguard` to cover my previous recommended functionality to it as well 

Do you mind expanding on this, Overmind? 

[OvermindDL1]

On Thursday, September 27, 2018 at 5:18:15 PM UTC-6, Christopher Keele wrote:

This proposal would also allow an expansion of `defguard` to cover my previous recommended functionality to it as well 

Do you mind expanding on this, Overmind? 

When `defguard` was initially being added I proposed that it should handle both a matcher and a guard and that it should be used in matcher placement like via `def blah(is_person(age: age)) when age>18` or so forth, it allowed for both variable binding so you could pull data 'out' of the match (via positional or named arguments), in addition it would have required `def` to do precisely what this proposal does, which is to distribute the `when`'s from inside the arguments to be added to main guard section of the function head.  I made a mini-library with 'just' enough functionality to demonstrate how it would work (overwriting `def` with my own version where it was `use`d).