Pipe and anonymous functions

[José Valim]

This comes up every once in a while... and a lot of people starting with the language seem to expect it to work.

So let's give it another try: should the following work?

def boundary do

  :crypto.rand_bytes(8)

  |> Base.encode16

  |> &("--------FormDataBoundary" <> &1)

end

Advantages:

1. Easy piping elsewhere

2. Apparently it is obvious to many people (please reply if you disagree)

Disadvantages:

1. If the function is defined outside of the pipe, we need to explicitly invoked it with .()

Thoughts?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Peter Hamilton]

It seems obvious to me. I'm trying to think about how to do it otherwise. I guess memoize and then concat would be my approach.

Is there a difference between a function defined inside a pipe and outside? What is that difference?

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alexei Sholik]

That would be a non-intuitive special case, imo. I've had my thoughts on pipes:

04:33 true_dro_: jadams: I wish pipes worked on functions though, without macro magic. Also that &1 would only work inside function argument list
04:33 true_dro_: you would then be able to write foo |> func(&1, other) |> other_func(bar, &1) |> anon_func

Or, if we keep & unchanged, it could work like this:

foo |> &func(&1, other) |> &M.other_func(bar, &1) |> anon_func

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Best regards
Alexei Sholik

[Kirill Romanov]

Such "sugar" on top of pipes can lead to wrong understanding of what pipe is. Right now it works perfectly like this:

"hello" |> (fn txt -> IO.puts(txt) end).() # full syntax

"hello" |> (&(IO.puts(&1))).()             # all parentheses

"hello" |> (&IO.puts&1).()                 # shorthand

When I wrote such code for a first time I didn't even have a doubt will it work or not.

среда, 9 июля 2014 г., 0:46:38 UTC+4 пользователь alco написал:

[Peter Hamilton]

On Tue, Jul 8, 2014 at 2:08 PM, Kirill Romanov <o...@aleph0.cc> wrote:

Such "sugar" on top of pipes can lead to wrong understanding of what pipe is. Right now it works perfectly like this:

"hello" |> (fn txt -> IO.puts(txt) end).() # full syntax

"hello" |> (&(IO.puts(&1))).()             # all parentheses

"hello" |> (&IO.puts&1).()                 # shorthand

Is there any possible meaning for:

  "hello" |> &IO.puts(&1)

?

I understand the consistency argument, but it could be mighty convenient without the loss of functionality. 

[Kirill Romanov]

среда, 9 июля 2014 г., 1:17:44 UTC+4 пользователь Peter Hamilton написал:

On Tue, Jul 8, 2014 at 2:08 PM, Kirill Romanov <o...@aleph0.cc> wrote:

Such "sugar" on top of pipes can lead to wrong understanding of what pipe is. Right now it works perfectly like this:

"hello" |> (fn txt -> IO.puts(txt) end).() # full syntax

"hello" |> (&(IO.puts(&1))).()             # all parentheses

"hello" |> (&IO.puts&1).()                 # shorthand

Is there any possible meaning for:

  "hello" |> &IO.puts(&1)

?

From the formal point of view it's the same as "hello" |> "world", what should it do? I think if we have a `call operator` in elixir it's a good place to use it. And for any newcomer only one look at current pipe examples with anonymous functions can tell all the story about functions as values, call operation etc. I think it's a good thing am I wrong?

[Bob Hutchison]

On Jul 8, 2014, at 3:24 PM, José Valim <jose....@plataformatec.com.br> wrote:

This comes up every once in a while... and a lot of people starting with the language seem to expect it to work.

So let's give it another try: should the following work?

def boundary do

  :crypto.rand_bytes(8)

  |> Base.encode16

  |> &("--------FormDataBoundary" <> &1)

end

Advantages:

1. Easy piping elsewhere

2. Apparently it is obvious to many people (please reply if you disagree)

Disadvantages:

1. If the function is defined outside of the pipe, we need to explicitly invoked it with .()

Thoughts?

at the risk of being painfully obvious, this works:

:crypto.rand_bytes(8)

|> Base.encode16 |> 

|> (&("--------FormDataBoundary" <> &1)).()

Clojure has a similar problem with its threading macros and hasn’t done anything to deal with it (in Clojure it is a weird ((+ 1)) kind of thing you write).

Cheers,

Bob

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Bob Hutchison]

at the risk of being painfully obvious, this works:

and incredibly late. Sorry, don’t know what happened to delay this email.

[José Valim]

at the risk of being painfully obvious, this works:

:crypto.rand_bytes(8)

|> Base.encode16

|> (&("--------FormDataBoundary" <> &1)).()

Yup, this form works and is usually the solution proposed for such cases.

I concur that allowing such forms would be a special case for pipe and possibly confusing in the long term. However, given that many were expecting it to work, I decided to gather everyone's feedback. And based on this thread, it doesn't seem we should change the pipe operator.

[Bob Hutchison]

On Jul 9, 2014, at 8:01 AM, Bob Hutchison <hutch...@recursive.ca> wrote:

On Jul 8, 2014, at 3:24 PM, José Valim <jose....@plataformatec.com.br> wrote:

This comes up every once in a while... and a lot of people starting with the language seem to expect it to work.

So let's give it another try: should the following work?

def boundary do

  :crypto.rand_bytes(8)

  |> Base.encode16

  |> &("--------FormDataBoundary" <> &1)

end

Advantages:

1. Easy piping elsewhere

2. Apparently it is obvious to many people (please reply if you disagree)

Disadvantages:

1. If the function is defined outside of the pipe, we need to explicitly invoked it with .()

Thoughts?

at the risk of being painfully obvious, this works:

:crypto.rand_bytes(8)

|> Base.encode16 |> 

|> (&("--------FormDataBoundary" <> &1)).()

Clojure has a similar problem with its threading macros and hasn’t done anything to deal with it (in Clojure it is a weird ((+ 1)) kind of thing you write).

Pardon me for doing this, but let me add a bit to this Clojure comment.

Clojure has similar macros to Elixir’s |>. Clojure requires you to wrap an anonymous function in parenthesis in their threading macros, even though you can drop the parenthesis around functions taking one argument:

(-> 1 inc inc ((fn [i] (+ 2 i))))

or

(-> 1 inc inc (#(+ 2 %)))

If you leave the outer () off you get the very unhelpful error message:

CompilerException java.lang.IllegalArgumentException: Parameter declaration inc should be a vector, compiling:(flow/core.clj:1:20)

(and to make things even worse, if you do this in the Clojure REPL you’ll have to restart the REPL)

There’s been discussion in the Clojure community about this, because basically everyone tries it the wrong way first and it’s not obvious what the problem is. Yet there’s no move in the Clojure community to change how it works because they maintain it’s actually the right thing to do. Basically the argument is that -> and company are macros, and macros manipulate sexps, and the definition of what -> does is in terms of sexps only, and there’s no way to distinguish an anonymous function definition from an sexp, and so what’s happening is exactly what you should expect (though the error message is useless). For example: https://groups.google.com/forum/#!topic/clojure-dev/Q95iISWA5CQ

Cheers,

Bob

[José Valim]

Thanks Bob, it is actually extremely helpful to know the same issue exists in the Clojure community.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Bob Hutchison]

Your timing of this thread is excellent :-)

https://groups.google.com/forum/#!topic/elixir-lang-talk/PEBKkCQ-tU4

[José Valim]

Haha, there was also an open issue and a stack overflow question.

Nonetheless, I have improved the error message:

iex(1)> 1 |> & &1

** (ArgumentError) cannot pipe 1 into &&1, can only pipe into local calls foo(), remote calls Foo.bar() or anonymous functions calls foo.()

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

On Thu, Jul 10, 2014 at 2:34 PM, Bob Hutchison <hutch...@recursive.ca> wrote:

Your timing of this thread is excellent :-)

https://groups.google.com/forum/#!topic/elixir-lang-talk/PEBKkCQ-tU4

[Josh Bourgeois]

Many significant apologies for reopening this very-very-well-treaded topic, but I do want to advocate this one more time.

The pipe operator is used to illustrate the flow of data transformation, but while functional programming is very expressive, there are times when a signature requires the data in question to be used as a different argument. In those cases, you must either break the pipe chain (and thus the flow of data transformation) to assign the transformed data to a variable, or send the data through an anonymous function to continue the program flow.

Take a process where data should be transformed before being saved to a database:

# intermediary value
def encrypt_password(user, password) do
  encrypted = password
    |> Salt.add
    |> BCrypt.hash
  put_in(user[:hashed_password], encrypted)
    |> EctoOrSomething.UpdateIGuess.ImStillNewHere
end

# anonymous function wrap

def encrypt_password(user, password) do
  password
    |> Salt.add
    |> BCrypt.hash
    |> (fn(encrypted) -> put_in(user[:hashed_password], encrypted) end).()
    # or |> (&(put_in(user[:hashed_password], &1)).()
    # but considerably messy either way

    |> ...
end

The main issue in conceptualizing a more elegant workflow to represent these transformations is that any construct implemented will make an affordance in one direction or the other. A pipe-to-argument operator would offer strong reusability, but proposals in this group suggested a token that poorly expresses its usage (I'm thinking of a thread where ~| had been proposed as an anonymous variable binding with |>) . The other proposals are for inflexible cases that would necessitate more extensions for further implementations, and/or borrow symbols from other languages that are ambiguous in their usage (such as proposals for <|, |<, and |>> as tokens for pipe-to-last).

I think Elixir is in a unique position to solve this problem, though. If I could draw a correlation, the pin macro, Kernel.^/1 is used to bind a variable in at a location for pattern-matching...

good_status = "200"
{^good_status, response} = fetch("www.example.com")
# expands to {"200", response} = fetch("www.example.com")

I believe there's syntactic justification for a new pipe macro, Kernel.|^/2, that would interact with a Kernel.^/0 macro, to combine the concepts of the pipe and the pin. Assuming the current pipe operator symbolically translates | to "pipe to" and > to "the left", |^ would represent "pipe to the pin". The pin would appear at least once in the expression on the right in order to represent where the expression on the left will appear, expanded, like such:

def encrypt_password(user, password) do
  password
    |> Salt.add
    |> BCrypt.hash
    |^ put_in(user[:hashed_password], ^)
    |> ...
# expands to ...(put_in(user[:hashed_password], BCrypt.hash(Salt.add(password)))
end

Benefits to this approach:

• The current pipe operator behavior remains pristine
• ^/0 continues to act as a reference to expanding a variable
• No introduction of new symbolic concepts
• Syntax is flexible enough to expand to any (or multiple) argument position

# more contrived examples!!
> map = %{little_bunny: %{}}
> :foo
    |^ put_in(map[:little_bunny][^], ^)
%{little_bunny: %{foo: :foo}}

> "Ton" |^ (^<>"y! " <> ^<>"i! " <> ^<>"é!")
"Tony! Toni! Toné!"

Again, I know there've been many discussions about the drawbacks to implementing a new pipe mechanism, and the Elixir community may have already come to a consensus around how to handle transformations like this, but I think the longevity of this topic points to a desire for some kind of better solution.

If everyone is tired of talking about this, I'll happily let it end here, but I thought one more perspective couldn't hurt.

Thanks for reading! ^_^

[OvermindDL1]

Hmm, I rather like it, `|>` is pipe-start (still think it should be pipe into the end of the argument list) and `|^` could be pipe-pin, interesting approach, I like it.

[José Valim]

As explained in the many others pipe proposals sent to this mailing list, we don't plan to add more complexity to the pipe operator.

Despite the suggestions of many, anonymous functions are not the solution when you can't pipe. Well named and defined private functions are. Elixir is a functional language. So create small named functions to improve code readability.

--

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Josh Bourgeois]

Thank you for the reply! We use Rails exclusively on my team, so all of my exposure to Elixir so far has come through HackerRank, Exercism, and poring over the online guide and core documentation.

May I ask for clarification around what you mean when you say "create small named functions [when you can't pipe]"?  From everything I've read and seen,

thing |> a_function

is simply converted to

a_function(thing)

at compile time, so... "when you can't pipe", literally translates to "when you would be unable to call a function"... Unless I've missed something? How would one apply the advice of creating a small named function to a practical example to a workflow point where you would be unable to call a function?

Since my password function was the only semi-noncontrived example, I guess you're saying it would be best practice to rewrite it from 

def encrypt_password(user, password) do
  encrypted = password
    |> Salt.add
    |> BCrypt.hash
  put_in(user[:hashed_password], encrypted)

    |> EctoLike.update
end

to something like 

def encrypt_password(user, password) do
  put_in(user[:hashed_password], encrypt(password))
    |> EctoLike.update
end

defp encrypt(password) do
  password
    |> Salt.add
    |> BCrypt.hash
end

or something similar, right? I've learned to try to keep code as close to where it winds up getting run as I can (jumps are bad, m'kay?), so my instinct is to keep processes inlined in their methods until I use the process in more than one place, or (in this new Elixir land) until I have a second clause I need to define.

[Josh Bourgeois]

As a concrete example, here's the implementation I came up with for this word count exercise:

defmodule Words do

  def count("" <> string), do: count(parse(string), %{})

  defp count([], map), do: map

  defp count(["" | words], map), do: count(words, map)

  defp count([word | words], map) do

    word = word |> String.replace("_", "-")

    count(words, put_in(map[word], (map[word] || 0) + 1))

  end

  # strip underscores, retain hyphens

  defp parse(string), do: string |> String.downcase |> String.replace("-", "&hyph;") |> String.replace("_", "-") |> String.replace("&hyph;", "_")  |> String.split(~r/\W/u)

end

To apply your advice to my final definition of count, I guess the cleanest way to rewrite it would be along the lines of...

  defp count([], map), do: map

  defp count(["" | words], map), do: count(words, map)

  defp count([word | words], map), do: count(words, increment(map, word))

  defp increment(map, word)
    word = word |> String.replace("_", "-")
    put_in(map[word], (map[word] || 0) + 1)  
  end

But I'm ... not really seeing the upshot of pulling out the logic this way. Besides having another place to look to find out what my code is doing, I haven't actually simplified the final function body that much.

I'll keep practicing, and it'll probably click soon enough.

[Josh Bourgeois]

I thought of practicing, but then I just wound up doing a lot of thinking. Then it clicked 😬

defmodule Words do

  def count("" <> string), do: parse(string) |> Enum.reduce(%{}, &count/2)

  defp count("", map), do: map

  defp count("" <> word, map) do

    word |> String.replace("_", "-")

      |> increment(map)

  end

  defp increment(word, map), do: put_in(map[word], (map[word] || 0) + 1)

  defp parse(string), do: string |> String.downcase |> String.replace("-", "&hyph;") |> String.replace("_", "-") |> String.replace("&hyph;", "_")  |> String.split(~r/\W/u)

end

It's just a challenge to think of meaningful names sometimes. Breaking function/method bodies into smaller discrete blocks of work is obviously a fundament to making code more readable, but it takes discipline and, sometimes, a thesaurus

Thank you for entertaining this zombie discussion once more 😶

[Onorio Catenacci]

To paraphrase Rich Hickey--"Simplicity isn't easy".  Figuring out those simple function signatures is not the first thing that comes to mind for most of us.  But it's worth the effort.

[Robert Virding]

Actually this was said by Dijkstra long before Rich:

“Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better.”

Unfortunately all 3 points are true. He also said:

“Simplicity is prerequisite for reliability.”

which is also true.

[Onorio Catenacci]

That's a great point Robert and Mr. Dijkstra certainly deserves credit.  Thanks for reminding me about that.

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/dwbNOh_yNd4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/59c6d90d-ded2-4794-8a73-920ab9350007%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Onorio Catenacci
 
http://onor.io

http://www.google.com/+OnorioCatenacci

[Josh Bourgeois]

+1. I do have to say, though, I'll be far more likely to implement the refactor based on Rich's implementation. 

One might be able to refactor it even farther, simply to "easy is hard", but then it may lose some nuance and portability.

I really like the constructs Elixir uses to encourage code simplicity. Pattern matching and guard signatures allow functions to take on a level of granularity where you legitimately can write functions that perform exactly one action, exactly the same way with the same inputs. What emerges is a coding style where, as a general rule, you can say that if your Elixir function needs to set a variable, it can be refactored to a simpler function body.

My rationale for suggesting the pinpipe operator was because, within some function bodies, the only reason you might consider introducing a variable is so that your transformed data can be used in a specific position of your function result. The code example of def boundary at the start of this thread, for instance, is a self-contained, pure function as it is. The anonymous function called at the end does not need to exist, as the SHA sent in could just as easily be a variable:

def boundary do
  sha = :crypto.rand_bytes(8)
  |> Base.encode(16)
  "---------FormDataBoundary" <> sha
end

DRY principles (at least, as far as The Rails Way was taught to me) would say that if there were no other way you would want a "---------FormDataBoundary" to be generated, then you would want to keep this method body self-contained, and with as few introduced variables as necessary. Pipe workflows keep that intent in focus by dissuading you from making functions that work on anything but their inputs. When I'm in that mindset, variables set within a function are an anti-pattern, but I would still sooner catch myself moving the pipes to the place where they were necessary:

def boundary do
  "--------FormDataBoundary" <> (:crypto.rand_bytes(8)
    |> Base.encode(16))
end # sha, you know what? uh-uh.

Coming back to being able to overload signatures as you can in Elixir, after half a decade of not being able to in Ruby and Javascript, the above is still far more natural to me (even though I know inside how kludgey it feels) compared to something like:

def boundary, do: :crypto.rand_bytes(8) |> Base.encode(16) |> boundary
defp boundary(sha), do: "--------FormDataBoundary" <> sha

But almost all of my hesitation around a convention like this comes from not being used to it. Portable, reusable functions are very good things to have central to a language design, and if Elixir has A Right Way to implement those, it's within the maintainers' rights to foster that happy path and discourage what would be impure solutions according to the language philosophy. As the community grows and more code examples become available, fewer people will wonder about how they should do these things.

To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/59c6d90d-ded2-4794-8a73-920ab9350007%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Wojtek Mach]

Shameless plug: I just created a small library that allows to do: `4 |> & &1 * 2` and similar. You might find it occasionally useful in a local iex session but for the good reasons mentioned in this thread please don't use it in production code :)

https://github.com/wojtekmach/pipe_capture

[Louis Pilfold]

It's not that weird when you think about it in terms of the AST that the macro operates on, and not in terms of types.

The former would require rewriting the forms into anon function calls, while the latter just prepends a value onto the argument list. `|>` is a very very simple macro that can be implemented in a single line. There's a bunch of friendly error handling in the one in Kernel, but it's still just as simple.

Cheers,

Louis

On Wed, 18 Apr 2018 at 11:36 Dylan Johnston <dylan.r....@gmail.com> wrote:

Sorry to dredge up an old thread but I wanted to push back against this again. The point of the pipe operator is to be able to clearly express the flow of data through a transformation. Claiming the pipe operator shouldn't be able to handle captured or anonymous functions because it confuses how it works is confusing an implementation detail of how the pipe operator macro works now to how it is semantically expected to behave. Having to wrap them in parenthesis and an invocation introduces a great deal of noise to what could be an extremely simple syntax. As Wojtek Mach showed, it's fairly easy to get the pipe operator to handle these cases, especially since you're pattern matching against them anyway to give error messages. Why pattern match only to give an error when it's easy and straightforward to just make it work, what else could a user intend other than to call the function?. There's a pull request here I made than extends his work so that captured functions and anonymous functions both work. https://github.com/wojtekmach/pipe_capture/pull/1

 

"Hello"
|> fn x -> IO.puts x end
|> &IO.puts/1
|> &IO.puts(:stderr, &1)

 vs

"Hello"
|> (fn x -> IO.puts x; x end).()
|> (&IO.puts/1).()
|> (&IO.puts(:stderr, &1)).()

It seems weird to me that the pipe macro works exclusively on already invoked functions and doesn't know what to do with a function reference.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/b2182e21-939e-4762-b603-9cd492e1650b%40googlegroups.com.

[OvermindDL1]

Going on that route though, it is weird when compared to the vast variety of languages that had pipes before elixir though.  Still though, it's not easy to change the current pattern without either lots of case's being generated in the backend or for the language to get a somewhat half decent type system.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFDB7oMcaXNfrYbkh%3DrLz7YcZMRzcuODjHWaJKp5DkNxZw%40mail.gmail.com.