Infix functions / named operators

[Marco Matthies]

Hi,

let me first start by saying that Julia is awesome! :)  I'm having a lot of fun programming in it.

I think it would be even better though with infix functions / named operators.
Please forgive me if this has been discussed before, but a quick search didn't show much except for the following bug on github:

    https://github.com/JuliaLang/julia/issues/552

In that bug it is also mentioned that Julia is out of ascii characters.  Here is a proposal to change that.

=============

In Haskell infix functions are written as

    `op`

so you can do

    a `op` b

and this would call op(a,b).

Backticks are already taken in Julia for commands.

So how about this syntax

    a .op. b

taken from Fortran.  I think it looks better than backticks (Haskell) or % (R).

The whitespace around .op. would be mandatory to avoid mixup with the Module.function syntax (and a possible future dot operator).

It would be best if
   a .op. b
calls
   operator_op(a,b)

This way one can have short names for operators without polluting the namespace.

Then one could define

   .bitor.
   .bitand.
   .bitxor.
   .mod.

and liberate the |, &, $, and % symbols for which more important uses can be found.
Perhaps | could then become the pipe / function chaining symbol ( instead of |> ).
There are probably good other uses for &, $, and %.

Another nice thing would be the vector cross product:

   a .cross .b

or maybe even

   a .x. b

as well as wedge product etc etc.  Maybe even symbols could be allowed inside the dots, although i'm not sure if that results in more readable code. (Would be nice to be able to do it nonetheless).

What do you think --- would this work?

[Stefan Karpinski]

Allowing functions to be used as infix operators is certainly an appealing idea. It might well be possible even without the dots, but it's not a change to be undertaken lightly.

[Jeff Bezanson]

I think the dots are a good proposal; it's a fairly light syntax and
the fortran precedent makes it not-totally-random.

The "operator_" prefix is interesting. It never would have occurred to
me, as I'm generally dead-set against any magic renaming or having
"real" names different from apparent names. It's neat that you can do
"a .x. b", but I don't think it's worth it. I'd much rather have this
be an alternate syntax for calling the same set of functions.
Otherwise confusing things become possible, like `in(a,b)` behaving
differently from `a .in. b`. Some people might dislike the infix
syntax and never use it, while others might badger those people to add
operator_ versions of their library functions so they become
infixable. There could be a lot of unnecessary debate there.

[Steven G. Johnson]

On Tuesday, February 11, 2014 1:52:08 PM UTC-5, Jeff Bezanson wrote:

I think the dots are a good proposal; it's a fairly light syntax and
the fortran precedent makes it not-totally-random.

There is already some confusion between 2.^3 and 2. ^3 ... do we want to also add a confusion between 2.^.3 and 2.^ .3?

[Jeff Bezanson]

Well, the spaces around the .op. would be required, but true that this
does combine oddly with existing dot-operators.

[Steven G. Johnson]

On Tuesday, February 11, 2014 1:52:08 PM UTC-5, Jeff Bezanson wrote:

The "operator_" prefix is interesting. It never would have occurred to
me, as I'm generally dead-set against any magic renaming or having
"real" names different from apparent names.

... says the man who implemented "foo_str" for foo"...." macros.  ;-)
 

It's neat that you can do
"a .x. b", but I don't think it's worth it.

Yes, especially since you can do "a .×. b" (i.e. use the Unicode ×) if you want a short name without conflicting with "x".

[Stefan Karpinski]

On Feb 11, 2014, at 3:02 PM, "Steven G. Johnson" <steve...@gmail.com> wrote:
>
> ... says the man who implemented "foo_str" for foo"...." macros. ;-)

That one was my idea. I seems to have less aversion to that sort of thing.

[Stefan Karpinski]

I would imagine it would only apply to non-operators. Operators, by definition, already have special syntax, so this construct shouldn't apply to them. I still prefer it without the dots, but it's a bit of a delicate syntax.

[Glen O]

In that case, how about using the same notation currently being used for range operator? That is, you'd write 2:^:3 if ^ weren't already an infix function. Similarly, you might write 17:mod:5 to evaluate mod(17,5). Right now, if you try either of these (in 0.2.1), it responds with "ERROR: no method colon(Int64,Function,Int64)".

As a result, even now, you can actually use this notation to manually create infix operators by simply adding this to ~/julia-rc.jl:

colon(x,y::Function,z)=y(x,z)

After this, 17:mod:5 produces 2, 2:^:3 produces 8, and "Hello ":print:"world!" prints Hello world!

The only binary functions I know of that you can put into that notation and NOT get the desired result are + and -, which still trigger the normal range operator (1:+:3 = 1:3, 3:-:1 = 3:-1:1). I'd suggest removing this curious behaviour of a:+:b and a:-:b (given that the former is the same as a:1:b and the latter is a:-1:b, anyway), and then building the aforementioned colon functionality into the language directly.

[Glen O]

If I may offer an alternative... there's already a way to create infix functions using an existing character that shouldn't create any particular ambiguities*, simply through adding a line to (in Ubuntu, at least) ~/.juliarc.jl - in particular, you can put the line

colon(x,y::Function,z)=y(x,z)

With this line, you can write, for instance, 2:^:3 (=8), 17:mod:5 (=2), or "Hello ":print:"world!" (which prints Hello world!). I believe it will work with any binary function except + or -, which seem to be handled separately (1:+:3 works the same as 1:3, and 3:-:1 works the same as 3:-1:1). It even works with anonymous functions - "7:((x,y)->x^2+y^2):9" outputs 130, which is indeed 7^2+9^2.

If this could be incorporated into the base code, it would address the issue cleanly and efficiently, in my opinion.

* The only possible ambiguity is with the regular ternary operator... that is, how would it parse true?"Hi":print:" there":1 - as it turns out, with the definition provided above, it outputs the text value "Hi" rather than printing "Hi there". Note that the same ambiguity already exists - true?1:2:3:5 will result in 1, not 1:2:3, while false?1:2:3:5 will result in 2:3:5.

On Wednesday, 12 February 2014 05:13:33 UTC+10, Steven G. Johnson wrote:

[Stefan Karpinski]

Sorry about that weird delay on your email, Glen. It took Google Groups a while to ask me to ok this – it thought it was spam, no idea why.

[Stefan Karpinski]

Also, that's a clever idea.

[Steven G. Johnson]

On Thursday, May 22, 2014 8:54:07 AM UTC-4, Glen O wrote:

If I may offer an alternative... there's already a way to create infix functions using an existing character that shouldn't create any particular ambiguities*, simply through adding a line to (in Ubuntu, at least) ~/.juliarc.jl - in particular, you can put the line

colon(x,y::Function,z)=y(x,z)

This is a great idea!   I've added an issue for this proposal: https://github.com/JuliaLang/julia/issues/6946

(Note that in Julia 0.3, a couple of more colon definitions are required to eliminate method ambiguities.)

[gael....@gmail.com]

What would

a :b: c :d: e

do? I know this is unambiguous for dot products but

b(a, d(c, e))

is always obvious.

I really think this needs some thoughts : to me, it seems just like "hey guys, why not doing this just because we can and it seems shinny". I would love to see what problem such suggar would help to solve. And if it's really helpful, then why not.

However, I'm definitly against creating infix functions without any distinctive syntax features. The column trick seems great in that regard.

But again, I'm not sure adding complexity and redundancy to the language is a good idea : c++ has done this for years and now it's a real mess. If someone only learns c++11 and wants to contribute to any project, he will have a really bad time trying to catch up with all the alternative "syntax" people are using. I'm not saying infix functions should not exist, only that it should be a part of "Julia's way of solving problem", not a shinny stuff that make programs harder to understand. If it can't then it might not be a good idea after all.

I know that most of you focus on ease of use, but we are in the 21st century. Let's stop doing approximative science on our own. Let's share our progress and more importantly, built upon what already exists instead redoing everything from scratch each time.

This can't be done with ugly source code for which the incentive to rebuild everything from scratch is very high. That's why code readability should be considered just as important as ease of use.

Fortran is simple but some of its design choices made it unreadable.

Even personal, quickly writen python scripts are readable which make their translation to a proper lib a painless operation. If Julia can't provide that, it's gonna follow Fortran trend : successful but painful to reuse. There will be some very high quality libs available and aside from that, spaghetti.

[Glen O]

The usual standard is that operators at equal precedence run left-to-right. So a :b: c :d: e would be (a :b: c) :d: e, and thus d(b(a,c),e) - just as 25/5/5 evaluates to 1.0, not 25.0. And in the majority of situations, this is the behaviour that users would expect. Interestingly, the colon operator is lower precedence than addition. What this means is that 4+7 :*: 3+9 would actually be same as 11*12. I just mention this as an aside.

The idea behind having an infix operator is that, in many situations, the roles of the two inputs to the function are asymmetric, in the sense that one is the input being operated on, and the other is part of the operator itself. For instance, modulo. mod(13,7) is less clear than 13 mod 7.

A bit of redundant coding choice is good for code readability - it means that the programmer can, if they're paying attention, choose the best way to write their code in order to make it as readable as possible. If the programmer is thinking about it, they probably aren't going to write x :div: 13 :mod: 11, but rather, (x :div: 13) :mod: 11. And they'll write print("Hello ", "world!") rather than "Hello ":print:"world!", because the former is clearer than the latter. Same as how we use 3+4 rather than +(3,4), because 3+4 is much clearer. In a similar sense, I often construct ranges by writing 1:(x+1), even though 1:x+1 evaluates the same... because it avoids the ambiguity with (1:x)+1. But having the choice to use 1:x+1 means that, in some cases, clearer code can be written.

Of course, it's important to ensure that it's not 100 different ways to write the same thing. Basically, it's a matter of balance. And I'd suggest that infix functions improve the balance.

[gael....@gmail.com]

You see, I'm more used to right to left evaluation... :) I guess I'm not the only one.

I understand for :mod: but isn't it more an exception than the rule? That's why many languages use % as the modulo operator. Having a few infix functions does not mean that user defined infix operators should be allowed. I'm not well versed in mathematics but to my knowledge, only very few asymetric operations are widely used. All I'm saying is that Julia would become more complicated with that and I'm wondering if the extra complexity is worth it.

Regarding code clarity, it's pretty easy to craft a very clear, understandable and maintainable library in Fortran. However in general, code base in that language are ugly.

It is not about letting users writing clear code, it's about letting non computer scientist, time-constrained users writing a code as clear as possible from the start. Allowing clear constructs is not enough, those clear constructs should be considered the default, fast, obvious Julia way's to do things. In that regard adding redundancy might be a very good thing (your modulus example is very good), but is it worth it as a new language feature?

[Kevin Squire]

On Sat, May 24, 2014 at 9:12 AM, <gael....@gmail.com> wrote:

In that regard adding redundancy might be a very good thing (your modulus example is very good), but is it worth it as a new language feature?

From https://github.com/JuliaLang/julia/issues/6946, the consensus seems to be "no". :-)

Cheers,

   Kevin

[Will]

Pvnpo

[Jameson Quinn]

I'm reviving a very old thread here, about the possibility of user-defined infix operators.

In R, you can define infix operators by creating a function with a name like %this%. Why couldn't we do something similar in Julia? That is, have 

   a %whatever% b 

parse to 

   whatever(a,b)

?

I think that % is a relatively safe character for this use. There's not much use for "a % b % c" if you're using it as "mod", so people are unlikely to accidentally put an identifier between two "%" signs with no whitespace.

This would be obviously useful for DSLs.

[Diego Javier Zea]

Could we use _ instead of % ? a _whatever_ b looks cleaner than a %whatever% b.

[Kristoffer Carlsson]

But  _whatever_  is a valid variable name.

[Jameson Quinn]

The proposals for a delimiter character so far are:

(from the old thread)

:whatever: ... there was a cute idea for implementing this, but it was pointed out that this would create ambiguity for ranges (0:2:10)

.whatever. ... fortran precedent; but also creates ambiguity; a .b .2 is different from a .b. 2

`whatever` ... not too ugly, but potentially conflicts with current use of backticks.

(from https://github.com/JuliaLang/julia/issues/6946)
unicode such as ‖whatever‖ or ⋮whatever⋮ ... avoids issues with ambiguity, and potentially could be done with just one of whatever character akin to :whatever. However, there's no precedent for this kind of thing (unless someone can find a math usage that is relevant?), and since a lot of the use of this would be for DSLs, it seems strange to force people to use unicode.

(now)

_whatever_: valid variable name

%whatever%: R precedent. Potentially conflicts with stuff like 20 % 11 % 4, but that is not very useful as it is.

There haven't been any proposals for something like 

    a %whatever b

or

    a whatever@ b

, that is, one character (which might save a keystroke). But there's certainly no ascii character that's "free" for using like that, and I think that the readability and R precedent of %whatever% compensates for the extra keystroke and ugliness.

I don't think there would be any call for stuff like %(operatorfactory(".*[Gg]rep"))%, though I guess there is an argument that it's a slippery slope to such ugliness.

I think that the natural expectation would be that 

    a %whatever% b %whatever% c

would parse to

    whatever(a,b,c)

by analogy with "+", but that would almost certainly be bad programming style in my book, and we could explicitly discourage it. (Although I guess you could argue that 

   f %curriedwith% arg1 %curriedwith% arg2

would be natural)

R calls these things "special binary operators", and gives them precedence in between "^" (exponentiation) and "<" (less than).

[Glen O]

I don't recall where I did it, but I proposed another option that works pretty well, and only extends an existing notation.

At present, we can chain functions in a left-to-right notation using |> as

15 |> sin

So we can create a natural inbuilt function notation by simply adding <| so that we can write

15 |> mod <| 4

which would end up parsing as mod(15,4). And it would require nothing more than a definition of <| as

<|(a,b) = i->a(i,b)

and making <| have slightly higher precedence than |>

The only possible downside is that |> has quite a low precedence, below +. It would be better if precedence was above +, at least. Right now, if you do 100 + 14 |>mod<|10, you'll get 4, rather than 104.

Perhaps a second notation could be applied to carry the higher-precedence version?

The benefit of |>func<| is that it stands out, matches existing usage, and is easily recognised as a function setup.

[Jameson Quinn]

I like the readability of |>op<|, but I don't like the tricks like

    <|(a,b) = i->a(i,b)

 Finally, it allows defining functions in their infix form directly:

    a::Integer 

What about

    >% whatever %<

? 

Pros:

-riffs off both the `|>` pipe and the R syntax

-allows `whatever` to be an expression and not just an identifier

-no need for white space to be significant (unlike %whatever%)

-syntax error, rather than obscure bug, if you get it slightly wrong

-Could allow for ternary operators:

    func >% curriedwith %< arg1 %< arg2

-allows defining functions in their infix form directly:

    a::Integer >% umod %< b::Integer = ((a % b == 0) ? b : (a % b)) #Bad example but you get the idea.

Cons:

-hard to type

-Some people find `%` ugly.

[Jameson Quinn]

Whoops, I glitched editing that email. The first paragraph, instead of saying:

On Tuesday, 14 June 2016 09:18:49 UTC-4, Jameson Quinn wrote:

I like the readability of |>op<|, but I don't like the tricks like

    <|(a,b) = i->a(i,b)

 Finally, it allows defining functions in their infix form directly:

    a::Integer 

should have said instead: 

I like the readability of |>op<|, but I don't like the tricks like
    <|(a,b) = i->a(i,b)

because doing it in the parser directly avoids function overhead, makes combining with macros for DSLs easier, and allows defining functions in their infix form directly:

[Scott Jones]

That does sound rather good to me, even though I wasn't accustomed to the R syntax.

One question, why '>% whatever %<', instead of '<% whatever %>' or '%< whatever >%' ?

Personally, I think having the the angles < > is much more readable, because we are used to using them in pairs, like (), [], {}.

[Marco Matthies]

On Tuesday, June 14, 2016 at 2:11:52 PM UTC+2, Jameson Quinn wrote:

The proposals for a delimiter character so far are:

[...]

In my ideal world, \operator would allow for custom operators (like in LaTeX), and you could specify precedence like in Haskell.

And function chaining would use | (like Unix shell).

Both of these clash with current usage of symbols of course.

In practice i'm fine with |> and whatever people can agree on for named operators, as long as we can have them :-)

[Scott Jones]

Cons:

-hard to type

Much easier to type than any of the Unicode character proposals above, I don't really think this is at the level of being a con.

 

-Some people find `%` ugly.

Personal preferences, but hopefully they'd be won over by the number of pros that you have listed. 

[Jameson Quinn]

One question, why '>% whatever %<', instead of '<% whatever %>' or '%< whatever >%' ?

Personally, I think having the the angles < > is much more readable, because we are used to using them in pairs, like (), [], {}.

The idea is that  `a >% whatever %< b` reads as "send a and b in towards whatever" a la pipes, while `a <% whatever %> b` reads as "a, chunk of whatever, b" a la html. I'm kinda agnostic about which is better in that case, but if we're going to allow the ternary/quaternary/etc syntax I suggested (`a >% whatever %< b %< c` for `whatever(a,b,c)`) then I think the former is definitely better. 

[Jameson Quinn]

In my ideal world, \operator would allow for custom operators (like in LaTeX),

Hmm... that's pretty readable, very easy-to-type in principle, and it does respect a precedent most Julia programmers would know. On the other hand, I'm wary of creating a parser which can't use the regex tradition of using backslash for single-character escaping, and it seems as if it would be hard to extend to the more sophisticated usages that I've suggested, like having the infix thing be an expression, or the ternary usage. 

 

and you could specify precedence like in Haskell.

My strong intuition is that this is a bad idea. You do not want the parser's behavior to depend on prior configuration; it's both a performance nightmare and a source of deeply-encrusted bugs.

 

And function chaining would use | (like Unix shell).

I sympathize, but that ship has sailed, and anyway it's not related to the current discussion. 

[Jameson Quinn]

In my ideal world, \operator would allow for custom operators (like in LaTeX),

I forgot one more possible argument against this: in editors which tab-complete LaTeX to unicode, this might get in the way?

[Scott Jones]

OK, I understand the point, but I'm not sure that people would use that ternary/quaternary/etc. syntax, since in this case, they can simple call `whatever(a,b,c)` with less typing,

so I'd lean towards having something like `a %<whatever>% b` and only having it supported as a binary operation.

Whichever way it goes, either the above or `a %>whatever<% b`, I do really like your suggestion to use `%<` and `>%`.

[Glen O]

You call it a trick, but I think it's actually quite a neat property, because it allows you to do things like, say,

  lpad<|3

to create a function that pads a string on the left to length 3. So if you had an array of strings, and you wanted to pad them to the same size, instead of saying

  map(i->lpad(i,10),myarray)

you can say

  map(lpad<|10,myarray)

which, to me, is a lot cleaner. Similarly, you have map(mod<|12,myarray), which is a lot neater than map(i->mod(i,12),myarray).

Mind you, I'm not actually suggesting that, in the long term, it go through the function definition I provided. That would be the short-term implementation, until the parser is updated. 

[Stefan Karpinski]

Just to set expectations here, I don't think there's going to be much in the way of "syntactic innovation" before Julia 1.0. (The only exception I can think of is the new `f.(v)` vectorized calling syntax.) While having some way of making arbitrary functions behave as infix operators might be nice, it's just not a pressing issue in the language.

[Scott Jones]

Right, there are much more important things going on to make it to Julia 1.0.

A nice thing I think about Jameson's idea (whether `<%op%>`, `%<op>%` or `>%op%<`) is that it could be added post-1.0, as it doesn't introduce any incompatibilities.

[Jameson Quinn]

On Tuesday, 14 June 2016 11:25:30 UTC-4, Scott Jones wrote:

Right, there are much more important things going on to make it to Julia 1.0.

Fair enough. I'm a nooB here so I don't know what the roadmap is, but I understand that this isn't a drop-everything idea. However, If it's just a matter of "energy to implement", I myself could make a patch to put any of the below options into the parser and create a few test cases. 

 

A nice thing I think about Jameson's idea (whether `<%op%>`, `%<op>%` or `>%op%<`) is that it could be added post-1.0, as it doesn't introduce any incompatibilities.

Right.

Here's a list of options and #rationale comments:

`<%op%>` #percent inside a la R, angle brackets outside a la html

`>%op%<` #percent inside a la R, direction outside a la julia pipes

`%>op<%` #just like julia pipe, but replace pipe symbol with percent

`/>op</` #tilty pipe, easy to type
`/|op|/` #clockwise vortex! Involves only two characters.

...etc.

The differences between the above options are pretty unimportant and I think a long thread of bikeshedding is probably unhelpful, so I'd happily hand the decision of which of the above (or anything else) is best to some benevolent dictator (Stefan?). If I get such a ruling, the next thing I'd do would be to create an issue, so we can move the roadmap discussion and/or any proposed patches over there.

[Diego Javier Zea]

```julia

(a f b)

```

is a clean and available syntax.

--

Diego Javier Zea

Departamento de Ciencia y Tecnología
Universidad Nacional de Quilmes
Roque Saenz Pena 182
1876 Bernal
Argentina
Phone:+541143657100 ext 4135

[Jameson Quinn]

I like the cleanness of `(a f b)` parsing to `f(a,b)`. Also, it seems to me that the right precedence is more obvious than with the options like `a <% f %> b`. What are the downsides?

-Potential for WTF bugs if you forget commas like `f(a b c)`

-Something like (a + b f c ^ d) could be pretty unreadable. On the other hand, any reasonably-complicated syntax can be used perversely; the answer may just be, don't.

-Slippery slope: people might expect `(a f)` and `(a f b c)` to parse to `f(a)` and `f(a, b, c)` respectively. But again: when you see a rabbit hole, don't go down it.

-if you named f poorly, it could be hard to read. But if not — if you used an operator-looking unicode character or a known text naming convention — then it's fine.  

I think the first of those objections is the only serious one. Is there a similar syntax that avoids it? `{a f b}` or, for extra yes-I-really-mean-it emphasis, `{{a f b}}` would work.

I've thought a bit about how to tweak the parser to handle any of the options we've discussed. I realize it would not be trivial, but I think it could be done cleanly. 

I said earlier that I was just waiting for Word of God on best syntax to take this discussion to a new issue. But I think that the two overall styles — `a <% f %> b` and friends versus `(a f b)` and friends — are different enough that a little more discussion is worthwhile.

[Scott Jones]

One problem with that syntax is that it would probably have all Scheme/Lisp/Clojure/Racket/etc. fans pulling out their hair, since they'd constantly be wanting to read that as "apply a to the arguments f and b".

I think it would be especially bothersome, since Julia is a kind of a "lisp-1", in that functions and data live in the same namespace, so if you are wanting to pass a value which happens to be a function to another function

(quite common in Julia), being able to immediately see *which* is being applied, and which is being passed, is important.

[Jameson Quinn]

I agree, the "don't melt the poor Lisper's minds" argument is telling. So I'll go back from being neutral, to preferring the ternary-style syntax as in

    ```

    a <% f %> b

    ```

Of the various ternary ideas I listed earlier, I think my favorite is the "tilty pipe" one: `a /> f </ b`.

In terms of precedence, this is a strange one. It seems reasonable that, as in R, this would have precedence somewhere above addition, so that `100 + 14 /> mod </ 4` would be 104, not 4. But then consider `a + b /> f |> g </ c + d`. 

If that's not a syntax error, it would be logical for it to parse to `a + (f |> g)(b,c) + d`. That is, the precedence for the thing in the middle should arguably be very low; probably as low as the existing ?: ternary operator, but the precedence at the edges should be above `+`.

Getting the parser to do that wouldn't be that hard. But explaining it to a new programmer kinda is. On the other hand, if the precedence is the same across the board, then `a ^ b /> f ^ g </ c ^ d` would be legal but `a + b /> f + g </ c + d` would be a syntax error, which is also kinda strange. 

Is that what we want to do? Or is it better to scale this proposal back to just `a %identifier% b`, with no whitespace allowed around `identifier`; that is, directly parallel to R, as I suggested originally?

[Tamas Papp]

Not pulling out my hair, just scratching my head about this: why would
one need infix syntax for operators longer than a single character?

Ie in what context would 14 /> mod </ 4 or one of the proposed
variations be preferable to mod(14,4)?

I understand that people want to take advantage of all those fancy
unicode characters for operators, but all the proposals for applying
multi-character function names as infix just look convoluted. The
advantages of infix are convenience and clarity, and without those, it
loses a lot of appeal.

On Tue, Jun 14 2016, Scott Jones wrote:

> One problem with that syntax is that it would probably have all
> Scheme/Lisp/Clojure/Racket/etc. fans pulling out their hair, since they'd
> constantly be wanting to read that as "apply a to the arguments f and b".
> I think it would be especially bothersome, since Julia is a kind of a
> "lisp-1", in that functions and data live in the same namespace, so if you
> are wanting to pass a value which happens to be a function to another
> function
> (quite common in Julia), being able to immediately see *which* is being
> applied, and which is being passed, is important.
>
> On Tuesday, June 14, 2016 at 1:14:56 PM UTC-4, Diego Javier Zea wrote:
>>
>> ```julia
>> (a f b)
>> ```
>> is a clean and available syntax.
>>
>> 2016-06-14 12:55 GMT-03:00 Jameson Quinn <jameso...@gmail.com

>> <javascript:>>:

[Glen O]

It's about readability. "14 mod 4" is the natural way, mathematically, to express the modulo operation. If we had "choose" as an alias for "binomial", then you might say "15 choose 7", and that would be a lot easier to understand than "choose(15,7)". Basically, it's for the same reason as why we don't write "-(4,5)" - because "4-5" is easier to interpret from a human standpoint.

It's like why the |> operator was introduced.

My issue with the other proposed methods is that they're about making a special infix operation that is specifically for infixing, which means that they provide no other benefits - if it's going to require multiple characters, it needs to provide more than just the one use. I stand by my suggestion of having <| as a new operator, designed to pair with |> for infix and otherwise work nicely to create a new function expression.

Incidentally, I'd probably also add the counterpoint operators:   >| and |<, to the set, with analogous definitions. That is, a>|b would be parsed as i->b(a,i) and a|<b would be parsed as a(b). The last one would mostly be nice for situations where you want to apply a naturally prefix operator to an expression, like sin |< x+1

It allows for some nice readable code, like map(lpad<|3,myarray), or map(15>|choose,myarray). And it's consistent - pipe first for evaluation, pipe last for function. Note the relative complexity of the equivalents at the moment - map(i->lpad(i,3),myarray) and map(i->choose(15,i),myarray).

I just realised that I left out a detail when proposing a way to implement it without modifying any parser code. I suggested <|(a,b)=i->a(i,b)

But it should be <|(a,b...)=(i...)->a(i...,b...) to allow it to handle more general cases.

[Jameson Quinn]

I agree with Glen that anything we do should be general-purpose, and have more potential applications than just binary infix operators. I'm going to propose something different from Glen's idea; then ask a question or two about his idea; then discuss his idea.

First, my proposal: that `%op%` marks the function. So you could have:

    `a %op% b` --> `op(a,b)` #infix
    `a %op% b c` --> `op(a,b,c)` #innfix
    `%op% a b` --> `op(a,b)` #vaguely coffeescript-like
    `a b c %op%` --> `op(a,b,c)` #forth-like, aka German style

Obviously, if you were doing this somewhere that whitespace separation had another meaning, such as an array literal, you couldn't have more than one argument on either side. So 

    `[a %op% b c]` --> `[op(a,b) c]`
    `a b %op% c d %op2% e f` --> `op(a,b,c,op2(d,e,f))`

This is still much more of an "empty calorie" proposal than Glen's; it's syntactic sugar without the meatiness. But it allows more readable code, especially for DSLs. As a stopgap, it could be implemented by a macro which would later turn into a no-op:

    `@inf a %op% b c` --> `op(a,b,c)`

    `@inf 2^n % 10 %op% b` --> `10(2^n,op % c)` #oopsie. Can't win 'em all.

....

Questions about Glen's proposal:

-Glen says that "it should be <|(a,b...)=(i...)->a(i...,b...) to allow it to handle more general cases." But wouldn't that imply a parser change, so that

    `a b |> f` --> `f(a,b)` #or were you thinking of commas? I'd weakly prefer spaces myself...

    `a b |> f <| c d` --> `f(a,b,c,d)`

?

-In Glen's proposal, there would superficially appear to be extra function overhead:

    `a |> b <| c` --> `((xxx) -> b(xxx,c))(a)` #not just `b(a,c)`

Is that extra overhead optimized out by the compiler? What about if we're still in the stopgap situation where

    `a |> b <| c` -parser-> `(<|(b,c))(a)` -interpretsto-> `((xxx) -> b(xxx,c))(a)`

Note that even if it is compiled, it is still significant in one sense: you can't say, for instance, `(a |> normby <| c) = sum(map((x)->x^c,a)`. I think that it would be nice to be able to define infix operators in their infix form like that, which is enabled by my `%op%` proposal above; but as far as I understand, enabling this with Glen's proposal would mean an unholy melding of parser and compiler and thus would be a bad idea.

....

Discussing his idea: despite the reservations I expressed above, I like Glen's idea. It enables some nice, intuitive idioms (as well as, of course, some mind-melting ones). 

There are two more possible operators that would round out the set. I'm not actually proposing these, just noting that they're possible:

    `a <<| b` --> `(x)->a(b,x)` #first argument last

    `a >>| b` --> `(x)->b(x,a)` #last argument first

I think I slightly prefer my `%op%` proposal, because Glen's might be a temptation towards showy abuses, whereas mine is clearly more limited in scope. But I'd be satisfied with either. I like the fact that my proposal can be implemented by a macro starting now (modulo the oopsie above), and that it allows defining operators using the infix notation and not just using them thus.

[Glen O]

To answer your questions...

The <|(a,b...)=(i...)->a(i...,b...) definition can actually be used right now - it can be done in Julia without any changes to the parser. To use the "b...", you need to use ... in the call, at the moment (so you might say 5|>(max<|(2,6)...) to make it equivalent to max(5,2,6)). It's not idea, and a parser change would be necessary to make it all work nicely... but that's true of any other option to implement a notation that permits a ternary operation with a single operator.

The (i...) part works far more neatly, and you can do something like map(max<|10,array1,array2) and have it work fine.

At the moment, you're right, it would have extra overhead. But that's because it's being defined within the language, rather than in the parser. Obviously, longer-term, it would be shifted to parser form. It's also worth noting that, as far as I can see, |> is already being defined in this way - in operators.jl, it's defined as "|>(x, f) = f(x)". I'm not sure whether the compiler simplifies this out, nor whether it would be able to do so with <|, but the point is, if there's any optimisation issue, it can be dealt with later.

Note that I would like to see a further parser adjustment, ultimately - it would be nice to make n-ary operations work using something like a|>func<|b<|c to evaluate to func(a,b,c). Right now, if the only change is giving <| higher precedence than |>, then it would evaluate to func(a,c,b).

As for the suggestion of defining infix operators in an infix form... that's purely a matter of preference, and to me, infix operator definitions of this sort would be more of an obscuring thing than a clarifying thing, no matter what the notation itself is. That said, for future use, I'd suggest having a (>func<) b = ... to define a function in an infix notation, if it's necessary. It helps to keep the definition and usage notations distinct.

You mention the possible <<| and |>> operators to provide modified orders. I prefer maintaining a standard, such as >| for inserting a variable at the start and |< for evaluating at the end. That way, pipe first means evaluation, and pipe last maintains it as a function, while the > and < indicate which one is the function.

Concerns with your proposal are:

- Confusion of notation, with % being used for multiple meanings, which might conflict. If you want to perform the remainder operation twice, (a%b)%c could easily be written as a%b%c - and that is easily confused with a %b% c - this is precisely why my original suggestion of a:b:c was rejected, because what it does depends on what b is, and they're very different depending on that choice.

- Major parser alterations, because the %op% notation doesn't exist at this time, whereas my proposal can be implemented functionally with a single line of code and two small edits to the parser's precedence list (namely, editing the line in julia-parser.scm that says "(define prec-pipe '(|\|>| |<\||))" to be two lines, with "(define prec-pipe '(|\|>|))" followed on the next line by "(define prec-pipe-left '(|<\||))", then editing the "(define prec-names" section to add "prec-pipe-left" after "prec-pipe". This fixes the fact that |> and <| are currently equal precedence, and thus evaluate left-to-right, whereas we need it to go the other way.

- If you have A=[sum,prod,maximum], you'll run into some issues with the idea of %A[1]%, as it's not clear how it should be read. This could be a real issue if someone wants to define % for their variable types, and can take a function as an argument.

- Needing whitespace to distinguish between uses of % could lead to some difficult to find bugs in code.

[Jameson Quinn]

Replies inline.

On Wednesday, 15 June 2016 11:37:28 UTC-4, Glen O wrote:

To answer your questions...

...

You mention the possible <<| and |>> operators to provide modified orders. I prefer maintaining a standard, such as >| for inserting a variable at the start and |< for evaluating at the end. That way, pipe first means evaluation, and pipe last maintains it as a function, while the > and < indicate which one is the function.

You've misread my suggestion. I suggested >>|, not |>>; which already answers your objection.

 

Concerns with your proposal are:

- Confusion of notation, with % being used for multiple meanings, which might conflict. If you want to perform the remainder operation twice, (a%b)%c could easily be written as a%b%c - and that is easily confused with a %b% c - this is precisely why my original suggestion of a:b:c was rejected, because what it does depends on what b is, and they're very different depending on that choice.

That's true, but I have a hard time imagining when you'd want a hard-coded number of back-to-back remainder operators other than 1 of them. I guess you could have two consumers taking different-size chunks off of a stream, and then want to know how much will be left over... but in that case, you'd probably want to generalize to n consumers, and the problem would go away.

 

- Major parser alterations, because the %op% notation doesn't exist at this time, whereas my proposal can be implemented functionally with a single line of code and two small edits to the parser's precedence list (namely, editing the line in julia-parser.scm that says "(define prec-pipe '(|\|>| |<\||))" to be two lines, with "(define prec-pipe '(|\|>|))" followed on the next line by "(define prec-pipe-left '(|<\||))", then editing the "(define prec-names" section to add "prec-pipe-left" after "prec-pipe". This fixes the fact that |> and <| are currently equal precedence, and thus evaluate left-to-right, whereas we need it to go the other way.

On the other hand, my suggestion (at least, the infix part of it) could be done by a macro, precisely because the % operator already exists; while yours, I think, couldn't (though I don't understand why  <| already has a precedence if it doesn't exist).

- If you have A=[sum,prod,maximum], you'll run into some issues with the idea of %A[1]%, as it's not clear how it should be read. This could be a real issue if someone wants to define % for their variable types, and can take a function as an argument.

I think it would be OK if %A[1]% were illegal as an infix, or if it were legal. In either case, the only reason you'd have this in existing code would be for successive applications of `%`. So this is either the same issue as above (two hard-coded applications of the remainder operator), or it is an unexpected redefinition of the operator so that it works on functions. It seems to me very strange to want to use an operator which has the same precedence as an arithmetic operator, for functional stuff... and then, on top of that, not using whitespace... but OK, I guess it's possible that such code exists in the wild. So my proposal would have to go in two stages, first deprecating back-to-back remainder operators without spaces, and then adding the new behavior.

 

- Needing whitespace to distinguish between uses of % could lead to some difficult to find bugs in code.

To some degree, this is just a tradeoff between terseness and debuggability. It seems to me that the Julia philosophy is to try to get both, but to be willing to sacrifice some corner-case debuggability if necessary to preserve terseness. (For instance, `a + b = 3` could be a very nasty bug, and Julia so far has lived with it.) Also, as long as you don't use infix %ops% AND redefine % to work on functions, I think the error messages would lead you pretty directly to the problem. Finally, I know that "proposal A vs proposal B" is a false dilemma because we can always do neither, but it is not at all hard to imagine hard-to-find bugs using your proposal (starting with, "I put the pipe on the wrong side of the arrow").

[Jameson Quinn]

On the other hand, my suggestion (at least, the infix part of it) could be done by a macro,

Here's a hacked-up version of that macro. It doesn't work on things like `a %op% b + c`, but that could be fixed.

```julia

    using Base.Meta

    function _parseinfix(ex)

      (ex,false)

    end

    function parseinfix(ex)

      _parseinfix(ex)[1]

    end

    function _parseinfix(ex::Expr)

      if (isexpr(ex,:call)

          && ex.args[1] == :%

          && isexpr(ex.args[2],:call)

          && ex.args[2].args[1] == :%)

        return (:($(ex.args[2].args[3])($(ex.args[2].args[2]),$(ex.args[3]))),

                true)

      end

      parsedargs = map(_parseinfix,ex.args)

      (Expr(ex.head,map(first,parsedargs)...),parsedargs[1][2])

    end

    function addargs(ex::Expr,left,right)

      return Expr(ex.head,ex.args[1],left...,ex.args[2:end]...,right...)

    end

    macro %(args...)

      if length(args) == 1

        return parseinfix(args[1])

      end

      parsedargs = map(_parseinfix,args)

      for (i, (parsedarg, changed)) in enumerate(parsedargs)

        if changed

          return(addargs(parsedarg,map(first,parsedargs[1:i-1]),

                                map(first,parsedargs[i+1:end])))

        end

      end

      throw("multiple arguments but I can't find the governing %operator%")

    end

``` 

[Glen O]

Regarding the multiple % usage, it's certainly not going to be common, but you really don't even want a corner case to be vague when it comes to a parser. Beyond that, suppose you have a%b %c% d. Suddenly, it's not obvious at a glance what is going on.

The operator <| already exists... it just isn't defined. It's set up so that it *can* be used as an operator, but if you try to use it without defining it, it will throw an error. The concern isn't existence, it's precedence. The logical option, when setting up the |> operator, was to make <| have the same precedence. But with the current precedence, a|>b<|c gets interpreted as (a|>b)<|c, which means it become i->b(a)(i,c), which is clearly not the intended interpretation.

Regarding the "I put the pipe on the wrong side of the arrow", I don't see that as likely. |> is hard to confuse with >|, and similarly, <| is hard to confuse with |< - they look completely different, the difference between a triangle and a K. It's much different from the comparison between a%b %c% d and a %b% c%d. Whitespace isn't a good way to distinguish between two uses of an operator, when those uses are dramatically different but could be applied to the same objects.

One example of where someone might define % used with a function is functional modulo. For instance, (x->x^3-3x^2+2x-1)%(x->x^2-1) would evaluate to 2x-3.

[Jameson Quinn]

On Wednesday, 15 June 2016 13:52:29 UTC-4, Glen O wrote:

Regarding the multiple % usage, it's certainly not going to be common, but you really don't even want a corner case to be vague when it comes to a parser. Beyond that, suppose you have a%b %c% d. Suddenly, it's not obvious at a glance what is going on.

I think that if %op% is used, then the % operator without whitespace on either side should be deprecated.

 

The operator <| already exists... it just isn't defined. It's set up so that it *can* be used as an operator, but if you try to use it without defining it, it will throw an error. The concern isn't existence, it's precedence. The logical option, when setting up the |> operator, was to make <| have the same precedence. But with the current precedence, a|>b<|c gets interpreted as (a|>b)<|c, which means it become i->b(a)(i,c), which is clearly not the intended interpretation.

OK, right. 

Regarding the "I put the pipe on the wrong side of the arrow", I don't see that as likely. |> is hard to confuse with >|, and similarly, <| is hard to confuse with |< - they look completely different, the difference between a triangle and a K. It's much different from the comparison between a%b %c% d and a %b% c%d. Whitespace isn't a good way to distinguish between two uses of an operator, when those uses are dramatically different but could be applied to the same objects.

I think they're easy to confuse, not visually, but mentally. And that's just the start of the problems this could bring up. If you're good at debugging Lisp, this stuff is child's play; but I'm sure that there's subtle bugs that this could bring. My proposal is less powerful, and so gives less chance to shoot yourself in the foot.

(again, I'm not really opposed to your proposal, just taking a side for stress testing)

 

One example of where someone might define % used with a function is functional modulo. For instance, (x->x^3-3x^2+2x-1)%(x->x^2-1) would evaluate to 2x-3.

Gah! I guess you could probably actually do that using some trick like automatic differentiation, but wouldn't it be so much easier with your polynomials as expressions? 

But fine, I concede the point. Again, for the long term, the solution would be to deprecate %-as-an-operator without whitespace. That's not very painful if stays works-but-warns for a reasonably long period.

I think such deprecation would be reasonable for 1.0, even if it were documented only ("will probably be deprecated") and not rolled into the parser yet.

So we have two proposals:

%op%:

Pros: 

-can be hacked up with a macro

-fewer keystrokes

-once it's clearly documented, it's hard to misuse

-direct parallel to R

-"lightweight" (in that it solves just the one issue)

-allows infix definition style naturally

Cons:

-Requires deeper parser changes

-Could conflict with existing use of %, though that's likely to be rare

|> op <|

Pros:

-creates nice "curry" and "apply" operators which have other uses

-logically clean

-more power. For instance, you can put an expression in the middle. (But is this a good idea? Isn't it like trying to define a new verb in the middle of a sentence?)

Cons:

-function call overhead. In principle this can be handled by the compiler but in practice that seems as if it would take some ugly mixing of responsibilities between compiler and parser, so it would likely not come soon.

Personally, I think it would be easier for a naive coder reading a module to confidently guess a meaning for "10 %choose% 4" than for "10 |> choose <| 4". Those pipes look sneaky.

[Scott Jones]

On Wednesday, June 15, 2016 at 3:09:42 PM UTC-4, Jameson Quinn wrote:

On Wednesday, 15 June 2016 13:52:29 UTC-4, Glen O wrote:

Regarding the multiple % usage, it's certainly not going to be common, but you really don't even want a corner case to be vague when it comes to a parser. Beyond that, suppose you have a%b %c% d. Suddenly, it's not obvious at a glance what is going on.

I think that if %op% is used, then the % operator without whitespace on either side should be deprecated.

That might break a lot of code.   If you are doing bit twiddling work in Julia (which I do!), you end up using the idiom `value%type`, typically with Type being UInt8/16/32 or Int8/16/32, which kind of "casts" the value to the given type, but without the `InexactError` which can occur using the constructor (i.e `Int8(1000)`).

[Jameson Quinn]

I think that if %op% is used, then the % operator without whitespace on either side should be deprecated.

That might break a lot of code.   If you are doing bit twiddling work in Julia (which I do!), you end up using the idiom `value%type`, typically with Type being UInt8/16/32 or Int8/16/32, which kind of "casts" the value to the given type, but without the `InexactError` which can occur using the constructor (i.e `Int8(1000)`).

Good argument.

So the issue of, say, `a %choose% b%c` is a real one.

I think we'd at least want to deprecate the use of % as an operator with whitespace on precisely one side. In that case, the above "problem" code is at least unambiguous, and even `a %long_operator_name% b` is clear from either side.

Conversely, we'd want `%op%` to require whitespace on either side.

With these two rules, and with reasonably judicious names for operators (verbs or single operator-ish unicode characters), I think you'd have readable and debuggable code, even when mixing infix operators with %-as-modulo.  

[Diego Javier Zea]

I don't know exactly how infix function are declared right now. Could some kind of function metadata be used to indicate that infix notation if possible for a function and a @infix macro to be used in function declaration?

```

@infix function f(...) = ...

a f b

```

[Jameson Quinn]

I don't think that an @infix macro (basically acting like a python decorator) would work. The point is that the parser should not have to keep track of the state of the program; it should be able to simply parse. A given line of code should either be a syntax error, or not, independent of what has come before.

[Glen O]

Your main concern regarding |> <| seems to be the "functional overhead".

Eventually, I would expect this to be handled by the parser automatically, in the same way that double-colon is - it detects the double-colon and automatically interprets it as a ternary operator rather than the regular infix operator.

In the shorter term, if we just modify the "parseinfix" function for your macro, we can make it work for |> <|, and the benefit is that it's not changing the meaning of anything. That is, if it detects a|>(b<|c) - (brackets necessary until the precedence is fixed), it will alter it to be in regular function form b(a,c), but if you forget or intentionally leave off the macro, the code will still work correctly - it will just be a bit slower. Here's the macro for this (just a modification of your code):

```julia

    using Base.Meta

    function _parseinfix(ex)

      (ex,false)

    end

    function parseinfix(ex)

      _parseinfix(ex)[1]

    end

      function _parseinfix(ex::Expr)

         if (isexpr(ex,:call)

             && ex.args[1] == :|>

             && isexpr(ex.args[3],:call)

             && ex.args[3].args[1] == :<|)

           return (:($(ex.args[3].args[2])($(ex.args[2]),$(ex.args[3].args[3]))),

                   true)

         end

         parsedargs = map(_parseinfix,ex.args)

         (Expr(ex.head,map(first,parsedargs)...),parsedargs[1][2])

       end

    function addargs(ex::Expr,left,right)

      return Expr(ex.head,ex.args[1],left...,ex.args[2:end]...,right...)

    end

    macro infixed(args...)

      if length(args) == 1

        return parseinfix(args[1])

      end

      parsedargs = map(_parseinfix,args)

      for (i, (parsedarg, changed)) in enumerate(parsedargs)

        if changed

          return(addargs(parsedarg,map(first,parsedargs[1:i-1]),

                                map(first,parsedargs[i+1:end])))

        end

      end

      throw("multiple arguments but I can't find the governing %operator%")

    end

```

Note that I changed the name of the macro from @% to @infixed. Now, if you use @infixed 16|>(mod<|5), it will convert it into mod(16,5)... but whether you put "@infixed" at the front or not, it will evaluate to 1 (assuming that you have <|(a,b)=i->a(i,b) defined).

[Jameson Quinn]

On Friday, 17 June 2016 03:49:18 UTC-4, Glen O wrote:

Your main concern regarding |> <| seems to be the "functional overhead".

Yes, "functional overhead" was the main concern, and I think you've addressed that. I had some subsidiary concerns, though, too.

- function definition syntax: your points about the macro and parseability resolve this, too.

- operator precedence: my suggestion naturally has the same precedence as *, which is probably about where you want most newly-constructed operators to fall. The pipe operator naturally has a much lower precedence. However, you could argue that that's good, because it encourages clarifying parentheses.

- number of keystrokes and readability (partially through familiarity from R, but partially also just a subjective "what does it look as if that means"). This is pretty weak tea unless there's a chorus of people who feel the same way. Also, I personally would be much assuaged if » and « were synonyms for |> and <| (those are easy to type on my keymap).

So basically, I'm becoming convinced that your proposal is better, and I'm left with nothing more than a little residual wistfulness for the keystrokes saved by my proposal.

I've written this up as an issue: https://github.com/JuliaLang/julia/issues/16985

ps. I should probably give a reminder that the macro I posted, and thus your edited version as well, is currently just at the level of a proof-of-concept hack. It works in some basic cases but it's not at all hard to get it to break.

[Jameson Quinn]

The conversation is progressing over on the bug: https://github.com/JuliaLang/julia/issues/16985

The latest, and it seems to me the best, proposal there is by @toivoh: `(a f b)`, with parentheses mandatory. That echoes the suggestion by Diego Javier Zea in this thread; but when Diego said it, I didn't think of having the parens as a mandatory part of the syntax. Note that this also would naturally suggest a macro version: `(a @m b)`.