Question about 'dot' notation (and max/maximum)

[Hans W Borchers]

I have to admit that I am quite unhappy with some of the changed features in

Julia version 0.3.0, especially the 'dot' notation. Here are some examples.

Let x be a vector defined as  x = [0.1, 0.2, 0.3] . Then typing

    julia> 5 + x

    WARNING: x::Number + A::Array is deprecated, use x .+ A instead.

but  5 + x  is a universal mathematical notation that should be allowed

to be used regardless of any programming language considerations.

On the other hand, both

    julia> 5 * x;

    julia> 5 .* x;

work without warning. Why is  5 * x  not also deprecated?

If I want to write, e.g., Runge's function in a vectorized form,

    julia> runge(x) = 1 ./ (1 .+ 5.*x.^2)

then this looks quite ugly and difficult to grasp on first view.

As another example, look at the max / maximum 'dichotomy':

    julia> maximum([x, 0.5])

    0.5

    julia> maximum(x, 0.5)

    3-element Array{Float64,1}:

     0.1

     0.2

     0.3

The first answer looks natural, but I have difficulties understanding the

meaning of the second case. On the other hand:

    julia> max(x, 0.5)

    3-element Array{Float64,1}:

     0.5

     0.5

     0.5

while  max(x, [0.5])  will lead to a dimension error, and  max([x, 0.5]) 

to a deprecation warning (which I seem to understand why).

I think all this is quite confusing for someone wanting to use Julia mostly

for technical computing, as the logo promises.

I am sure this has been discussed before, so probably I missed it. Sorry.

[Tomas Lycken]

I can't answer for the max/maximum case, but regarding dot notation for vector operations, it all quite makes sense if one tries to be as strict as possible with using actual mathematical notation. 

For example, having previously defined x = [0.1, 0.2, 0.3], then saying

x + 5

isn't really defined in a general sense. What you might mean by that is really x + 5 * [1, 1, 1], where 5 * [1, 1, 1] is interpreted as the regular vector-by-scalar multiplication. But this is not an operation that is required to define a general vector space - rather, a vector space is basically just a set V and a field F, with two operators defined:

+(v1, v2) is defined for v1, v2 in V yielding v3 = v1+v2
*(f,v) is defined for a scalar f in F and a vector v in V, such that w = f*v = v*f

An even more thorough definition is given here: http://en.wikipedia.org/wiki/Linear_algebra#Vector_spaces

In this context, it makes sense to have defined scalar multiplication and vector addition for x, but not define scalar addition - after all, it's a concept which is, in a strict mathematical sense, a little fuzzy. It's easy to define scalar addition once you consider element-wise operations, but that is exactly the distinction the dot notation is for - to make that distinction explicit. In your definition of the runge function, you're not actually working with a vector x in the mathematical sense of a vector space, but rather with an array as a container for many elements for which you want to perform the same calculations. Julia is really doing the same thing as e.g. Matlab here, but in a way that's less error-prone since it actually requires the user to decide whether it's an element-wise operation that is desired, or if it's really something else (and in that case, to better specify what else).

I hope this made some sense =)

// T

[Andreas Noack Jensen]

but  5 + x  is a universal mathematical notation that should be allowed

to be used regardless of any programming language considerations

Actually, it is the other way around. 5+x does not make sense mathematically if x is a vector, but it is accepted in many programming language as a broadcasting operation. 5*x and x/5 do make sense when x is a vector. After a long discussions we decided to try to use the dot notation consistently for broadcasting operations.

--
Med venlig hilsen

Andreas Noack Jensen

[Ivar Nesje]

See also the discussion in https://github.com/JuliaLang/julia/pull/5810 and https://github.com/JuliaLang/julia/issues/5807 for the .+ vs + question.

The second argument to maximum is called region, and lets you compute the maximum over a specific direction. The error checking seems to be non existent, so the result you get is nonsense when you use 0.5 as region.

[Ethan Anderes]

It took me a second to get used to .+ but now it feels very natural to me. Maybe it helps to see the benefits (rather than just the changes). From my perspective you gain functionality in that .+ has implicit broadcasting:

x = [1 2;
2 3]
row = [3 4]

x + row # this correctly gives an error
x .+ row # julia knows I want `row` broadcasted

[Hans W Borchers]

Okay, no problem, I'll get used to it.

I give Matlab courses and almost every year the 'dot' notation is one of the most 

difficult things for the students to learn and to remember in their homework.

At one point I wondered myself what the  .'  operator is doing. 

Is there something similar in Julia?

[Ivar Nesje]

[1:3]' is the conjugate transpose of [1:3]

[1:3].' is the regular transpose

The difference is that the first apply conj() on all the elements while transposing. This is confusing for everyone who does not live and breathe complex linear algebra, but apparently very natural for those who do. See https://github.com/JuliaLang/julia/issues/6395 for a discussion of how this works out for string arrays.

Ivar

[Carlo Baldassi]

Small hint: the easiest way I found to clarify the distinction between "max" and "maximum", and why it is necessary in the first place, is that it's the same as that between "the + function" vs "sum", or "the * function" vs "prod".

(Of course, not checking the region argument in maximum is just a bug, as noted by others.)

Hope that helps. Cheers.

On Friday, May 2, 2014 3:55:20 PM UTC+2, Hans W Borchers wrote:

[Billou Bielour]

I have to say the difference between "max" and "maximum" is pretty much unintelligible by just looking at the names. I've read the help for both three days ago and I already forgot which does what. Maybe "maximum" should be renamed "maxoverdims" or something of the sort.

[Tomas Lycken]

I do think the help texts just from help() on the two functions is quite clear:

julia> help(maximum)

INFO: Loading help data...

Base.maximum(itr)

   Returns the largest element in a collection.

Base.maximum(A, dims)

   Compute the maximum value of an array over the given dimensions.

julia> help(max)

Base.max(x, y, ...)

   Return the maximum of the arguments. Operates elementwise over

   arrays.

With this text in front of me, I'm not at all surprised at the following results:

julia> A = [1 2; 3 4]; B = [4 3; 2 1]

2x2 Array{Int64,2}:

 4  3

 2  1

julia> max(A,B)

2x2 Array{Int64,2}:

 4  3

 3  4

julia> max(A)

WARNING: max(x) is deprecated, use maximum(x) instead.

 in max at deprecated.jl:26

4

julia> c = [0;5]

2-element Array{Int64,1}:

 0

 5

julia> max(A,c)

ERROR: dimensions must match

 in promote_shape at operators.jl:166

[Billou Bielour]

I'm not saying that the help is not clear, or that the behavior is surprising, but only that there no information in the words "max" and "maximum" that allows you to guess or remember their behaviors. I mean one is just the abbreviation of the other.
For example if I tell you I have two functions, "ceil" and "ceiling", can you guess what is the difference between the two ?

[Stefan Karpinski]

Honestly, I've always been dissatisfied with this solution and would have preferred the keyword-argument solution that I proposed back when we were discussing this. Having lived with the maximum thing for a while and seen others encounter it, I'm no less dissatisfied. I still type max when I need maximum almost every time.

[Tomas Lycken]

After thinking another few moments about it, I think the confusion might be lifted slightly with some clever use of dispatch. Consider if we'd do something like this:

max(itr) # returns the largest element in the collection
max(a,b,c...) = max([a,b,c...])

max{N}(A,dims::NTuple{N}) # compute maximum element along dimensions
max(A,dim::Integer) = max(A, (dim,))
max(a::Number, b::Integer) = max([a,b]) # there's probably a more efficient way to do this, to avoid the array allocation

Since the last one is more specific than the second to last, there is no longer a conflict between max(A,2) to calculate the maximum of each row of a matrix, and max(a,2) to get the largest value out of a and the integer 2. There might of course be use cases I've forgotten about, but to me this seems to cover most of the use cases and still avoid having to remember two different function names. (I tried to take a look at methods(max) to see if I could find a reason for this not already being the way things were, but I ran into trouble...)

// T

[Stefan Karpinski]

Since any way you might express the dimensions argument could also be a collection to reduce over, I don't think there's any way to squeeze all this functionality into a single function unambiguously without using keyword arguments. But with a keyword argument, you certainly can do it. We could even leave the maximum and minimum versions in place as the unambiguously "reducer" versions and have min and max server both roles depending on how they're called. The main concern is avoiding potential performance problems caused by using keyword arguments.

[Toivo Henningsson]

I think there's too much ambiguity here to get away with overloading both in the same function like this.

Say that I have an array A and I want to clamp all elements so that they are no larger than 1:

    A_clamped = min(A, 1)

But it's equally reasonable to want to use

A_col_minima = minimum(A, 1)

to get the minimum over each row. There needs to be some way do distinguish a 1 that is a dimension from a 1 that is the scalar value 1, be it different function names, a keyword argument, or something else. I still think it's too bad that keyword arguments can't be used when low overhead is needed (or maybe they can, sometimes?)

A less drastic solution would be to rename maximum to something like maxover that would at least indicate that you are maximizing over something (dimensions).

(Or we could maximum to max_ as a pun on the fact that maximizing over a dimension is typically typed as e.g. max_i A_i in latex :)

[Milan Bouchet-Valat]

Le lundi 05 mai 2014 à 11:00 -0400, Stefan Karpinski a écrit :
> Since any way you might express the dimensions argument could also be
> a collection to reduce over, I don't think there's any way to squeeze
> all this functionality into a single function unambiguously without
> using keyword arguments. But with a keyword argument, you certainly
> can do it. We could even leave the maximum and minimum versions in
> place as the unambiguously "reducer" versions and have min and max
> server both roles depending on how they're called. The main concern is
> avoiding potential performance problems caused by using keyword
> arguments.

In R, the element-wise/parallel version is called pmax(), while the
reducing version if called max(). Could be an alternative to keyword
arguments.

Or it could be useful to have a general syntax to apply element-wise
operations, as a short-hand for manual iteration over all the dimensions
of one or several arrays.


Regards

[Stefan Karpinski]

The thing I had proposed was

min(A,1) => smallest value of A or 1, whichever is smaller

min(A,1, dim=()) => same shape as A, clamped above at 1

min(A,1, dim=1) => the minimum of each column or 1 as a row matrix

min(A,1, dim=2) => the minimum of each row or 1 as a column matrix

Efficient implementation is a bit tricky, but the usage I think is pretty clear.

[Ethan Anderes]

+1 for max() and pmax(). It seems the easiest to write and to interpret.

[Stefan Karpinski]

I think at this point, while the maximum name may not be everyone's favorite choice, that bikeshed has sailed. Unless we're going to solve this problem differently, it's going to stay the way it is: max is to maximum as + is to sum.

[Tobias Knopp]

In the github issue it was also proposed to call the multiple argument max function maxof which is IMHO a little clearer than pmax.

[Tim Holy]

An option we considered and discarded was to introduce a Dim immutable, i.e.,
min(A, Dim(2))
That would not have the performance disadvantage of a keyword argument.

--Tim

> >> I ran into trouble... <https://github.com/JuliaLang/julia/issues/6755>)

[Ethan Anderes]

yeah, I can imagine how annoying it is to the developers that all us newbies feel like we can chime in on language design:) However, I do think its a sign of how natural and inclusive the language feels to the everyday user.

BTW: I like maxof better than pmax, but maybe like .max() better than either ;)

[Stefan Karpinski]

On Mon, May 5, 2014 at 12:53 PM, Ethan Anderes <ethana...@gmail.com> wrote:

yeah, I can imagine how annoying it is to the developers that all us newbies feel like we can chime in on language design:) However, I do think its a sign of how natural and inclusive the language feels to the everyday user.

BTW: I like maxof better than pmax, but maybe like .max() better than either ;)

Not at all – I think it's great to have lots of input on these kinds of decisions – we all have to live with the language. But at some point you just need to make a decision and stick with it. We had an epic bikeshed and you'll note that my position did not win out. The most popular option that won out was max, as in "the max of a and b", and maximum, as in "the maximum of this vector". While you can say "the maximum of a and b" as well as "the max of this vector", I do think there's a pretty clear linguistic indication that max is a choice between two things while the maximum is the largest value in a collection. It's a subtle linguistic difference, but I do think it's there.

[cnbiz850]

My question is if x + A does not make sense in any way other than x .+
A, then why introduce .+ at all since + means intuitively .+ and is simpler?

[Stefan Karpinski]

I'm pretty sure I've explained the ambiguity at least three or four times at this point.

[Yuuki Soho]

K leo, I think it's mainly a problem of consistency across operators (+ behave like * and .+ behave like .*) and across dimensions (adding a scalar to a vector behave the same as adding a vector to a matrix). 

The nice thing with Julia is that you can easily (re) define operators to do what you want, if you prefer + to work with scalar you can just define:

+{T,K}(a::Array{T,K},x::Number) = a .+ x

And it should work.

[Hans W Borchers]

What has happened, two months later, to the (in)famous 'dot' notation in Julia?
There was such a convincing discussion that  5 + x  shall not be correct, so
I got used to it. When I now by chance try

    julia> x = [0.1, 0.2, 0.3];

    julia> 5 + x
    3-element Array{Float64,1}:
     5.1
     5.2
     5.3

    julia> versioninfo()
    Julia Version 0.3.0-prerelease+3921
    Commit 0b46af5* (2014-06-28 02:01 UTC)
    Platform Info:
      System: Linux (x86_64-linux-gnu)
      CPU: Intel(R) Core(TM) i3-3217U CPU @ 1.80GHz
      WORD_SIZE: 64
      BLAS: libblas.so.3
      LAPACK: liblapack.so.3
      LIBM: libopenlibm

[ PS: This was the last PPA update I could get on Ubuntu. ]

On Friday, May 2, 2014 3:55:20 PM UTC+2, Hans W Borchers wrote:

I have to admit that I am quite unhappy with some of the changed features in

Julia version 0.3.0, especially the 'dot' notation. Here are some examples.

Let x be a vector defined as  x = [0.1, 0.2, 0.3] . Then typing

[Milan Bouchet-Valat]

Le mardi 08 juillet 2014 à 01:51 -0700, Hans W Borchers a écrit :
> What has happened, two months later, to the (in)famous 'dot' notation
> in Julia?
> There was such a convincing discussion that 5 + x shall not be
> correct, so
> I got used to it. When I now by chance try
>
> julia> x = [0.1, 0.2, 0.3];
>
> julia> 5 + x
> 3-element Array{Float64,1}:
> 5.1
> 5.2
> 5.3
>
> julia> versioninfo()
> Julia Version 0.3.0-prerelease+3921
> Commit 0b46af5* (2014-06-28 02:01 UTC)
> Platform Info:
> System: Linux (x86_64-linux-gnu)
> CPU: Intel(R) Core(TM) i3-3217U CPU @ 1.80GHz
> WORD_SIZE: 64
> BLAS: libblas.so.3
> LAPACK: liblapack.so.3
> LIBM: libopenlibm
>
>
> [ PS: This was the last PPA update I could get on Ubuntu. ]

See https://github.com/JuliaLang/julia/pull/7226


Regards

[Hans W Borchers]

Has this been announced somewhere? (Frankly,I'm not reading julia-dev on a regular basis.)
I see that the manual does not reflects this change. See section "Vectorized Operators and Functions":

    Some operators without dots operate elementwise anyway
    when one argument is a scalar.
    These operators are *, /, \, and the bitwise operators.

though ".+", ".-" are not in the list of binary arithmetic operators anymore (and do work properly).

And actually,  1 / [1.0, 2.0]  does now operate elementwise, but gives a "deprecated" warning once.
Will it be deprecated, or stay alive as the manual claims.

Dear core developers, don't listen to Julia newbies (like me) too closely -- stick to what you think is reasonable.

On Tuesday, July 8, 2014 10:51:30 AM UTC+2, Hans W Borchers wrote:

What has happened, two months later, to the (in)famous 'dot' notation in Julia?
There was such a convincing discussion that  5 + x  shall not be correct, so

I got used to it. When I now by chance try ...

[Tobias Knopp]

Just commenting on the lack of anouncement. As all this happend on a developement branch of Julia which is always a little in flux these things should not really be anounced. What counts is how the NEWS file looks in the end.

[Jutho]

Fully realising that this discussion has been settled and the convention is here to stay, I nevertheless feel obsessed to make the remark that there would have been more elegant solutions. Other languages have been able to come up with acceptable operators for a binary 'min' or 'max':

https://gcc.gnu.org/onlinedocs/gcc-2.95.2/gcc_5.html#SEC107

(less elegant alternative since the difference between min and max is confusing:

http://support.sas.com/documentation/cdl/en/imlug/59656/HTML/default/viewer.htm#langref_sect10.htm )

The linguistic difference between max and maximum is very subtle, maybe because I am not a native English speaker. From the mathematical point of view, I think of a maximum (but preferably abbreviated as max) always in the context of a function or a set, never in the context of the largest out of two numbers

(not really a reference or definition, but http://en.wikipedia.org/wiki/Maxima_and_minima only/directly discusses functions and sets).

>? compares to max as + compares to sum would have been so much more convincing :-). That's all, enough bikeshedding for now, back to work.

Op maandag 5 mei 2014 17:46:05 UTC+2 schreef Stefan Karpinski:

[Jutho Haegeman]

Since I just read these operators were later removed from gcc again, it must not all have been perfect either :D.