Indexing with negative numbers as in R

[Douglas Bates]

In R a negative index to an array indicates that the corresponding slice should be dropped.  That is, A[-k,] returns all the rows of A except the k'th. 

I can create the same behavior in Julia with the somewhat more complex A[[1:(k-1), (k+1):end], :] but I was wondering if I had missed a simpler version.  Have I?

[John Myles White]

I've never seen it either, but think it would be a great idea. I've written some resampling code that would be much easier to read with that syntax.

-- John

[Kevin Squire]

Of course, this will thoroughly confuse people coming from Python, for which negative indexing indexes from the end of the array (A[-1] gives the last element of the array).

Kevin

[Patrick O'Leary]

It will--I did a double take to Douglas' post--but Julia has access to MATLAB-style A[end-1] indexing with the special "end" token, so maybe it all comes out in the wash?

[Stefan Karpinski]

As seductive as using negative indices to mean something cute like this may be, it's a bad idea. If some computation accidentally gives a negative value you get completely bizarre, unexpected behavior instead of an out-of-range index error. Having something syntactic to do this, however, wouldn't have the same problem since it would depend on the syntax, which can't be accidental, as opposed to a value, which can.

[John Myles White]

I think it's a bit much to refer to this tradition as cute: the use of negative indexing to describe the jackknife and other resampling methods dates back to the 1950's.

That said, I'd be happy with basically any clean method for doing this.

 -- John

[Stefan Karpinski]

Old things can be cute too! I mean cute in the sense of "excessively clever". Regardless of whether the tradition is venerable or not, I think my argument holds.

Moreover, as there often is, there's an alignment between the interests of the programmer and the compiler since both are trying to interpret code. Consider that if we did this, not only would the programmer need to keep in mind that A[:,k] might slice 1 column out of A or slice n-1 columns out of A — the compiler would too. When I say that the compiler needs to keep this in mind, I mean that it would have to emit code for every single indexing operation that checks the sign of k and does very different things depending on that sign. That's more overhead on every indexing operation, which is bad, but far worse is that it completely destroys the ability to cleanly infer the type of A[:,k] — whether the result is 1-d or 2-d depends on the *sign* of k, which the type system knows nothing about. Unless k is unsigned, in which case this is a much bigger disaster because -0x1 is 0xffffffffffffffff.

This is exactly the kind of feature that prevents languages like R and Python from being successfully type-inferenced. That's not a dig on their design — this kind of feature is perfectly reasonable in a language that's meant to be interpreted — but it's exactly the kind of thing that makes it impossible to just "rub some type inference" on existing dynamic languages and make them fast.

[J Luis]

Yep, FORTRAN used to accept negative indices (ant it probably still does).

That's horrible, indices are positions from the origin, no more than that. Please don't replicate.

[Stefan Karpinski]

I'm not sure what it means in Fortran, but at least the Pythonic/Perlish meaning of indexing from the end of an array when the index is negative doesn't destroy type inference.

[Stefan Karpinski]

Also: I'm not saying we can't have a way of slicing all but one index — because writing A[:,[(1:k-1) (k+1:end)] is admittedly awful. I'm just saying that it shouldn't be done using negative index values.

One possibility would be to use A[:,!k]. Currently !Int doesn't mean anything, and this plays nicely with logical indexing since writing !x where x is a vector of logicals inverts the set of selected indices. !Int would have to return some kind of InvertedIndex object or something like that.

[John Myles White]

I think something like that use of exclamation marks would be perfect. Ultimately all you want is a clean mechanism for writing: partition this array into two disjoint pieces: A[indices] and A[!indices].

 -- John

[J Luis]

Sorry, quite old memories of doing things like

   do i=-n/2,n/2-1

       ...

for fourier series

but a quick search comes up with things like

http://stackoverflow.com/questions/7065832/negative-array-indexing-and-placement-in-memory-pointing 

[Stefan Karpinski]

Perfect, I'll create an issue for implementing this. It's pretty independent from other things, so it can be an "up for grabs" issue too.

[Stefan Karpinski]

Done: https://github.com/JuliaLang/julia/issues/1032.

[John Myles White]

Great.

 -- John