Array Index Limits

[LarryD]

Fortran offers the ability to arbitrarily set array limits, e.g." real x(-30:40, 0:100)".  This is very useful when using an array to represent grid points on a dimensioned physical structure with, say, (0,0) somewhere in the structure. Is there any chance that this could be added to an upcoming version of Julia?

LarryD

[Matt Bauman]

See the previous discussion here: https://groups.google.com/forum/#!topic/julia-dev/NOF6MA6tb9Y

… which looks like it culminated in the OffsetArrays.jl package: https://github.com/alsam/OffsetArrays.jl

But you'll need to be very careful about how you use them since they break a core assumption about how arrays work.  I would advise against doing much more than scalar indexing unless you manually vet the implementations (in fact, it'd be safest if they weren't listed as subtypes of AbstractArray).  And you can't use the `end` keyword safely within an indexing expression, either.

Things will get a little better with 0.4, assuming OffsetArrays is updated to take advantage of some of the new array work there (like eachindex and colon lowering).  But even then, you'll still be flirting with trouble (which may manifest itself as silent computation errors).

[Tim Holy]

There's no need to have it in base julia, you can add it in a package.

https://github.com/alsam/OffsetArrays.jl
https://groups.google.com/forum/#!topic/julia-dev/NOF6MA6tb9Y

--Tim

[lawrence dworsky]

Thanks. I'll check it out. 

[Sisyphuss]

I read the "interfaces" chapter of the documentation today. I learned that, if you define an iterable as a subtype of AbstractArray, with only defining three methods (including `size()`, excluding `start()`), you can iterate on it just like iterate on an normal Array.

I think this advantage is based on the convention that Julia Array is 1-based. That's why @Matt mentioned: "it (OffsetArray) would be safest if they weren't listed as subtypes of AbstractArray"

[Matt Bauman]

On Monday, August 17, 2015 at 1:03:17 PM UTC-4, Sisyphuss wrote:

I read the "interfaces" chapter of the documentation today. I learned that, if you define an iterable as a subtype of AbstractArray, with only defining three methods (including `size()`, excluding `start()`), you can iterate on it just like iterate on an normal Array.

Iteration should work just fine in 0.4 if OffsetArray defines its own `eachindex` method.

Although more and more for loops are written generically using `eachindex`, there are still a lot of methods that use the old linear indexing standby:

for i=1:length(A)
   @inbounds A[i] = …
end

This is where things get really hairy for OffsetArrays.  That `@inbounds` propagates through to the inner array assignment, which will lead to silent data corruption and/or segfaults.  That's really why it shouldn't be an AbstractArray.

[lawrence dworsky]

Thanks for putting your time into this. Right now I'm still using 0.3.11, waiting for 0.4 to be the standard release. Then I'll dig into this and see if I get it to do what I want without undue aggravation. I had the same indexing issue with MatLab. Sometimes I miss the brute straightforwardness of Fortran.

Larry

[Matt Bauman]

One way to make OffsetArrays safe would be to make them index normally with integer indices, but introduce a *new* Integer type that gives it the special offset behavior.  You could name it something short, like F (for Fortran or oFFset) to reduce typing:

`A[F(-10), F(0)]`

... and you could do even better by making it construct by multiplication with the typename:

`A[-10F, 0F]`

Or even better, you should be able to write a macro that automatically converts all indices to this special type:

`@offset A[-10,0]` # This also could disallow the pesky end keyword!

I was playing with RaggedArrays over the weekend, and there I had to solve a similar problem because there are two possible meanings for linear indexing.  So I just created a LinearIndex type that behaves differently: https://github.com/mbauman/RaggedArrays.jl/blob/d2b6aeb854855c7912c104cb5312c13e989f2cf4/src/core.jl#L229-L247

Matt

[lawrence dworsky]

This sounds like the right approach. I'll try it soon and report what I did/didn't succeed at.​  Thanks.