ragged array indexing

[Mauro]

We are working on an implementation of ragged arrays, i.e. arrays

which have columns of varying length.  We need those to specify the

connectivity between mesh entities of an unstructured mesh.

One aim of our implementation is that indexing should work as for normal

2D arrays, at least as far as possible.  This would allow us to use

normal arrays if the connectivity is not ragged.

Question 1: has someone implemented a ragged array datatype already?

The indexing semantics would need to look like this:

ra::RaggedArray
# should give the n-th column:
ra[:,n]
# should give entries '3:end-2' of the n-th column (i.e. this would mean
# to first look up the n-th column and check how long it is)
ra[3:end-2,n]

As far as I can tell, when lowering the code the Julia parser replaces 

ra[3:end-2,n]

with

getindex(ra, 3:size(ra,1)-2, n)

Trouble is that size(ra,1)is ill-defined for a ragged array.

Questions 2: Is there a way to change what the parser does with the

'end'?  Or is there a more generic slice object?

If the answer to question 2 is negative then the best implementation I

see is to define a end-type:

immutable RaggedEnd
  offset::Real
  factor::Real
end
RaggedEnd() = RaggedEnd(0,1)
show(io::IO, re::RaggedEnd) = print(io, "ragged")
-(nn::Number, re::RaggedEnd) = RaggedEnd(re.offset-nn, re.factor)
/(re::RaggedEnd, nn::Number) = RaggedEnd(re.offset, re.factor/nn)
# also define +,* (and others?!)
...
# and colon probably like this:
colon(nn::Integer, re::RaggedEnd) = (nn, re)

size(ra::RaggedArray) = (RaggedEnd(), number_of_columns)

Now finally I could dissect the '3:end-2' inside the getindex method for a

RaggedArray and figure out the value of 'end' for the n-th column.

Question 3: This seems like quite a lot of code to just define 'end'.

Is this the way to go or do you have any ideas on how to make this more compact/better?

Thanks! 

Mauro

[Steven G. Johnson]

On Wednesday, November 27, 2013 10:29:44 AM UTC-5, Mauro wrote:

# should give entries '3:end-2' of the n-th column (i.e. this would mean
# to first look up the n-th column and check how long it is)
ra[3:end-2,n]

As far as I can tell, when lowering the code the Julia parser replaces 

ra[3:end-2,n]

with

getindex(ra, 3:size(ra,1)-2, n)

Trouble is that size(ra,1)is ill-defined for a ragged array.

Questions 2: Is there a way to change what the parser does with the

'end'?  Or is there a more generic slice object?

In general, the answer to the question "can I change what the parser does" in Julia is "no".  The parser is not implemented in Julia, so its behavior is not modifiable by Julia code.

However, one option might be to have your size(ra,1) method return some kind of symbolic type T, so that its evaluation could be delayed until getindex is executed.  You'd have to overload integer arithmetic operations like +/- so size(ra,1)-2 works and keeps track of the -2 offset.  You'd probably also need to define a Range type for this symbolic type, so that 3:size(ra,1)-1 works.  Should all be straightforward stuff.

[Mauro]

Steven G. Johnson wrote:
> In general, the answer to the question "can I change what the parser does"
> in Julia is "no". The parser is not implemented in Julia, so its behavior
> is not modifiable by Julia code.

Makes sense.

> However, one option might be to have your size(ra,1) method return some
> kind of symbolic type T, so that its evaluation could be delayed until
> getindex is executed. You'd have to overload integer arithmetic operations
> like +/- so size(ra,1)-2 works and keeps track of the -2 offset. You'd
> probably also need to define a Range type for this symbolic type, so that
> 3:size(ra,1)-1 works. Should all be straightforward stuff.

This sounds quite like the end-type I suggested, or is there something
else to 'some kind of symbolic type T'?

[Steven G. Johnson]

On Wednesday, November 27, 2013 11:24:48 AM UTC-5, Mauro wrote:

This sounds quite like the end-type I suggested, or is there something
else to 'some kind of symbolic type T'?

Whoops, I missed the second half of your post.  Yes, your RaggedEnd type is exactly what I had in mind.