A "take" function

[Paul G]

Hi,

I'm totally new to matrix programming, so I'll apologize in advance for my naïveté.

After watching the video for Conway's Game of Life in APL I figured that it couldn't be too hard to do in Clojure. I also figured it would be a good chance to try out core.matrix. For now I have just stayed with the central project, and haven't tried any other implementations. (The matrices in the demo are small, so performance isn't an issue for me yet).

For the curious, my attempt is on Github. I'm not proud of it, but it was a fun learning experience. (Incidentally, I know nothing about APL)

The reason I'm writing was because I ran into some difficulties, and I was wondering if I've missed anything in core.matrix, or if there is room for more functionality.

The first issue I ran into was the "take" function. This was used in 2 ways in the video. The first is a simple expansion, where the rest of the matrix is padded with zeros. The second call also did an expansion, but also added 2 new parameters that seem to operate like "rotate" though with inverted sign. The idea of this function is to embed the data of a small matrix inside a larger field. I emulated the first with a compute-matrix, and the second with compute-matrix followed by 2 rotations. Is there a better way to do this?

I also found that I wanted to perform "add" style operations between matrices of identical shape, but for operations other than addition. The particular example was doing AND and OR operations. Fortunately, because of the numerical booleans in this application I was able to use the "add" function with emap to renormalize. However, it felt like there was a missing abstraction for performing an element-wise binary operation between matrices. Is there a function I should be using?

One APL operation that really tripped me up was how to perform an outer product of rotations to be applied to a matrix. I came up with one approach that was abstracting the outer product for the function "rotate" but I'm not comfortable that I got it right (it worked for my case, but was it really general?). In the end, I decided that creating a seq of matrices via a "for" comprehension was better suited in this particular case. But in general this outer-product-rotate seems to be awkward operation to implement in Clojure (not too hard - just awkward). Does anyone know this function, and have some ideas?

There was one small issue I kept running into, and I was wondering if it is just an inconvenience for generalizing over N-dimensional matrices, or if there is an approach that may help. As part of my attempt at an "outer product rotation", I tried to pair up the items from each vector, and store these in a matrix. I planned to use emap on these pairs. However, my "pairs" were really 2 element vectors, and so emap thought I was working on a 3D matrix instead of a 2D one. I got around it by using a map (with keys of :a :b) to wrap the data. Similarly, the Life demo uses matrices and vectors with 2D matrices as elements, but some functions treated this  as a 3D or 4D matrix. Consequently, when transpose reversed the dimensions, it converted a 3x3 grid of 5x7 matrices into a 5x7 grid of 3x3 matrices, whereas I just wanted to switch the 3x3 dimensions (and keep the 5x7 elements). The workaround in that case is (slices m 1). Similarly, I transposed a row of 5x7 into a column of 5x7 by calling (map vector m). So I could always get around it, but since it was an issue that kept showing up I thought it was worth mentioning.

Overall, most of what I needed to recreate the demo worked perfectly. Thank you for continuing to make Clojure cool. :)

Paul

[Mike Anderson]

On Friday, 27 June 2014 19:28:11 UTC+1, Paul G wrote:

Hi,

I'm totally new to matrix programming, so I'll apologize in advance for my naïveté.

After watching the video for Conway's Game of Life in APL I figured that it couldn't be too hard to do in Clojure. I also figured it would be a good chance to try out core.matrix. For now I have just stayed with the central project, and haven't tried any other implementations. (The matrices in the demo are small, so performance isn't an issue for me yet).

For the curious, my attempt is on Github. I'm not proud of it, but it was a fun learning experience. (Incidentally, I know nothing about APL)

The reason I'm writing was because I ran into some difficulties, and I was wondering if I've missed anything in core.matrix, or if there is room for more functionality.

The first issue I ran into was the "take" function. This was used in 2 ways in the video. The first is a simple expansion, where the rest of the matrix is padded with zeros. The second call also did an expansion, but also added 2 new parameters that seem to operate like "rotate" though with inverted sign. The idea of this function is to embed the data of a small matrix inside a larger field. I emulated the first with a compute-matrix, and the second with compute-matrix followed by 2 rotations. Is there a better way to do this?

I think subvector / submatrix may be useful here?

With mutable arrays you can take a submatrix view of a bigger array and set it to equal the source matrix - this can be a useful and efficient method if you are building larger arrays

 

I also found that I wanted to perform "add" style operations between matrices of identical shape, but for operations other than addition. The particular example was doing AND and OR operations. Fortunately, because of the numerical booleans in this application I was able to use the "add" function with emap to renormalize. However, it felt like there was a missing abstraction for performing an element-wise binary operation between matrices. Is there a function I should be using?

`emap` should work here? It works roughly the same as add but you can supply an arbitrary function, e.g.

(emap #(or %1 %2) [true false false] [false false true])

=> [true false true]

 

One APL operation that really tripped me up was how to perform an outer product of rotations to be applied to a matrix. I came up with one approach that was abstracting the outer product for the function "rotate" but I'm not comfortable that I got it right (it worked for my case, but was it really general?). In the end, I decided that creating a seq of matrices via a "for" comprehension was better suited in this particular case. But in general this outer-product-rotate seems to be awkward operation to implement in Clojure (not too hard - just awkward). Does anyone know this function, and have some ideas?

I think I would just use list comprehensions here? We could define a generic outer-product that takes arbitrary operations but I haven't seen too much demand for this yet....

 

There was one small issue I kept running into, and I was wondering if it is just an inconvenience for generalizing over N-dimensional matrices, or if there is an approach that may help. As part of my attempt at an "outer product rotation", I tried to pair up the items from each vector, and store these in a matrix. I planned to use emap on these pairs. However, my "pairs" were really 2 element vectors, and so emap thought I was working on a 3D matrix instead of a 2D one. I got around it by using a map (with keys of :a :b) to wrap the data. Similarly, the Life demo uses matrices and vectors with 2D matrices as elements, but some functions treated this  as a 3D or 4D matrix. Consequently, when transpose reversed the dimensions, it converted a 3x3 grid of 5x7 matrices into a 5x7 grid of 3x3 matrices, whereas I just wanted to switch the 3x3 dimensions (and keep the 5x7 elements). The workaround in that case is (slices m 1). Similarly, I transposed a row of 5x7 into a column of 5x7 by calling (map vector m). So I could always get around it, but since it was an issue that kept showing up I thought it was worth mentioning.

This is a fundamental limitation with using Clojure vectors as an array implementation. I don't think there is any way around it - the only way you can determine the dimensionality of the array is to introspect the nested vectors, which creates ambiguity if you want to use vectors as elements.

My general advice is to use a different implementation if you want to do something like this (NDArray should work for example since the NDArray storage classes encode shape information)

 

Overall, most of what I needed to recreate the demo worked perfectly. Thank you for continuing to make Clojure cool. :)

Paul

Glad it helps! 

[Maik Schünemann]

The other way would be to store the shape attribute in metadata - that was discussed before and also has its drawbacks 

My general advice is to use a different implementation if you want to do something like this (NDArray should work for example since the NDArray storage classes encode shape information)

 

Overall, most of what I needed to recreate the demo worked perfectly. Thank you for continuing to make Clojure cool. :)

Paul

Glad it helps! 

--
You received this message because you are subscribed to the Google Groups "Numerical Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numerical-cloj...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Anderson]

On Saturday, 28 June 2014 12:11:00 UTC+1, Maik Schünemann wrote:

There was one small issue I kept running into, and I was wondering if it is just an inconvenience for generalizing over N-dimensional matrices, or if there is an approach that may help. As part of my attempt at an "outer product rotation", I tried to pair up the items from each vector, and store these in a matrix. I planned to use emap on these pairs. However, my "pairs" were really 2 element vectors, and so emap thought I was working on a 3D matrix instead of a 2D one. I got around it by using a map (with keys of :a :b) to wrap the data. Similarly, the Life demo uses matrices and vectors with 2D matrices as elements, but some functions treated this  as a 3D or 4D matrix. Consequently, when transpose reversed the dimensions, it converted a 3x3 grid of 5x7 matrices into a 5x7 grid of 3x3 matrices, whereas I just wanted to switch the 3x3 dimensions (and keep the 5x7 elements). The workaround in that case is (slices m 1). Similarly, I transposed a row of 5x7 into a column of 5x7 by calling (map vector m). So I could always get around it, but since it was an issue that kept showing up I thought it was worth mentioning.

This is a fundamental limitation with using Clojure vectors as an array implementation. I don't think there is any way around it - the only way you can determine the dimensionality of the array is to introspect the nested vectors, which creates ambiguity if you want to use vectors as elements.

The other way would be to store the shape attribute in metadata - that was discussed before and also has its drawbacks 

Yes of course this would be an technically feasible - though I think the drawbacks to using metadata for shapes are bad enough that they make this solution effectively impractical (significant performance overhead, implementation complexity, need to handle the non-metadata case separately etc.)