Aref on steroids
Slobodan Blazeski
I need an array *accessor* with elided indices for example
(defun elide (array &rest subscripts)
...
For foo being 2 dimensional array (2 5)
0 1 2 3 4
5 6 7 8 9
(elide foo 1 2) <=> (aref foo 1 2)
7
However for every subscript that's not an integer:
(elide foo 1 t)
#(5 6 7 8 9)
(elide foo t 4)
#(4 9)
After creating the array to carry result (*) the only idea to fill
the elements that comes in mind is with some recursive macro that
expands into something like (for the (elide foo t 4) case)
(let ((j -1))
(dotimes (i1 2)
(setf (row-major-aref result (incf i))
(aref argument-array i1 4))
I'm not sure that will be best idea for array with dimensions 3 or
more, so could someone advise better approach?
thanks
bobi
;; My code so far
;;; Elide accessor
(defun elide (array &rest subscripts)
(cond ((every #'integerp subscripts)
(apply #'aref array subscripts))
((notany #'integerp subscripts)
array)
(t
(fill-res-array array
subscripts
(res-array subscripts
(array-dimensions array))))))
(defun res-array (sub dim &optional res)
(if (null sub)
(make-array res)
(res-array (cdr sub)
(cdr dim)
(if (integerp (car sub)) res
(cons (car dim) res)))))
John Thingstad
<slobodan...@gmail.com>:
>
> (defun res-array (sub dim &optional res)
> (if (null sub)
> (make-array res)
> (res-array (cdr sub)
> (cdr dim)
> (if (integerp (car sub)) res
> (cons (car dim) res)))))
>
That's a perfectly acceptable way. Another is to use a displacement array.
(make-array (array-total-size whatever) :displaced-to whatever)
---------------------
John Thingstad
Slobodan Blazeski
> På Wed, 10 Jun 2009 10:37:50 +0200, skrev Slobodan Blazeski
>
>
>
> > (defun res-array (sub dim &optional res)
> > (if (null sub)
> > (make-array res)
> > (res-array (cdr sub)
> > (cdr dim)
> > (if (integerp (car sub)) res
> > (cons (car dim) res)))))
>
> That's a perfectly acceptable way. Another is to use a displacement array.
> (make-array (array-total-size whatever) :displaced-to whatever)
If I need array
a1
0 1 2 3 4
5 6 7 8 9
And call (elide a1 t 3)
(make-array 2 :displaced-to a :displaced-index-offset 3)
#(3 4)
while the correct result should be #(3 8)
bobi
>
> ---------------------
> John Thingstad
Slobodan Blazeski
wrote:
Ok I have an idea for array accessing without recursive macro. I could
use variations.
thanks
bobi
Rob Warnock
+---------------
| I have another problem with looping over multi dimensional arrays:
| I need an array *accessor* with elided indices for example
| (defun elide (array &rest subscripts)
| ...
| For foo being 2 dimensional array (2 5)
| 0 1 2 3 4
| 5 6 7 8 9
| However for every subscript that's not an integer:
| (elide foo 1 t)
| #(5 6 7 8 9)
|
| (elide foo t 4)
| #(4 9)
People often call this function "slicing" an array (rather than eliding).
There was some discussion of this here back in February of this year
with the "Subject: multidimensional array slices". In particular,
Tamas Papp pointed to this package:
If you want the general slices, I wrote an experimental (but working)
package which might do what you want. You can find it here:
http://www.princeton.edu/~tpapp/index.html#affi
affi
An affine indexing package that provides convenience functions for
traversing slices of arrays, optionally with index permutations and
other convenient transformations. A driver clause for iterate is
provided, along with map-subarray, a simple yet powerful function
for mapping arrays into each other using affine indexes.
[GIT repo] [tarball]
Marco Antoniotti also discussed his MAKE-ARRAY-SLICE function.
-Rob
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Slobodan Blazeski
> Slobodan Blazeski <slobodan.blaze...@gmail.com> wrote:
> +---------------
> | I have another problem with looping over multi dimensional arrays:
> | I need an array *accessor* with elided indices for example
> | (defun elide (array &rest subscripts)
> | ...
> | For foo being 2 dimensional array (2 5)
> | 0 1 2 3 4
> | 5 6 7 8 9
> ...
> | However for every subscript that's not an integer:
> | (elide foo 1 t)
> | #(5 6 7 8 9)
> |
> | (elide foo t 4)
> | #(4 9)
> +---------------
>
> People often call this function "slicing" an array (rather than eliding).
> There was some discussion of this here back in February of this year
> with the "Subject: multidimensional array slices". In particular,
> Tamas Papp pointed to this package:
>
> If you want the general slices, I wrote an experimental (but working)
> package which might do what you want. You can find it here:
>
> http://www.princeton.edu/~tpapp/index.html#affi
> affi
> An affine indexing package that provides convenience functions for
> traversing slices of arrays, optionally with index permutations and
> other convenient transformations. A driver clause for iterate is
> provided, along with map-subarray, a simple yet powerful function
> for mapping arrays into each other using affine indexes.
> [GIT repo] [tarball]
also need a setter.
thanks for the pointers
bobi
Pascal J. Bourguignon
If you don't need to extract the slice, then you can just write:
(defun make-argument-list (indices)
(mapcar (lambda (i)
(if (integerp i)
i
(gensym)))
indices))
(defun parameter-list (argument-list)
(remove-if (function integerp) argument-list))
(defun make-reader (indices)
(let ((args (make-argument-list indices)))
(compile nil `(lambda (array ,@(parameter-list args))
(aref array ,@args)))))
(defun make-writer (indices)
(let ((args (make-argument-list indices)))
(compile nil `(lambda (new-value array ,@(parameter-list args))
(setf (aref array ,@args) new-value)))))
(defclass slice ()
((array :initarg :array)
(reader :initarg :reader)
(writer :initarg :writer)))
(defun slice (array &rest indices)
(make-instance 'slice :array array
:reader (make-reader indices)
:writer (make-writer indices)))
(defmethod ref ((self slice) &rest indices)
(apply (slot-value self 'reader) (slot-value self 'array) indices))
(defmethod (setf ref) (new-value (self slice) &rest indices)
(apply (slot-value self 'writer) new-value (slot-value self 'array) indices))
(let* ((a #4A((((1101 1102 1103)
(1111 1112 1113)
(1121 1122 1123))
((1201 1202 1203)
(1211 1212 1213)
(1221 1222 1223))
((1301 1302 1303)
(1311 1312 1313)
(1321 1322 1323)))
(((2101 2102 2103)
(2111 2112 2113)
(2121 2122 2123))
((2201 2202 2203)
(2211 2212 2213)
(2221 2222 2223))
((2301 2302 2303)
(2311 2312 2313)
(2321 2322 2323)))))
(s (slice a 1 t 2 t)))
(setf (ref s 1 1) 0)
(loop for i from 0 to 2
do (loop for j from 0 to 2
do (princ (ref s i j)) (princ " ")
finally (terpri)))
a)
prints:
2121 2122 2123
2221 0 2223
2321 2322 2323
returns:
#4A((((1101 1102 1103) (1111 1112 1113) (1121 1122 1123))
((1201 1202 1203) (1211 1212 1213) (1221 1222 1223))
((1301 1302 1303) (1311 1312 1313) (1321 1322 1323)))
(((2101 2102 2103) (2111 2112 2113) (2121 2122 2123))
((2201 2202 2203) (2211 2212 2213) (2221 0 2223))
((2301 2302 2303) (2311 2312 2313) (2321 2322 2323))))
--
__Pascal Bourguignon__
Tamas K Papp
Have a look at
http://github.com/tpapp/xarray/tree/master
which will supersede affi. Currently beta.
I will push a new version on Monday, with set-view that does what you
want. You can also write it yourself, using rm-subscripts to convert
flat indices 0,...,1-size to subscripts. Sorry, I have to rush now,
catching an airplane,otherwise I would write it for you.
HTH,
Tamas
Thomas A. Russ
> > Slobodan Blazeski ��<slobodan.blaze...@gmail.com> wrote:
> > +---------------
> > | I have another problem with looping over multi dimensional arrays:
> > | I need an array *accessor* with elided indices for example
> > | (defun elide (array &rest subscripts)
> > | ...
> > | For foo ��being 2 dimensional array (2 5)
> > | 0 1 2 3 4
> > | 5 6 7 8 9
> > ...
> > | However for every subscript that's not an integer:
> > | (elide foo 1 t)
> > | #(5 6 7 8 9)
> > |
> > | (elide foo t 4)
> > | #(4 9)
> > +---------------
...
> I'm looking at it but it seems that only provides traversing slices. I
> also need a setter.
Does that mean you need to maintain the mapping from the slice back into
the original array?
In other words, if you so something like
(setf (aref (elide foo t 4) 1) -9)
would you want the original array foo updated to
0 1 2 3 4
5 6 7 8 -9
as well?
That would require a much more involved solution compared to just
extracting a COPY of some part of the existing array.
--
Thomas A. Russ, USC/Information Sciences Institute
Slobodan Blazeski
wrote:
>
> If you don't need to extract the slice,
> then you can just write:
> (defun make-reader (indices)
> (let ((args (make-argument-list indices)))
> (compile nil `(lambda (array ,@(parameter-list args))
> (aref array ,@args)))))
>
> (defun make-writer (indices)
> (let ((args (make-argument-list indices)))
> (compile nil `(lambda (new-value array ,@(parameter-list args))
> (setf (aref array ,@args) new-value)))))
Those are very cool
This is the problem of manually specifying the loops
bobi
Slobodan Blazeski
> Does that mean you need to maintain the mapping from the slice back into
> the original array?
>
> In other words, if you so something like
>
> (setf (aref (elide foo t 4) 1) -9)
No I need two things for array foo:
0 1 2 3 4
5 6 7 8 9
I need to be able to do :
(slice foo t 4)
Returns new created array :
4 9
And a setter :
(setf (slice foo t 4) 17)
Now foo is:
0 1 2 3 17
5 6 7 8 17
>
> would you want the original array foo updated to
> 0 1 2 3 4
> 5 6 7 8 -9
> as well?
>
> That would require a much more involved solution compared to just
> extracting a COPY of some part of the existing array.
Actually all I need is variations generator then I will create array
with res-array and aref will do all the job:
For (slice foo t 4)
The access indices are:
0 4
1 4
For three dimensional array
baz (2 3 2)
0 1 2
3 4 5
6 7 8
9 10 11
For filling (slice baz 0 t t)
I need indices:
0 0 0
0 0 1
0 1 0
0 1 1
0 2 0
0 2 1
(let ((res (make-res-array ..))
(i -1)
(subscripts (gen-subcripts)))
(loop
(if (null subscripts)
(return res)
(setf (row-major-aref res (incf i))
(apply #'aref subscripts))))
That's it.
cheers
bobi
Slobodan Blazeski
wrote:
(let ((res (make-res-array ..))
(i -1)
(subscripts (gen-subcripts)))
(loop
(if (null subscripts)
(return res)
(progn
(setf (row-major-aref res (incf i))
(apply #'aref subscripts))
(setq subscripts (gen-subscripts))); update with new
indices
Slobodan Blazeski
> which will supersede affi. Currently beta.
>
> I will push a new version on Monday, with set-view that does what you
> want. You can also write it yourself, using rm-subscripts to convert
> flat indices 0,...,1-size to subscripts. Sorry, I have to rush now,
> catching an airplane,otherwise I would write it for you.
>
> HTH,
>
> Tamas
bobi
Barry Margolin
<459bdc7a-0cb1-47e0...@c9g2000yqm.googlegroups.com>,
Slobodan Blazeski <slobodan...@gmail.com> wrote:
> On Jun 10, 7:48�pm, t...@sevak.isi.edu (Thomas A. Russ) wrote:
>
> > Does that mean you need to maintain the mapping from the slice back into
> > the original array?
> >
> > In other words, if you so something like
> >
> > �(setf (aref (elide foo t 4) 1) -9)
> No I need two things for array foo:
> 0 1 2 3 4
> 5 6 7 8 9
>
>
> I need to be able to do :
> (slice foo t 4)
> Returns new created array :
> 4 9
>
> And a setter :
> (setf (slice foo t 4) 17)
>
> Now foo is:
> 0 1 2 3 17
> 5 6 7 8 17
This seems like a strange requirement. Usually accessors and setters
mirror each other -- if the accessor returns an array, the setter should
take an array as the new value. So shouldn't it be:
(setf (slice foo t 4) #(17 17))
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Slobodan Blazeski
> This seems like a strange requirement. Usually accessors and setters
> mirror each other -- if the accessor returns an array, the setter should
> take an array as the new value. So shouldn't it be:
>
> (setf (slice foo t 4) #(17 17))
Yes you will be able to specify that too.
I want to give a flavor of apl/j/q integrated looping and treatment of
array operations:
The operators prefixed with a are array versions of normal
operators
(a+ 1 2)
3
(a- 1 (vector 2 3 4))
#(-1 -2 -3)
(a* #(2 3 4) #(3 4 5))
#(6 12 20)
Here's the code so far
http://paste.lisp.org/display/81678
you can see the treatment of slicing at https://code.kx.com/trac/wiki/QforMortals2/lists
user/pass anonymous
cheers
bobi
http://www.linkedin.com/pub/slobodan-blazeski/6/b9b/151
Pascal J. Bourguignon
> On Jun 10, 12:24�pm, p...@informatimago.com (Pascal J. Bourguignon)
> wrote:
>>
>> If you don't need to extract the slice,
> I must return a new array as a result of slicing.
Then you will have to copy the slice to an array, which you can easily
do by generating a copier function like the reader.
Here is again the code with added a copier function.
(defun argument-list (indices)
(mapcar (lambda (i)
(if (integerp i)
i
(gensym)))
indices))
(defun parameter-list (argument-list)
(remove-if (function integerp) argument-list))
(defun make-reader (indices)
(let ((args (argument-list indices)))
(compile nil `(lambda (array ,@(parameter-list args))
(aref array ,@args)))))
(defun make-writer (indices)
(let ((args (argument-list indices)))
(compile nil `(lambda (new-value array ,@(parameter-list args))
(setf (aref array ,@args) new-value)))))
(defmacro popn (n stack &environment env)
(multiple-value-bind (vars vals store-vars writer-form reader-form)
(get-setf-expansion stack env)
(when (cdr store-vars) (error "Can't expand this."))
(let ((vstore (car store-vars)))
`(let* (,@(mapcar (function list) vars vals)
(,vstore ,reader-form))
(progn
,@(if (integerp n)
(let ((vn (1- n)))
(cond
((zerop vn) '())
((= 1 vn) `((setf ,vstore (cdr ,vstore))))
(t `((setf ,vstore (nthcdr ,vn ,vstore))))))
`((setf ,vstore (nthcdr (1- ,n) ,vstore))))
(prog1 (pop ,vstore)
,writer-form))))))
(defun slice-indices-offsets (indices)
(loop
:for offset = (position-if-not (function integerp) indices)
:while offset
:collect offset
:do (popn (1+ offset) indices)))
(defun make-project-list (indices)
(compile nil `(lambda (list)
(list ,@(mapcar (lambda (offset) `(popn ,(1+ offset) list))
(slice-indices-offsets indices))))))
(defun make-copier (indices dimensions)
(let* ((args (argument-list indices))
(pars (parameter-list args)))
(compile nil `(lambda (array)
(let ((copy (make-array ',dimensions)))
,(loop
:with form = `(setf (aref copy ,@pars) (aref array ,@args))
:for index :in (reverse pars)
:for maxim :in (reverse dimensions)
:do (setf form `(loop :for ,index :below ,maxim :do ,form))
:finally (return form))
copy)))))
(defclass slice ()
((array :initarg :array)
(reader :initarg :reader)
(writer :initarg :writer)
(copier :initarg :copier)))
(defun slice (array &rest indices)
(make-instance 'slice :array array
:reader (make-reader indices)
:writer (make-writer indices)
:copier (make-copier indices
(funcall (make-project-list indices)
(array-dimensions array)))))
(defmethod copy-to-array ((self slice))
(funcall (slot-value self 'copier) (slot-value self 'array)))
(defmethod ref ((self slice) &rest indices)
(apply (slot-value self 'reader) (slot-value self 'array) indices))
(defmethod (setf ref) (new-value (self slice) &rest indices)
(apply (slot-value self 'writer) new-value (slot-value self 'array) indices))
(let* ((a #4A((((1101 1102 1103)
(1111 1112 1113)
(1121 1122 1123))
((1201 1202 1203)
(1211 1212 1213)
(1221 1222 1223))
((1301 1302 1303)
(1311 1312 1313)
(1321 1322 1323)))
(((2101 2102 2103)
(2111 2112 2113)
(2121 2122 2123))
((2201 2202 2203)
(2211 2212 2213)
(2221 2222 2223))
((2301 2302 2303)
(2311 2312 2313)
(2321 2322 2323)))))
(s (slice a 1 t 2 t)))
(copy-to-array s))
--> #2A((2121 2122 2123) (2221 2222 2223) (2321 2322 2323))
>> (let* ((a #4A((((1101 1102 1103)
>> [...]
>> � � � �(s (slice a 1 t 2 t)))
>> � (setf (ref s 1 1) 0)
>> � (loop for i from 0 to 2
>> � � �do (loop for j from 0 to 2
>> � � � � � �do (princ (ref s i j)) (princ " ")
>> � � � � � �finally (terpri)))
>> � a)
> This is the problem of manually specifying the loops
Well obviously, you may add slots to the slice class to be able to
implement all the nice functions like the one we have on arrays, such
as rank, dimensions, row-major reference, etc.
--
__Pascal Bourguignon__
Barry Margolin
<83b55227-c78d-40a9...@h23g2000vbc.googlegroups.com>,
Slobodan Blazeski <slobodan...@gmail.com> wrote:
> On Jun 11, 1:17�am, Barry Margolin <bar...@alum.mit.edu> wrote:
>
> > This seems like a strange requirement. �Usually accessors and setters
> > mirror each other -- if the accessor returns an array, the setter should
> > take an array as the new value. �So shouldn't it be:
> >
> > (setf (slice foo t 4) #(17 17))
> Yes you will be able to specify that too.
Then you have an ambiguity. Since arrays are first-class objects, an
element of an array can be another array. So does the above expression
mean to set each element of the slice to #(17 17), or set each element
to 17?
IIRC, APL gets around this by requiring you to use special syntax to
create references to arrays.
Slobodan Blazeski
> In article
> <83b55227-c78d-40a9-b36f-c2852027b...@h23g2000vbc.googlegroups.com>,
> Slobodan Blazeski <slobodan.blaze...@gmail.com> wrote:
>
> > On Jun 11, 1:17 am, Barry Margolin <bar...@alum.mit.edu> wrote:
>
> > > This seems like a strange requirement. Usually accessors and setters
> > > mirror each other -- if the accessor returns an array, the setter should
> > > take an array as the new value. So shouldn't it be:
>
> > > (setf (slice foo t 4) #(17 17))
> > Yes you will be able to specify that too.
>
> Then you have an ambiguity. Since arrays are first-class objects, an
> element of an array can be another array. So does the above expression
> mean to set each element of the slice to #(17 17), or set each element
> to 17?
would require to box it first, thus making it a scalar.
Actually under q it will be impossible to setf elements in an array
with elements of type integer with a vector.
>
> IIRC, APL gets around this by requiring you to use special syntax to
> create references to arrays.
I never worked with APL, only j and q. For this exercise I use q
terminal as reference point how should operators behave.
bobi
Slobodan Blazeski
wrote:
(defun slice (array &rest indices)
(make-instance 'slice :array array
:reader (make-reader indices)
:writer (make-writer indices)
:copier (make-copier indices
(funcall (make-project-
list indices)
(array-
cheers
bobi
Pascal J. Bourguignon
> Great this is just what I needed, many thanks:
> (defun slice (array &rest indices)
> (copy-to-array
> (make-instance 'slice :array array
> :reader (make-reader indices)
> :writer (make-writer indices)
> :copier (make-copier indices
> (funcall (make-project-list indices)
> (array-dimensions array))))))
Here the writer is not used, you may leave it uninitialized.
--
__Pascal Bourguignon__
Barry Margolin
<175b0d88-cde4-415e...@y9g2000yqg.googlegroups.com>,
Slobodan Blazeski <slobodan...@gmail.com> wrote:
> On Jun 11, 1:41�am, Barry Margolin <bar...@alum.mit.edu> wrote:
> > In article
> > <83b55227-c78d-40a9-b36f-c2852027b...@h23g2000vbc.googlegroups.com>,
> > �Slobodan Blazeski <slobodan.blaze...@gmail.com> wrote:
> >
> > > On Jun 11, 1:17�am, Barry Margolin <bar...@alum.mit.edu> wrote:
> >
> > > > This seems like a strange requirement. �Usually accessors and setters
> > > > mirror each other -- if the accessor returns an array, the setter should
> > > > take an array as the new value. �So shouldn't it be:
> >
> > > > (setf (slice foo t 4) #(17 17))
> > > Yes you will be able to specify that too.
> >
> > Then you have an ambiguity. �Since arrays are first-class objects, an
> > element of an array can be another array. �So does the above expression
> > mean to set each element of the slice to #(17 17), or set each element
> > to 17?
> It means to set each element to 17. Setting each element to #(17 17)
> would require to box it first, thus making it a scalar.
> Actually under q it will be impossible to setf elements in an array
> with elements of type integer with a vector.
I've never heard of q, I'm talking about Common Lisp. I didn't see
anywhere in your description of this problem that the arrays only
contain integers. By default Lisp doesn't require arrays to be
homogeneous.
Slobodan Blazeski
don't know, and extremely fast columnar database http://www.kx.com/ .
I've thought first of basing it on j http://www.jsoftware.com/ but
that will be much more work to do in order to support tacit style.
> I'm talking about Common Lisp. I didn't see
> anywhere in your description of this problem that the arrays only
> contain integers.
q as a reference point I would try to stick as close to q as I can, to
keep my reference point valid. Else I would have to design the
language something that's a lot of work to do it right.
> By default Lisp doesn't require arrays to be
> homogeneous.
experience.
cheers
bobi
http://www.linkedin.com/pub/slobodan-blazeski/6/b9b/151
Slobodan Blazeski
http://paste.lisp.org/display/82220#1
Printing mimics j/q style of printing arrays with example at
http://slobodanblazeski.blogspot.com/2009/06/dissecting-masterpiece.html