negative indexes in vectors
didier deshommes
Is this a bug? negative indexes are currently not allowed for vectors:
{{{
sage: vector(RR,range(3))[2]
2.00000000000000
sage: vector(RR,range(3))[-1]
----------------------------------------------------
/home/dfdeshom/custom/sage/devel/sage-gcd2/<ipython
console> in <module>()
/home/dfdeshom/custom/sage/devel/sage-gcd2/free_modu
le_element.pyx in sage.modules.free_module_element.F
reeModuleElement_generic_dense.__getitem__()
<type 'exceptions.IndexError'>: index (i=-1) must be
between 0 and 2
}}}
I can post a patch if interested.
didier
William Stein
That was a design decision I made a long time ago. In retrospect it is
not very pythonic. Feel free to open a trac ticket and post a patch. Thanks!
William
Mike Hansen
--Mike
Soroosh Yazdani
I think we should stick to the original throw the exception way.
Soroosh
William Stein
> so is this really a good idea? Based on William's comment, I'm assuming that
> this is consistent with python index handling, but usually basis vectors
> aren't indexed cyclically in math world.
>
> I think we should stick to the original throw the exception way.
Some points:
* Wrap-around index is one of the standard idioms in Python.
* Sage matrices already behave this way for getting rows but *NOT* for
getting elements, so if we make this change we introduce an _inconsistency_
with matrices
sage: a = random_matrix(RDF,2); a
[-0.460858935239 -0.823833430104]
[ 0.496208720606 0.481204094124]
sage: a[-1]
(0.496208720606, 0.481204094124)
sage: a[-1,1]
Traceback (most recent call last):
...
IndexError: matrix index out of range
* Numpy arrays allow negative indices for getting rows *and*
getting entries,
though the error message says they don't:
sage: b = a.numpy()
sage: b[-1]
array([ 0.49620872, 0.48120409])
sage: b[-1,-1]
0.48120409412365595
sage: b[2,2]
Traceback (most recent call last):
...
IndexError: index (2) out of range (0<=index<=2) in dimension 0
* Matlab, Magma and PARI do not allow accessing matrices using
negative indices
(in fact, I can't think of any program but numpy that *does*):
MATLAB
>> a = rand(2)
a =
0.4447 0.7919
0.6154 0.9218
>> a(0,0)
??? Subscript indices must either be real positive integers or logicals.
>> a(1,1)
ans =
0.4447
>> a(-1,-1)
??? Subscript indices must either be real positive integers or logicals.
MAGMA
> A := RMatrixSpace(Integers(),2,2)![1..4];
> A[1,1];
1
> A[-1,-1];
>> A[-1,-1];
^
Runtime error in '[]': Argument 2 (-1) should be in the range [1 .. 2]
PARI:
? a = matrix(2,2,i,j,i+j)
%1 =
[2 3]
[3 4]
? a[1,1]
%2 = 2
? a[-1,-1]
*** array index (-1) out of allowed range [1-2]: a[-1,-1]
-----------
In light of everything above, along with Soroosh comment, and finally that when
implementing algorithms getting indices off is a common source of error, I now
vote for *not* allowing negative indices. E.g., often one prototypes an
algorithm in Python (with bounds checking) to make sure there are no
index errors,
then switches to C (without bounds checking) to make it fast. If we
allow negative
indexing, this could lead to weird segfaults.
Thoughts?
William
John Cremona
I agree. No negative indices for vectors or matrices.
--
John Cremona
didier deshommes
As you pointed out, the above CASs do not use negative indexing
anywhere I can think of in their basic data structures so I think they
are justified in not allowing it for vectors. Python does so I expect
v[-1] to be valid. The main reason I wanted this was so I would get
the next-to-last element in a vector without worrying (explicitly)
about its length.
>
> In light of everything above, along with Soroosh comment, and finally that when
> implementing algorithms getting indices off is a common source of error, I now
> vote for *not* allowing negative indices. E.g., often one prototypes an
> algorithm in Python (with bounds checking) to make sure there are no
> index errors,
> then switches to C (without bounds checking) to make it fast. If we
> allow negative
> indexing, this could lead to weird segfaults.
Could you explain this a little more (the segfaults)? From my POV,
v[-1] is just a shortcut for v[len(v)-1] which works well for me when
I'm writing Python code. That wouldn't be valid in C so we would have
to use positive indices anyway.
didier
mhampton
personally I hope that Sage tries to stay as pythonic as possible. I
often use negative indices, and I think it would be great if Sage
vectors behaved as much like python lists as possible. It sounds like
I am in the minority though.
-M. Hampton
William Stein
One could write a Python function that is algorithmically wrong but
doesn't raise
an IndexError exceptions. When translating that code to C it would suddenly
segfault.
I am looking from hearing more feedback from people. So far the vote is:
Against negative indices: Cremona, Soroosh, all other math software
For negative indices: Didier, Marshall Hampton, the Numpy/Scipy community
I'm not completely decided yet, though I tend toward being against (given
that I how I already implemented things).
-- William
Nick Alexander
On 28-Feb-08, at 10:39 AM, mhampton wrote:
>
> One of the reasons I switched to Sage is that I like python a lot, and
> personally I hope that Sage tries to stay as pythonic as possible. I
> often use negative indices, and I think it would be great if Sage
> vectors behaved as much like python lists as possible. It sounds like
> I am in the minority though.
Then I also am in the minority. Slicing and dicing sage datatypes
using python's conventions is very powerful; I do it all the time.
I would really like, by abuse of notation, S[slice] == S[range(len(S)
+1)[slice]] to hold for all sensible datatypes. A matrix is not list-
like, so I don't expect such a property to hold.
Nick
William Stein
Ironically in Sage, matrices are exactly the one place where that
property *does* hold. A matrix can be viewed as list-like if you view
it as the list of its rows:
sage: m = random_matrix(ZZ,3); m
[-2 -2 -1]
[-1 -3 -1]
[ 2 1 2]
sage: m[0:-1]
[-2 -2 -1]
[-1 -3 -1]
Looking at the code for m.__getitem__ one finds:
return self.matrix_from_rows(range(0, self._nrows).__getitem__(key))
-- William
didier deshommes
> One could write a Python function that is algorithmically wrong but doesn't raise
> an IndexError exceptions. When translating that code to C it would suddenly
> segfault.
Yes, I see. I could debate that the error lies in the programmer, but
I see how this more rigorous rule might help the programmer.
> Against negative indices: Cremona, Soroosh, all other math software
Minus Maple :)
> with(LinearAlgebra):
> v:= Vector([1,2,3]):
> v[-1];
3
> M:=RandomMatrix(4):
> M[1]
> M[1,3];
8
> M[-1,3];
29
> M[1..-1,3];
[ 8]
[ ]
[69]
[ ]
[99]
[ ]
[29]
Then again , I don't think whether or not a CAS allows negative
indexing is a big deal because each CAS is dictated by its basic data
structures.
David Harvey
On Feb 28, 2008, at 1:42 PM, William Stein wrote:
> One could write a Python function that is algorithmically wrong but
> doesn't raise
> an IndexError exceptions. When translating that code to C it
> would suddenly
> segfault.
>
> I am looking from hearing more feedback from people. So far the
> vote is:
>
> Against negative indices: Cremona, Soroosh, all other math software
>
> For negative indices: Didier, Marshall Hampton, the Numpy/Scipy
> community
>
> I'm not completely decided yet, though I tend toward being against
> (given
> that I how I already implemented things).
Hmmmm.... I'm so torn.
If I had it my way, I would get rid of negative indices in python
itself. They're cute and I admit they are convenient (I use them
frequently), but in my gut I don't like them.
HOWEVER, given that this is how python is, and that's not going to
change, I think it's better to do what python programmers expect,
which is to support negative indexing. Vectors are very much like
sequences, and they should behave like sequences. For example, if you
have a function that takes a sequence as input, it would be nice if
that kind of function Just Worked on a vector.
david
Joel B. Mohler
> > > In light of everything above, along with Soroosh comment, and finally
> > that when > implementing algorithms getting indices off is a common
> > source of error, I now > vote for *not* allowing negative indices.
> > E.g., often one prototypes an > algorithm in Python (with bounds
> > checking) to make sure there are no > index errors,
> > > then switches to C (without bounds checking) to make it fast. If we
> > > allow negative
> > > indexing, this could lead to weird segfaults.
> >
> > Could you explain this a little more (the segfaults)? From my POV,
> > v[-1] is just a shortcut for v[len(v)-1] which works well for me when
> > I'm writing Python code. That wouldn't be valid in C so we would have
> > to use positive indices anyway.
>
> One could write a Python function that is algorithmically wrong but
> doesn't raise
> an IndexError exceptions. When translating that code to C it would
> suddenly segfault.
I don't understand this argument. All of these same things could be said when
using python lists as well. The take-away is that when converting from
python/sage code to C you need to be aware of python semantics with indices.
That said, I'm with mhampton, I am a sage user because I like python.
Therefore, let us make sage idioms which build on python idioms. After all,
the standard advice to new users is "learn python first". I think it is
excellent advice.
--
Joel
Robert Bradshaw
> I am looking from hearing more feedback from people. So far the
> vote is:
>
> Against negative indices: Cremona, Soroosh, all other math software
>
> For negative indices: Didier, Marshall Hampton, the Numpy/Scipy
> community
>
> I'm not completely decided yet, though I tend toward being against
> (given
> that I how I already implemented things).
Vectors are so much like lists, that I think that we should allow
negative indexing as well. However, I don't think we should allow
negative indexing for everything (e.g. I think it's odd for matrices,
and wouldn't want it on polynomials, etc.)
- Robert
Dan Christensen
> I am looking from hearing more feedback from people.
Not only would I vote for negative indices, I would like more of the
numpy features to work within Sage, when appropriate:
- indexing with slices
- indexing with ...
- broadcasting
- views
As a somewhat artificial example, the combination of these allows things
like a[::2,0] *= -1 to negate every second element in the 0th column.
Numpy arrays are a real pleasure to work with, and they are the de facto
standard for numerical work in python, so being as compatible with them
as possible is important.
In fact, I sometimes wonder if Sage should simply use numpy arrays for
its array types...
Dan
Jason Grout
> On Thu, Feb 28, 2008 at 1:42 PM, William Stein <wst...@gmail.com> wrote:
>> One could write a Python function that is algorithmically wrong but doesn't raise
>> an IndexError exceptions. When translating that code to C it would suddenly
>> segfault.
>
> Yes, I see. I could debate that the error lies in the programmer, but
> I see how this more rigorous rule might help the programmer.
>
>> Against negative indices: Cremona, Soroosh, all other math software
>
> Minus Maple :)
and Minus Mathematica :)
In[1]:= v={1,2,3}
Out[1]= {1, 2, 3}
In[2]:= v[[-1]]
Out[2]= 3
In[4]:= v[[1;;2]]
Out[4]= {1, 2}
(the 1;;2 is equivalent to [1..2] in Sage)
So now half of our mission-statement CASs have negative indices for vectors.
Personally, I'm for negative indices since it's a python idiom and
generally, if there's a debatable call, we typically tend to err on the
side of being consistent with Python, especially in this case since the
other major vector/matrix python package (numpy) has negative indices.
I also agree that slicing is a good thing.
Jason
David Joyner
+1 for wrap-around indices for several reasons, all mentioned already
in this thread.
>
>
>
> William
>
> >
>
boo...@u.washington.edu
+1 for -1
John Cremona
su=ince they look like Python lists and negative indices are ok in
Python lists is a weak one.
There are many ways in which vectors are *not* like Python lists,
since they represent a specific mathematical construct. I am waiting
for someone to suggest that for vectors v and w, v+w should return
their concatenation -- since that is what happens for Python lists --
make us use v.add(w) for vector addition!
John
2008/2/28 <boo...@u.washington.edu>:
--
John Cremona
Jaap Spies
> That said, I'm with mhampton, I am a sage user because I like python.
> Therefore, let us make sage idioms which build on python idioms. After all,
> the standard advice to new users is "learn python first". I think it is
> excellent advice.
>
+1
Jaap
David Roe
Justin C. Walker
On Feb 28, 2008, at 10:42 AM, William Stein wrote:
> On Thu, Feb 28, 2008 at 10:20 AM, didier deshommes
> <dfde...@gmail.com> wrote:
>
> I am looking from hearing more feedback from people. So far the
> vote is:
>
> Against negative indices: Cremona, Soroosh, all other math software
>
> For negative indices: Didier, Marshall Hampton, the Numpy/Scipy
> community
The decision I think should be based on the effectiveness of the
resulting system. If we diverge too much from Python, then two
things happen:
- we lose what advantage Python gives us
- we make the system that much harder to maintain,
since we have to worry about patching everything
in sight as time goes on.
I think making this change (outlawing negative indices in lists) will
have long-term negative effects on Sage.
I'm +1 for -1 :-}
Justin
--
Justin C. Walker, Curmudgeon-At-Large
Director
Institute for the Enhancement of the Director's Income
--------
"Weaseling out of things is what separates us from the animals.
Well, except the weasel."
- Homer J Simpson
--------
William Stein
>
>
> On Feb 28, 2008, at 10:42 AM, William Stein wrote:
> > On Thu, Feb 28, 2008 at 10:20 AM, didier deshommes
> > <dfde...@gmail.com> wrote:
> [snip]
>
> >
> > I am looking from hearing more feedback from people. So far the
> > vote is:
> >
> > Against negative indices: Cremona, Soroosh, all other math software
> >
> > For negative indices: Didier, Marshall Hampton, the Numpy/Scipy
> > community
>
> The decision I think should be based on the effectiveness of the
> resulting system. If we diverge too much from Python, then two
> things happen:
> - we lose what advantage Python gives us
> - we make the system that much harder to maintain,
> since we have to worry about patching everything
> in sight as time goes on.
>
> I think making this change (outlawing negative indices in lists) will
> have long-term negative effects on Sage.
Nobody is proposing "outlawing negative indices in lists". The proposal
was to not change the current behavior of Sage, which is as follows:
sage: v = vector([1,2,3]); v
(1, 2, 3)
sage: v[-1]
Traceback (most recent call last):
...
IndexError: index out of range
to be like this:
sage: v = vector([1,2,3]); v
(1, 2, 3)
sage: v[-1]
3
This suggested change makes some people nervous because in
mathematics nobody would ever write $v_{-1}$ for the last entry of a
vector $(v_i)$. However, leaving things as they are is also annoying
since for a list v, v[-1] gives the last entry, and the mathematical object
"vector" is really a lot like a list, so it would be nice if this
Python/Mathematica/Maple convention were also available in Python.
The majority vote at present is for allowing negative indices.
There was also one suggestion to just get rid of all Sage matrices
and vectors and use numpy. Sigh. I wish things were so easy....
-- William
John Cremona
2008/2/28 Justin C. Walker <jus...@mac.com>:
>
>
> On Feb 28, 2008, at 10:42 AM, William Stein wrote:
> > On Thu, Feb 28, 2008 at 10:20 AM, didier deshommes
> > <dfde...@gmail.com> wrote:
> [snip]
>
Director
> Institute for the Enh> >
> > I am looking from hearing more feedback from people. So far the
> > vote is:
> >
> > Against negative indices: Cremona, Soroosh, all other math software
> >
> > For negative indices: Didier, Marshall Hampton, the Numpy/Scipy
> > community
>
> The decision I think should be based on the effectiveness of the
> resulting system. If we diverge too much from Python, then two
> things happen:
> - we lose what advantage Python gives us
> - we make the system that much harder to maintain,
> since we have to worry about patching everything
> in sight as time goes on.
>
> I think making this change (outlawing negative indices in lists) will
> have long-term negative effects on Sage.
>
I never voted for any change whatsoever to what one can do with lists.
I just agreed with WIlliam and others that for *vectors*, which are
not lists but may be based on them for a small part of their
functionality, allowing negative indices is unnecessary and also
makes porting sage code to C harder to get right. Everything ython
allows for lists should still be allowed, for ever and ever.
John
> I'm +1 for -1 :-}
>
> Justin
>
> --
> Justin C. Walker, Curmudgeon-At-Large
> a
--
John Cremona
Mike Hansen
> I just agreed with WIlliam and others that for *vectors*, which are
> not lists but may be based on them for a small part of their
> functionality, allowing negative indices is unnecessary and also
> makes porting sage code to C harder to get right. Everything ython
> allows for lists should still be allowed, for ever and ever.
On the other hand, it makes porting Python/Sage code using lists to
using vectors harder to get right. I think that that case is probably
more common than moving from a vector object to a C array.
--Mike
John Cremona
But if you want to use vector operations (scalar multiplcation,
addition, and more complicated things such as matrix*vector) then you
would have never used lists in the first place. And if you don't,
then you don't need to convert from lists to vectors anyway!
John
> --Mike
>
>
>
> >
>
--
John Cremona
William Stein
This discussion is getting long and a little heated, and I'm just
going to use my little BDFL powers to declare that this should
work in Sage, i.e., yes to negative indexes:
sage: v = vector([1,2,3]); v
(1, 2, 3)
sage: v[-1]
3
It's clear this is most consistent with Python, and that most
(not all) people are comfortable with it.
John Cremona and I are personally the main people against
it -- I'm against it in the sense that I implemented vectors and
I definitely chose not to allow negative indexing, and also
coming from a Magma/PARI/C++ perspective doing so seems
really weird. But it's completely clear what the community wants
overall, and what is needed for consistency with Python, and
that is negative indexing, so +1 to -1. :-)
-- William
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
John Cremona
Soroosh Yazdani
At the same time, it is unclear to me how useful of a feature this is. The only benefit seems that if a code written in python is using list, and decides to switch to using vectors, the transition will be easier. I don't find the overhead that huge, but then again, I usually program in C when not in SAGE.
Soroosh
Joel B. Mohler
Well, yes, but I've actually have already done such a port in my own sage code
for exactly this reason. That is, I had code using python lists. I then
realized that I wanted to use mathematical vector operations on them as well.
Hence, a port of code from python list to sage vectors.
Personally, I find that to be a very natural migration path. But, that is
because I came to sage as a python user. Hence, I start writing in python
and grow into the sage construct.
--
Joel
Soroosh Yazdani
mhampton
and agree with this post - I have also ported code from python to sage
and been irritated at the lack of negative indices for vectors (this
was for some polytope code, moving around some data as lists and then
converting to sage vectors).
-M. Hampton
On Feb 28, 3:06 pm, "Joel B. Mohler" <j...@kiwistrawberry.us> wrote:
> On Thursday 28 February 2008 15:44, John Cremona wrote:
>
>
>