List slice assignment and custom sequences
Ronald Oussoren
I have a custom sequence-class (actually a proxy for a native type) and
would like to use this just like a normal list. This is mostly working
just fine by defining __getitem__ et.al., but I cannot use the type as
the RHS of an assigment to a list slice.
I traced this to the implementation of this feature in type 'list': It
requires that the new version of the slice is also a 'list':
l = [ 1, 2, 3, 4]
l[1:3] = range(10) # works fine
l[1:3] = xrange(10) # raises TypeError
Why must the RHS side of 'l[1:3] = expr()' be an instance of 'list'?
Ronald
Erik Max Francis
> I traced this to the implementation of this feature in type 'list': It
> requires that the new version of the slice is also a 'list':
>
> l = [ 1, 2, 3, 4]
> l[1:3] = range(10) # works fine
> l[1:3] = xrange(10) # raises TypeError
>
> Why must the RHS side of 'l[1:3] = expr()' be an instance of 'list'?
Well, you want to splice one list into another. Both have to be lists.
To convert any sequence S to an actual list, use the list builtin:
list(S)
So you'd just write your problem code as
l[1:3] = list(xrange(10))
Although in this case, needless to say, range(10) would make a great
deaal more sense.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, USA / 37 20 N 121 53 W / &tSftDotIotE
/ \ Triumph cannot help being cruel.
\__/ Jose Ortega y Gasset
Alcyone Systems' Daily Planet / http://www.alcyone.com/planet.html
A new, virtual planet, every day.
Ronald Oussoren
On Saturday, Nov 2, 2002, at 20:39 Europe/Amsterdam, Erik Max Francis
wrote:
> Ronald Oussoren wrote:
>
>> I traced this to the implementation of this feature in type 'list': It
>> requires that the new version of the slice is also a 'list':
>>
>> l = [ 1, 2, 3, 4]
>> l[1:3] = range(10) # works fine
>> l[1:3] = xrange(10) # raises TypeError
>>
>> Why must the RHS side of 'l[1:3] = expr()' be an instance of 'list'?
>
> Well, you want to splice one list into another. Both have to be lists.
No, I want to replace part of a sequence by another sequence. I don't
understand _why_ the RHS must be a list if the LHS is one. It doesn't
even allow a tuple, even though tuples can be used instead of lists in
almost all code that treats the sequence as read-only.
> To convert any sequence S to an actual list, use the list builtin:
>
> list(S)
>
> So you'd just write your problem code as
>
> l[1:3] = list(xrange(10))
>
> Although in this case, needless to say, range(10) would make a great
> deaal more sense.
Sure, but xrange was just an example. FYI the actual type is NSArray
(http://pyobjc.sourceforge.net). We'd like to make this type act as
closely as possible like a normal list to avoid unnecessary conversions
to/from python types.
Ronald
Erik Max Francis
> No, I want to replace part of a sequence by another sequence.
If the sequence that your splicing into is a custom type, then it's
going to depend on whether such splicing is implemented and how.
Presumably it would want a sequence on the right-hand side of the same
type as the left-hand side.
> I don't
> understand _why_ the RHS must be a list if the LHS is one. It doesn't
> even allow a tuple, even though tuples can be used instead of lists in
> almost all code that treats the sequence as read-only.
It just sounds like common sense to me; the splicing operation should
take the same type of sequence on the right-hand side as the left.
Since Python allows you trivial conversion from _any_ sequence to a list
with the list function/type, it seems like a non-problem to me if the
right-hand side isn't a list to start with.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, USA / 37 20 N 121 53 W / &tSftDotIotE
/ \ Weakness of attitude becomes weakness of character.
\__/ Albert Einstein
WebVal / http://www.alcyone.com/pyos/webval/
URL scanner, maintainer, and validator in Python.
Emile van Sebille
> Sure, but xrange was just an example. FYI the actual type is NSArray
> (http://pyobjc.sourceforge.net). We'd like to make this type act as
> closely as possible like a normal list to avoid unnecessary
conversions
> to/from python types.
>
Well then, as a list won't do what you want, at least since object/class
unification you can get closer by shadowing list:
F:\Python22>python
ActivePython 2.2.1 Build 222 (ActiveState Corp.) based on
Python 2.2.1 (#34, Apr 15 2002, 09:51:39) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import generators
>>> def xx():
... for i in range(10):
... yield i
...
>>>
>>> def listify(listwithiterables):
... retval = []
... from types import StringTypes
... try:
... for ii in listwithiterables:
... if type(ii) in StringTypes:
... retval.append(ii)
... else:
... try:
... retval.extend(ii)
... except:
... retval.append(ii)
... except: pass
... return retval
...
>>>
>>> class List(list):
... setslice = list.__setslice__
... def __setslice__(self, beg, end, iterobj):
... List.setslice(self, beg, end, listify(iterobj))
...
>>>
>>> list=List
>>>
>>> l = list((1,2,3,4))
>>>
>>> l[1:3] = xrange(10)
>>> l[6:10] = xx()
>>>
>>> print l
[1, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 4]
>>>
>>> l = [1,2,3]
>>> l[:]=xx()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: must assign list (not "generator") to slice
>>> l = list(l)
>>> l[:]=xx()
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
--
Emile van Sebille
em...@fenx.com
---------
Michael Hudson
> No, I want to replace part of a sequence by another sequence. I don't
> understand _why_ the RHS must be a list if the LHS is one.
Because Objects/listobject.c:list_ass_slice pokes directly into the
object passed on the RHS.
A patch to change this would have some chance of getting accepted;
wouldn't like to guess what, but I'd hazard non-zero.
Cheers,
M.
--
All programs evolve until they can send email. -- Richard Letts
Except Microsoft Exchange. -- Art
-- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
Alex Martelli
> Ronald Oussoren <ouss...@cistron.nl> writes:
>
>> No, I want to replace part of a sequence by another sequence. I don't
>> understand _why_ the RHS must be a list if the LHS is one.
>
> Because Objects/listobject.c:list_ass_slice pokes directly into the
> object passed on the RHS.
>
> A patch to change this would have some chance of getting accepted;
> wouldn't like to guess what, but I'd hazard non-zero.
Done -- just submitted as patch 633870 on Sourceforge, pls review.
Alex
Michael Hudson
Not only reviewed, but BDFL approved and checked in!
that-one-went-quickly-ly y'rs
M.
--
ZAPHOD: Listen three eyes, don't try to outwierd me, I get stranger
things than you free with my breakfast cereal.
-- The Hitch-Hikers Guide to the Galaxy, Episode 7
Ronald Oussoren
On Tuesday, Nov 5, 2002, at 15:03 Europe/Amsterdam, Michael Hudson
wrote:
> Ronald Oussoren <ouss...@cistron.nl> writes:
>
>> No, I want to replace part of a sequence by another sequence. I don't
>> understand _why_ the RHS must be a list if the LHS is one.
>
> Because Objects/listobject.c:list_ass_slice pokes directly into the
> object passed on the RHS.
>
> A patch to change this would have some chance of getting accepted;
> wouldn't like to guess what, but I'd hazard non-zero.
I sure hope so. I've posted a bug+patch at SF for a related problem:
With the introduction of new-style classes 'isinstance(obj, list)' no
longer guarantees that you can savely poke in the RHS instead of using
the __getitem__ accessor function.
Ronald
Terry Reedy
Ronald Oussoren <ouss...@cistron.nl> writes:
> No, I want to replace part of a sequence by another sequence. I
don't
> understand _why_ the RHS must be a list if the LHS is one.
The immediate, begging-the-question answer is that that is how slice
assignment is defined and hence, how it is programmed. Lang Ref 6.3:
"If the target is a slicing: The primary expression in the reference
is evaluated. It should yield a mutable sequence object (e.g., a
list). The assigned object should be a sequence object of the same
type."
But why define it so? Possibly simple pragmatics -- its easiest to
program that way (as per M. Hudson's answer). The other is the
constant dilemma facing the language definer: if something could be
correct, but could be an error, should the interpreter raise a flag?
or do the 'obvious' thing? -- which in this case is to call list() or
act as if it had been (which would actually be more efficient). Error
in this case seems unlikely enough that generalization seems
reasonable.
A third answer is inertia. Python has been moving to a more
generalized notion of sequence and even iterable but slice assignment
got overlooked -- until you gave a push. We'll see what happens for
2.3.
Terry J. Reedy
Michael Hudson
Well, that's gone in too, in effect. Guido seems to be in a receptive
mood today! It would be fun seeing what changes I could sneak in,
except I'm going to the pub :)
against-drinking-and-committing-ly y'rs
M.
--
Two things I learned for sure during a particularly intense acid
trip in my own lost youth: (1) everything is a trivial special case
of something else; and, (2) death is a bunch of blue spheres.
-- Tim Peters, 1 May 1998
Michael Hudson
> Ronald Oussoren <ouss...@cistron.nl> writes:
>
> > No, I want to replace part of a sequence by another sequence. I
> don't
> > understand _why_ the RHS must be a list if the LHS is one.
>
> The immediate, begging-the-question answer is that that is how slice
> assignment is defined and hence, how it is programmed. Lang Ref 6.3:
> "If the target is a slicing: The primary expression in the reference
> is evaluated. It should yield a mutable sequence object (e.g., a
> list). The assigned object should be a sequence object of the same
> type."
>
> But why define it so? Possibly simple pragmatics -- its easiest to
> program that way (as per M. Hudson's answer).
I'm 99% sure this is what happened -- I think list_ass_subscript is
very old code, possibly even predating __getitem__ on instances (if
there was such a time).
> The other is the constant dilemma facing the language definer: if
> something could be correct, but could be an error, should the
> interpreter raise a flag? or do the 'obvious' thing? -- which in
> this case is to call list() or act as if it had been (which would
> actually be more efficient). Error in this case seems unlikely
> enough that generalization seems reasonable.
One of the things used to be that handling user defined sequences is
subtle -- consider:
class Evil:
def __len__(self):
return 9
def __getitem__(self, i):
raise IndexError
Blindly trusting the answer of PyObject_Length() can easily lead to
core dumps. But now Python has this wonderful PySequence_Fast API
that wraps up all the nasty details while keeping the common case
fast, so supporting user-defined sequences is now trivial.
Everyone writing C extensions should know about PySequence_Fast().
> A third answer is inertia. Python has been moving to a more
> generalized notion of sequence and even iterable but slice assignment
> got overlooked -- until you gave a push.
Indeed.
> We'll see what happens for 2.3.
He pushed; it fell :)
Cheers,
M.
--
"Also, does the simple algorithm you used in Cyclops have a name?"
"Not officially, but it answers to "hey, dumb-ass!"
-- Neil Schemenauer and Tim Peters, 23 Feb 2001
Ronald Oussoren
On Tuesday, Nov 5, 2002, at 19:26 Europe/Amsterdam, Michael Hudson
wrote:
> Blindly trusting the answer of PyObject_Length() can easily lead to
> core dumps. But now Python has this wonderful PySequence_Fast API
> that wraps up all the nasty details while keeping the common case
> fast, so supporting user-defined sequences is now trivial.
>
> Everyone writing C extensions should know about PySequence_Fast().
Interesting, I'll have a look at this API.
>
>> A third answer is inertia. Python has been moving to a more
>> generalized notion of sequence and even iterable but slice assignment
>> got overlooked -- until you gave a push.
>
> Indeed.
Glad to be of help, even with an awful (and not applied) patch :-)
Ronald