Comment on PEP-0322: Reverse Iteration Methods
Raymond Hettinger
Basically, the idea looks like this:
for i in xrange(10).iter_backwards(): # 9,8,7,6,5,4,3,2,1,0
<do something with i>
The HTML version is much more readable than the ReST version.
See:
http://www.python.org/peps/pep-0322.html
Several interesting ideas surfaced in the pre-pep thread:
* Call it ireverse() instead of iter_backwards().
Good idea. This is much more pithy.
* Use object properties instead of methods.
I can't yet claim to understand what the author is really
proposing. It has something to do which providing
access to an object that responds to iter, getitem, and
getslice with reversed indices.
* using a single function that looks for an __riter__ magic
method and, if not found, use __getitem__ and __len__
to build a reverse iterator.
A workable version of this was posted.
It saves implementing some object methods at the
expense of adding a new builtin function and of
creating a new magic method name.
It is slower than direct access to the underlying object.
It crashes when applied to an infinite iterator.
It produces bizarre results when applied to mappings.
* special markers for infinite iterators
This idea is interesting but doesn't extend well
when the object is wrapped by another iterator.
Also, there is no automatic way to determine
which generators can be infinite.
Raymond Hettinger
Sean Ross
news:95rcb.2002$yU5....@nwrdny01.gnilink.net...
> Please comment on the new PEP for reverse iteration methods.
# from PEP 322
"""
Rejected Alternative Ideas
[snip]
Add a builtin function, reverse() ...
"""
Do you mean add a function reverse() (or ireverse()) to the itertools
library, or to __builtins__?
John Roth
+1
John Roth
"Raymond Hettinger" <vze4...@verizon.net> wrote in message
news:95rcb.2002$yU5....@nwrdny01.gnilink.net...
Stephen Horne
<vze4...@verizon.net> wrote:
>* Use object properties instead of methods.
>
> I can't yet claim to understand what the author is really
> proposing. It has something to do which providing
> access to an object that responds to iter, getitem, and
> getslice with reversed indices.
The idea is basically to have a way to get a proxy to a sequence which
behaves like the original sequence, but with indexes reversed. There
is no reason why member functions couldn't return those proxies
instead of using properties, except that when the result is sliced it
looks less ugly.
I have a demo of it for lists only (listing at end of post). It has
three properties...
backward : reversed, slicing returns a list
ibackward : reversed, slicing returns an iterator
iforward : forward, slicing returns an iterator
All properties currently support __getitem__, __setitem__,
__delitem__, __str__, __repr__, __iter__ and __len__ (they all return
an instance of the same proxy class but with different options set).
For simple iterating-over-ints, I'd use the same principles with that
object-acting-as-sliceable-set-of-all-integers idea from the PEP284
thread.
I had some problems getting the slice translation to work - there are
some little nasties with negative steps. You can write...
x [::-1]
...but trying to make it explicit...
x [len(x)-1:-1:-1]
...breaks it because of that -ve stop bound. Not being a Numeric user,
slice steps are quite a new thing to me - and that little trap just
caught me by surprise. It isn't too hard to work around it, though.
The code following isn't pretty but so far as I can tell it works.
Python 2.3 is assumed.
The question is - is the exercise worthwhile. I'm not sure even though
it's my own idea. I think it is nice in its way, and a lot more
flexible than the 'iter_backward' method proposal, but that is quite
likely a bad thing as it is probably massive overkill for the problem
being solved - and in particular the extra flexibility has costs.
What the hell, though - it was fun to play with for a bit!
Anyway, here is a quick test session...
>>> from rilist import rilist
>>> x=rilist(range(20))
>>> x.backward
[19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
>>> x.backward[:10]
[19, 18, 17, 16, 15, 14, 13, 12, 11, 10]
>>> x.backward[::2]
[19, 17, 15, 13, 11, 9, 7, 5, 3, 1]
>>> del x.backward[::2]
>>> x.backward
[18, 16, 14, 12, 10, 8, 6, 4, 2, 0]
>>> x
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
>>> for i in x.backward :
... print i
...
18
16
14
12
10
8
6
4
2
0
>>> for i in x.backward [5:]:
... print i
...
8
6
4
2
0
>>> for i in x [5:]:
... print i
...
10
12
14
16
18
I've done some 'ibackward' tests as well, and that seems to work - but
I haven't bothered with 'iforward'.
Here is the source - warning, it isn't very self-documenting ATM...
class c_List_Proxy (object) :
"""
Proxy to list class which supports a subset of methods with
modified behaviours, mainly...
- Slicing may create an iterator rather than a list.
- Subscripts and ranges may be reversed.
"""
def __init__ (self, p_List, p_Reverse=False, p_Iter_Slice=False) :
self.List = p_List
self.Reverse = p_Reverse
self.Iter_Slice = p_Iter_Slice
# need some support stuff
def __SliceIter__ (self, p_Slice) :
if p_Slice.step > 0 :
i = p_Slice.start
while i < p_Slice.stop :
yield self.List [i]
i += p_Slice.step
else :
i = p_Slice.start
while i > p_Slice.stop :
yield self.List [i]
i += p_Slice.step
def __FullIter__ (self) :
i = len (self.List) - 1
while i >= 0 :
yield self.List [i]
i -= 1
def __KeyTransformed__ (self, p_Key) :
if self.Reverse :
if p_Key < 0 :
return (-1) - p_Key
else :
return (len (self.List) - 1) - p_Key
else :
return p_Key
def __Bound__ (self, p_Given, p_Default) :
if p_Given is None :
return p_Default
elif p_Given < 0 :
return len (self.List) + p_Given
else :
return p_Given
def __SliceFixed__ (self, p_Slice) :
l_Step = p_Slice.step or 1
if l_Step == 0 :
raise IndexError, "Step must be non-zero"
elif l_Step > 0 :
l_Start = self.__Bound__ (p_Slice.start, 0)
l_Stop = self.__Bound__ (p_Slice.stop, len (self.List))
else :
l_Start = self.__Bound__ (p_Slice.start, len (self.List) - 1)
l_Stop = self.__Bound__ (p_Slice.stop, -1)
l_Count = (((l_Stop - 1) - l_Start) // l_Step) + 1
l_Stop = l_Start + l_Count * l_Step
return slice(l_Start,l_Stop,l_Step)
def __SliceTransformed__ (self, p_Slice) :
l_Slice = self.__SliceFixed__ (p_Slice)
if self.Reverse :
l_End = len(self.List) - 1
return slice (l_End - l_Slice.start, l_End - l_Slice.stop,
-l_Slice.step)
else :
return p_Slice
# some members are trivial
def __len__ (self) :
return len (self.List)
def __iter__ (self) :
return self.__FullIter__ ()
# some members need a bit more work...
def __str__ (self) :
if self.Reverse :
return str(self.List [::-1])
else :
return str(self.List)
def __repr__ (self) :
if self.Reverse :
return repr(self.List [::-1])
else :
return repr(self.List)
def __getitem__ (self, p_Item) :
if isinstance (p_Item, slice) :
if self.Iter_Slice :
return self.__SliceIter__ (self.__SliceTransformed__ (p_Item))
else :
l_Slice = self.__SliceTransformed__ (p_Item)
if l_Slice.stop == -1 : # annoying special case
return self.List [l_Slice.start::l_Slice.step]
return self.List [l_Slice.start:l_Slice.stop:l_Slice.step]
else :
return self.List [self.__KeyTransformed__ (p_Item)]
def __setitem__ (self, p_Item, p_Value) :
if isinstance (p_Item, slice) :
l_Slice = self.__SliceTransformed__ (p_Item)
if l_Slice.stop == -1 : # annoying special case
self.List [l_Slice.start::l_Slice.step] = p_Value
else :
self.List [l_Slice.start:l_Slice.stop:l_Slice.step] = p_Value
else :
self.List [self.__KeyTransformed__ (p_Item)] = p_Value
def __delitem__ (self, p_Item) :
if isinstance (p_Item, slice) :
l_Slice = self.__SliceTransformed__ (p_Item)
if l_Slice.stop == -1 : # annoying special case
del self.List [l_Slice.start::l_Slice.step]
else :
del self.List [l_Slice.start:l_Slice.stop:l_Slice.step]
else :
del self.List [self.__KeyTransformed__ (p_Item)]
class rilist (list) :
"""
Extend normal list to provide properties giving variants of normal
indexing behaviour - specifically...
indexing slicing returns
-------- ---------------
iforward forward iterator
ibackward backward iterator
backward backward list
"""
def __IFwd__ (self) :
return c_List_Proxy (self, False, True)
def __IBack__ (self) :
return c_List_Proxy (self, True, True)
def __Back__ (self) :
return c_List_Proxy (self, True, False)
iforward = property(__IFwd__ )
ibackward = property(__IBack__)
backward = property(__Back__ )
--
Steve Horne
steve at ninereeds dot fsnet dot co dot uk
Andrew Dalke
> Please comment on the new PEP for reverse iteration methods.
It says:
] Also, it only works with lists (strings, for example, do not define a
reverse method):
]
] rseqn = list(seqn)
] rseqn.reverse()
] for value in rseqn:
] print value
However, the example code will work on a string, because list(string)
returns a list, which does have a reverse.
] Extending slicing minimizes the code overhead but does nothing
] for memory efficiency, beauty, or clarity.
I don't agree with all three of those. Take for example your mhlib
use case
testfolders.reverse();
for t in testfolders:
do('mh.deletefolder(%s)' % `t`)
The PEP suggests that
for t in testfolders.iter_backwards():
do('mh.deletefolder(%s)' % `t`)
is more beautiful and clearer than
for t in testfolders[::-1]:
do('mh.deletefolder(%s)' % `t`)
(It definitely is more memory efficient if there are enough
folders.)
I don't think I agree with that. If I'm not concerned about memory
efficiency then extended slicing is nice because I don't have to
learn nor explain one more thing. ("There should be one ...")
The beauty and clarity, at least to me, only come into play
when memory is a concern, which is only rarely true for most
of the times iter_backwards is expected to be used. (Estimate
based on the use cases and my own judgement.)
] Should file objects be included? Implementing reverse iteration
] may not be easy though it would be useful on occasion.
I would suggest no - the file may not be seekable.
You suggest changing
] while _exithandlers:
] func, targs, kargs = _exithandlers.pop()
to
] for func, target, kargs in _exithandlers.iter_backwards():
] . . .
] del _exithandlers
One thing to bear in mind is that the semantics are slightly
different. In the first one, the finalizers for func, targs, kargs
*may* be called from last to first. This is the case for
CPython if there are no other references to the _exithandlers
objects. But since Python-the-language makes no guarantees
on that and Jython doesn't support it, it's a bad idea to write
code that way.
That is only a very, very minor nit. Feel free to ignore it :)
In difflib, I disagree that
result.sort()
return [x for score, x in result[-n:].iter_backwards()]
is easier to understand than
result.sort() # Retain only the best n.
result = result[-n:] # Move best-scorer to head of list.
result.reverse() # Strip scores.
return [x for score, x in result]
The former is more terse, but the latter, existing code is
broken down into bite-size chunks and allows verbose
documentation of each step. That isn't possible when
nearly everything is written in one line.
Also, a variant implementation may be even more concise,
which is to replace
result.append((s.ratio(), x))
with
result.append((-s.ratio(), x))
then get rid of the reverse (since the minus sign reverses the
sort order) and simply do
result.sort()
return [x for score, x in result[:n]]
] heapq.heapify() uses for i in xrange(n//2 - 1, -1, -1)
The proposed form is
for i in xrange(n//2-1).iter_backwards():
Later in your post you mention a reason against a function
which uses __riter__ magic is that it is "slower than direct
access to the underlying object."
If that's a concern then you should also point out that using
iter_backwards for cases like this is slower than the existing
solution. (It has an extra attr lookup and function call. OTOH,
doesn't the -1, -1 require two calls of the negative unary op
code, or has that been fixed? Good, it has, says dis on
some test code.)
] platform._dist_try_harder() uses for n in range(len(verfiles)-1,-1,-1)
] because the loop deletes selected elements from verfiles but needs
] to leave the rest of the list intact for further iteration. This use case
] could be addressed with itertools.ifilter() but would require the
] selection predicate to be in a lambda expression. The net result is
] less clear and readable than the original. A better reformulation is
] to replace the first line with the proposed method.
Are you sure about that? The code is
verfiles = os.listdir('/usr/lib/setup')
for n in range(len(verfiles)-1, -1, -1):
if verfiles[n][:14] != 'slack-version-':
del verfiles[n]
I think a better reformulation is
verfiles = os.listdir('/usr/lib/setup')
verfiles = [name for name in verfiles
if name.startswith("slack-version-")]
(you could put the os.listdir in the list comp. but I've
found that gets ungainly.)
] random.shuffle() uses for i in xrange(len(x)-1, 0, -1)
This isn't a use case. The xrange never returns a 0 while
the iter_backwards one will.
>>> list(xrange(5, 0, -1))
[5, 4, 3, 2, 1]
>>>
] Rejected Alternative Ideas
There were two intermediate 'reverse' proposals. Here's the four
I know about:
- call the magic method __riter__
- call the magic method __riter__. If it doesn't exist, use
for i in xrange(len(obj)): yield obj[i]
- call the magic method __riter__. If it doesn't exist, use
for i in itertools.count(): yield[obj[-i]]
(this has the advantage of supporting the infinite sequence
0, 1, 2, ... -3, -2, -1
)
- same as the 2nd but if len doesn't exist use
data = list(obj)
data.reverse()
for x in data: yield x
I agree with the conclusion that the last three have unexpected
interactions with mappings. However, iter *also* has unexpected
interactions with mappings.
>>> class Dict:
... def keys(self):
... return [0, 1, 2, 4]
... def __getitem__(self, key):
... if key not in self.keys(): raise KeyError(key)
... return key * 2
... def values(self):
... return [self[x] for x in self.keys()]
... def items(self): return zip(self.keys(), self.values())
... def len(self): return len(self.keys())
...
>>> for x in Dict():
... print x
...
0
2
4
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 5, in __getitem__
KeyError: 3
>>>
While I think I was the one who originally raised this
objection, given the way Python already behaves wrt
mappings, I retract my objection.
Another possibility, which I just thought of now, is
- call the magic method __riter__. If it doesn't exist, look
for the reverse-iterator adapter, as per PEP-0246.
I don't have any experience with the PEP to judge its
appropriateness nor usefulness. The motivation section
seems to suggest that it would be appropriate.
In your post list a set of problems with the function approaches.
> It saves implementing some object methods at the
> expense of adding a new builtin function and of
> creating a new magic method name.
> It is slower than direct access to the underlying object.
> It crashes when applied to an infinite iterator.
> It produces bizarre results when applied to mappings.
I would like to see this list added to the PEP. I do have'
some qualifiers for the list.
- it doesn't need to be a global function; it could be in,
say, itertools. (I know I originally suggested it as a builtin,
but I'm not saying I'm right ;)
- (see comments above that xrange(5).iter_backwards()
is likely slower than xrange(5, -1, -1) because of the
extra lookup and call)
- the first variant, which only looks for __riter__ and raises
an exception if it doesn't exist, has exactly the same problems
with infinite iterators as calling the new method.
- similarly, it doesn't have additional problems with mappings.
Also, none of the use cases dealt with infinite sequences so
the lack of appropriate support for them shouldn't be considered
an outright rejection.
BTW, my current belief is that if this PEP leads to code
then it should be:
- a new function, either builtin or in itertools
- which uses __riter__ if present (unless we decide to
use the adapt PEP)
- else uses "for i in itertools.count(): yield[obj[-i]]"
Andrew
da...@dalkescientific.com
Andrew Dalke
> - else uses "for i in itertools.count(): yield[obj[-i]]"
should be
- else uses "for i in itertools.count(1): yield[obj[-i]]"
Andrew
da...@dalkescientific.com
sebastien
> * Add a builtin function, reverse() which calls a magic method, __riter__.
> I see this as more overhead for no additional benefit.
I believe that this is the better choice because it is the way forward
iteration work in python: you have the iter() function which call a
__iter__ special method.
So, I'd expect to have a riter() function which would call the
__riter__ special method.
I like riter as name to all other alternative because (iter, riter)
looks really like (find, rfind) or (index, rindex) ....
And I still think you don't need it often enough to put it in the
builtin namespace, so the function should go in the itertools module.
Steve Holden
> Please comment on the new PEP for reverse iteration methods.
Looks good. But
[...]
> It crashes when applied to an infinite iterator.
[...]
Hmm. My little brain is having difficulty imagining anything that won't.
What am I missing?
regards
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/
Raymond Hettinger
> So, I'd expect to have a riter() function which would call the
> __riter__ special method.
> And I still think you don't need it often enough to put it in the
> builtin namespace, so the function should go in the itertools module.
If you make a magic method, then the function calling it
needs to be a builtin. They go hand in hand.
Raymond
Raymond Hettinger
> The question is - is the exercise worthwhile. I'm not sure even though
> it's my own idea. I think it is nice in its way, and a lot more
> flexible than the 'iter_backward' method proposal, but that is quite
> likely a bad thing as it is probably massive overkill for the problem
> being solved - and in particular the extra flexibility has costs.
>
> What the hell, though - it was fun to play with for a bit!
Yes, it's a one ton solution to a one ounce problem,
but it was an interesting exercise and diversion.
Raymond Hettinger
Raymond Hettinger
> So, I'd expect to have a riter() function which would call the
> __riter__ special method.
Okay, changed pep to list riter() as an alternative.
> And I still think you don't need it often enough to put it in the
> builtin namespace, so the function should go in the itertools module.
If you have a magic method, __riter__, then the corresponding
function needs to be a builtin. They go hand in hand. The
core parts of the language need to be usable without having
to know about itertools.
Raymond
Raymond Hettinger
> However, the example code will work on a string, because list(string)
> returns a list, which does have a reverse.
Right. I'll remove the comment.
> ] Extending slicing minimizes the code overhead but does nothing
> ] for memory efficiency, beauty, or clarity.
> I don't agree with all three of those.
Beauty and clarity are a matter of taste and we differ here.
To some (including me), s[::-1] looks more like line noise
than s.iter_backwards(). Also, I think the word form is
more clear. The situation becomes more acute when the
other indices have values. Compare range[n-1, 0, -1] to
range(1, n-1).iterbackwards(). Whenever negative indicies
are used, it takes more knowledge of the implementation
rules to be able to read,write, understand, and verify the code.
> ] Should file objects be included? Implementing reverse iteration
> ] may not be easy though it would be useful on occasion.
>
> I would suggest no - the file may not be seekable.
Agreed.
> One thing to bear in mind is that the semantics are slightly
> different.
. . .
> That is only a very, very minor nit. Feel free to ignore it :)
Agreed. If finalizer call order is important, then the
'while elist: elist.pop()' form is the way to go. If not,
then the for-loop is the way to go.
> The former is more terse, but the latter, existing code is
> broken down into bite-size chunks and allows verbose
> documentation of each step. That isn't possible when
> nearly everything is written in one line.
Even the former can be broken down into more lines if
that is what is needed:
result.sort()
result = result[-n]
return [x for score, x in result.iter_backwards()]
> Also, a variant implementation may be even more concise,
You must be pretty confident to take on the Timbot's code ;-)
> Later in your post you mention a reason against a function
> which uses __riter__ magic is that it is "slower than direct
> access to the underlying object."
>
> If that's a concern then you should also point out that using
> iter_backwards for cases like this is slower than the existing
> solution. (It has an extra attr lookup and function call. OTOH,
> doesn't the -1, -1 require two calls of the negative unary op
> code, or has that been fixed? Good, it has, says dis on
> some test code.)
Yes, it's true. All of the speed comments focus on the inner loop.
I'll reword to make that clear. It is definitely an issue. In Py2.2,
there was a generic sequence iterator that used __getitem__ in
iterators for lists and tuples. In Py2.3, we went to a custom
iterator for each object. There was a significant improvement
in performance.
> ] platform._dist_try_harder()
. . .
> I think a better reformulation is
>
> verfiles = os.listdir('/usr/lib/setup')
> verfiles = [name for name in verfiles
> if name.startswith("slack-version-")]
I like your version better and recommend you submit it as a patch.
I'll take out the ifilter() comment.
> ] random.shuffle() uses for i in xrange(len(x)-1, 0, -1)
> This isn't a use case. The xrange never returns a 0 while
> the iter_backwards one will.
It is an important use case. The replacement code is:
for i in xrange(1,len(x)). iter_backwards()
Whenever the indices have any complexity, the forwards version,
followed by .iter_backwards() is, IMHO, much easier to mentally verify.
> There were two intermediate 'reverse' proposals. Here's the four
> I know about:
I'll try to add these without making the PEP as long as this thread ;-)
> I agree with the conclusion that the last three have unexpected
> interactions with mappings. However, iter *also* has unexpected
> interactions with mappings.
Yes, but that doesn't make it less of a disadvantage.
The proposed list.iter_backwards() method does not have that
disadvantage.
> In your post list a set of problems with the function approaches.
...
> I would like to see this list added to the PEP.
Okay. They are added.
> I do have'
> some qualifiers for the list.
>
> - it doesn't need to be a global function; it could be in,
> say, itertools. (I know I originally suggested it as a builtin,
> but I'm not saying I'm right ;)
If there is to be a magic method, __riter__, then the access
function needs to be a builtin. They go hand in hand.
BTW, don't underestimate the intensity of resistance to adding
new builtins.
> - (see comments above that xrange(5).iter_backwards()
> is likely slower than xrange(5, -1, -1) because of the
> extra lookup and call)
Reworded it to talk only to the inner loop, "approaches using
__getitem__() are slower using a custom method for
each object."
> - the first variant, which only looks for __riter__ and raises
> an exception if it doesn't exist, has exactly the same problems
> with infinite iterators as calling the new method.
>
> - similarly, it doesn't have additional problems with mappings.
Okay, I've moved the comments around for you.
> Also, none of the use cases dealt with infinite sequences so
> the lack of appropriate support for them shouldn't be considered
> an outright rejection.
The use cases were just the ones I found in the library.
With generators and itertools being new, I expect
infinite iterators to become more common.
Also, I have a preference for creating something that
is as robust as possible. Making a builtin function that
doesn't apply to all objects; behaves badly with mappings;
and crashes with an infinite iterator is not my idea of robust.
I really don't understand the overpowering urge to make this a function
and add a new magic method protocol. Why isn't there a similar rage
against dict.iteritems()?
Perhaps, you'll like it better if the method is named ireverse()
instead of iter_backwards().
> BTW, my current belief is that if this PEP leads to code
> then it should be:
> - a new function, either builtin or in itertools
>
> - which uses __riter__ if present (unless we decide to
> use the adapt PEP)
>
> - else uses "for i in itertools.count(): yield[obj[-i]]"
Okay, we simply disagree here. That's fine.
I appreciate the thoughtful comments and careful analysis.
They have helped clarify the pep wording and helped to
define the issues.
Raymond
Tom Anderson
> "Raymond Hettinger" <vze4...@verizon.net> wrote ...
> > Please comment on the new PEP for reverse iteration methods.
>
> Looks good. But
>
> [...]
> > It crashes when applied to an infinite iterator.
> [...]
> Hmm. My little brain is having difficulty imagining anything that won't.
> What am I missing?
ideally, riter(riter(someInfiniteIterator)) would work, though!
tom
--
I see large and small words on this page, arranged in long rows separated by little dotty characters. Suspect written by little dotty characters, too. -- RonJeffries
Andrew Dalke
> Beauty and clarity are a matter of taste and we differ here.
> To some (including me), s[::-1] looks more like line noise
> than s.iter_backwards().
Understood. In all this, let me make it clearthat I
have a preference for a function over a method, but won't
call it a wart of the BFDL thinks otherwise. I point these
out because I'm rather detail oriented.
> > Also, a variant implementation may be even more concise,
>
> You must be pretty confident to take on the Timbot's code ;-)
Well, I did say "may". I wouldn't really want to use it since
inverting the score before the sort is a tricky thing to see and
somewhat unexpected.
(I'm half expecting some enlightening comment about how
negation has subtle interactions with IEEE 754 math -- though
I thought the point of 754 was to make naive approaches like
mine usually correct. :)
> > verfiles = os.listdir('/usr/lib/setup')
> > verfiles = [name for name in verfiles
> > if name.startswith("slack-version-")]
>
> I like your version better and recommend you submit it as a patch.
> I'll take out the ifilter() comment.
But that goes against the pretty good motto of "if it ain't broke,
don't touch it." Granted, XP-style tests are supposed to
help out with that, but I don't have a /usr/lib/setup where I
can test this.
> > ] random.shuffle() uses for i in xrange(len(x)-1, 0, -1)
> > This isn't a use case. The xrange never returns a 0 while
> > the iter_backwards one will.
>
> It is an important use case. The replacement code is:
>
> for i in xrange(1,len(x)). iter_backwards()
Ahh, right. Didn't think about that. You should include that
wording in the PEP.
> Whenever the indices have any complexity, the forwards version,
> followed by .iter_backwards() is, IMHO, much easier to mentally verify.
I agree. When I do need to go backwards I often forgot to get
both -1s in place. Ie, I'll do (n, 0) instead of (n-1, -1).
Well, don't forgot much and more, but there's some tension in
my mind as I worry if I got it correct or not.
> BTW, don't underestimate the intensity of resistance to adding
> new builtins.
For me it's two things: remembering what all these new things
do, and attempting to adhere to the 'don't use variables with
the same name as builtins' guiding philosophy. I don't adhere
to the latter wrt the type named 'file'.
> Also, I have a preference for creating something that
> is as robust as possible. Making a builtin function that
> doesn't apply to all objects; behaves badly with mappings;
> and crashes with an infinite iterator is not my idea of robust.
The backup which uses __getitem__ and negative offsets
does work for all objects, behaves exactly as badly as the
existing iter, and doesn't crash with an infinite iterator, no?
It's slow, but if you want the performance, make an __riter__.
> I really don't understand the overpowering urge to make this a function
> and add a new magic method protocol. Why isn't there a similar rage
> against dict.iteritems()?
Overpowering?
My preference is for a function because of:
- similaraties to iter, which is also a function
- there are many types of iteration orderings, and not all can be
special cased methods.
- I don't see backwards iteration as all that important
As for iteritems, I was annoyed about it because it meant my
dictionary-like code would become more complicated to
implement, until I came across UserDict.DictMixin, which
lets user-defined code implement just a few methods then
builds the rest from there.
That suggests a UserList.ListMixin might be useful, but it
isn't obvious to me that it would be.
I also note that items is a method so by similarity it makes
sense that iteritems also be a method. Additionally, many
things offer forward iteration, while only dicts offer items.
Should dicts support riteritems, ritervalues?
> Okay, we simply disagree here. That's fine.
Yup. I don't think I have anything useful or even interesting
to offer to this dicusssion. I also think that with these changes,
the PEP is pretty complete and solid.
Andrew
da...@dalkescientific.com
Stephen Horne
<vze4...@verizon.net> wrote:
>Please comment on the new PEP for reverse iteration methods.
One more thought.
Having an iter_backwards method on xrange seems wrong. This suggests
multiple functions/methods...
To generate backward ranges
range_backwards function
xrange_backwards funtion
for i in range_backwards (10) :
...
To iterate an object backwards
either method for appropriate objects, or
iter_backward() function and __iter_backward__ magic method
(I don't mind abbreviating the 'iter', but I think the 'backward'
needs to be kept very obvious and not reduced to an easily missed
'r' prefix).
for i in iter_backwards (seq) :
...
To enumate backwards etc
Add an enumerate_backward function which requires the input to
support __len__.
for i in enumerate_backwards (seq) :
...
For backward slicing
Add a function which translates the slice to a backward equivalent
before applying it...
for i in slice_backwards (seq, start, stop, step) :
...
sebastien
> If you have a magic method, __riter__, then the corresponding
> function needs to be a builtin. They go hand in hand. The
> core parts of the language need to be usable without having
> to know about itertools.
I don't think so:
We already have this situation for the the copy module:
copy.copy() use the __copy__ magic method and copy.deepcopy() use
__deepcopy__
I've never heard about problems with this behaviour.
In fact it's not the __riter__ magic method which will use riter()
function but the opposite, so an object which define __riter__ doesn't
need to now anything about riter().
Alex Martelli
>> And I still think you don't need it often enough to put it in the
>> builtin namespace, so the function should go in the itertools module.
>
> If you have a magic method, __riter__, then the corresponding
> function needs to be a builtin. They go hand in hand. The
> core parts of the language need to be usable without having
> to know about itertools.
I have already seen module copy presented as a counter-example to
your assertion, and I'd like to add module pickle as a second, and
I hope decisive, counter-example. It is, to put it simply, utterly
false that functions corresponding to magic methods "need to be
builtins": it's *perfectly* all right for such functions to live
in standard library modules. Oh, and let's not forget module sets:
the __as_immutable__ and __as_temporarily_immutable__ magic methods,
that are used when making a set of sets, or checking for a set's
membership in another set, can be seen as yet another example (and
in this case there is no wiggling out of it by claiming that the
magicmethod/NON-builtin correspondence is a historical/legacy thing,
as the BDFL approved module sets.py, with just this usage, so very
recently).
I second the motion that function riter, with check for __riter__
and all, should live in module itertools. Reverse iteration is a
RARE need -- far rarer than copying or even pickling -- and there
is no real reason to burden __builtins__ with a function for it.
Alex
John Roth
isn't a significant hardship.
John Roth
"Alex Martelli" <al...@aleax.it> wrote in message
news:8RScb.169633$R32.5...@news2.tin.it...
David Abrahams
Is there any concern about the fact that a sequence of direction
changes requires building a new iterator object each time under this
proposal? I was going to implement my own bidirectional iteration
protocol by using a prev() method and use an adaptor object to swap
the meaning of next() and prev(). Python iterators (and GoF
iterators) seem poorly suited as pure sequence position indicators
because you can't reposition them without also accessing elements, so
maybe this is not such a serious issue... but my instincts tell me
that the identity of a bidirectional iterator object could be useful
if we allow the same one to be used in both directions.
FWIW, burdening builtins aside I consider the proposed syntax
for i in xrange(n).iter_backwards():
much uglier than
for i in reverse_view(xrange(n)):
but far superior to
for i in reverse(xrange(n)):
which implies in-place modification of the sequence.
Also, the idea of denying tuples the ability to reverse iterate seems
arbitrary and capricious.
Cheers,
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
Sean Ross
news:uk77vh...@boost-consulting.com...
[snip]
> FWIW, burdening builtins aside I consider the proposed syntax
>
> for i in xrange(n).iter_backwards():
>
> much uglier than
>
> for i in reverse_view(xrange(n)):
>
> but far superior to
>
> for i in reverse(xrange(n)):
>
> which implies in-place modification of the sequence.
How about
from itertools import ireverse
for i in ireverse(xrange(n)):
# suite
ireverse(), like imap(), izip(), etc., suggests that the operation is
iterative, and that no modification of the original sequence will be
performed. Others have suggested riter() (right iteration), in order to form
an association with the iter() builtin. As a matter of taste, I prefer
ireverse().
Sean
David Eppstein
"Sean Ross" <sr...@connectmail.carleton.ca> wrote:
> How about
>
> from itertools import ireverse
> for i in ireverse(xrange(n)):
> # suite
>
> ireverse(), like imap(), izip(), etc., suggests that the operation is
> iterative, and that no modification of the original sequence will be
> performed. Others have suggested riter() (right iteration), in order to form
> an association with the iter() builtin. As a matter of taste, I prefer
> ireverse().
ireverse, like imap(), izip(), etc., suggests that the operation happens
without the memory overhead of copying the whole sequence into a list
before reversing it. Do you have some plan for how to do that e.g. with
simple generators? Or easy to understand explanation for which things
can be ireversed and which can't?
--
David Eppstein http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science
Lulu of the Lotus-Eaters
|ireverse, like imap(), izip(), etc., suggests that the operation happens
|without the memory overhead of copying the whole sequence into a list
|before reversing it. Do you have some plan for how to do that e.g. with
|simple generators? Or easy to understand explanation for which things
|can be ireversed and which can't?
Or even just a DECIDABLE explanation for which things can be 'ireversed'
and which can't[*].
Yours, Lulu...
[*] And if you -can- produce such an explanation, I can almost guarantee
you a Fields Medal, and probably a Nobel Prize also.
--
mertz@ _/_/_/_/_/_/_/ THIS MESSAGE WAS BROUGHT TO YOU BY:_/_/_/_/ v i
gnosis _/_/ Postmodern Enterprises _/_/ s r
.cx _/_/ MAKERS OF CHAOS.... _/_/ i u
_/_/_/_/_/ LOOK FOR IT IN A NEIGHBORHOOD NEAR YOU_/_/_/_/_/ g s
Lulu of the Lotus-Eaters
|without the memory overhead of copying the whole sequence into a list
|before reversing it. Do you have some plan for how to do that e.g. with
|simple generators? Or easy to understand explanation for which things
|can be ireversed and which can't?
Or even just a DECIDABLE explanation for which things can be 'ireversed'
Alex Martelli
...
> Also, the idea of denying tuples the ability to reverse iterate seems
> arbitrary and capricious.
Sure, but so is denying them, e.g., non-mutating methods such as .index()
and .count(). At least we're _consistently_ arbitrary and capricious!-)
Alex
Sean Ross
news:eppstein-B529F3...@news.service.uci.edu...
[snip]
> ireverse, like imap(), izip(), etc., suggests that the operation happens
> without the memory overhead of copying the whole sequence into a list
> before reversing it. Do you have some plan for how to do that e.g. with
> simple generators? Or easy to understand explanation for which things
> can be ireversed and which can't?
>
Nope. Nor have I intimated I had such, nor that any would be forthcoming.
This is a naming suggestion. I am neither the originator of the proposal nor
a proponent of it, so I see no reason to formulate such a plan nor to give
an explanation of how it should be accomplished. I saw a name, and offered
another. That is all.
P1: "If I make this thing, I propose to call it 'ShomozzleBoB'."
P2: "Really? How about just BoB?"
P3: "Do you have a plan for how to do BoB?"
P2: "Huh?"
David Eppstein
"Sean Ross" <sr...@connectmail.carleton.ca> wrote:
> Nope. Nor have I intimated I had such, nor that any would be forthcoming.
> This is a naming suggestion. I am neither the originator of the proposal nor
> a proponent of it, so I see no reason to formulate such a plan nor to give
> an explanation of how it should be accomplished. I saw a name, and offered
> another. That is all.
>
> P1: "If I make this thing, I propose to call it 'ShomozzleBoB'."
> P2: "Really? How about just BoB?"
> P3: "Do you have a plan for how to do BoB?"
> P2: "Huh?"
Your proposed name has an implication for the semantics. I was
questioning the feasibility of the implied semantics. I don't think
you're let off the hook by the excuse that "it's only a name".
Stephen Horne
<epps...@ics.uci.edu> wrote:
>In article <WNYcb.12686$yD1.1...@news20.bellglobal.com>,
> "Sean Ross" <sr...@connectmail.carleton.ca> wrote:
>
>> How about
>>
>> from itertools import ireverse
>> for i in ireverse(xrange(n)):
>> # suite
>>
>> ireverse(), like imap(), izip(), etc., suggests that the operation is
>> iterative, and that no modification of the original sequence will be
>> performed. Others have suggested riter() (right iteration), in order to form
>> an association with the iter() builtin. As a matter of taste, I prefer
>> ireverse().
>
>ireverse, like imap(), izip(), etc., suggests that the operation happens
>without the memory overhead of copying the whole sequence into a list
>before reversing it. Do you have some plan for how to do that e.g. with
>simple generators? Or easy to understand explanation for which things
>can be ireversed and which can't?
How about this - only support ireverse for sequences (identified using
len, and perhaps with a check for the keys method in case of mapping
types) - not for iterators at all.
Add an xrange_backward function to complement xrange rather than
trying to modify the result of xrange, and where an object can
logically support reverse iteration have a method or property that
returns an alternative generator/iterator.
At its simplest, something like...
class newlist (list) :
def ibackward (self) :
i = len(self)
while i > 0 :
i -= 1
yield i
...is no great burden for most built-in sequence types, and would
define a protocol for other types to follow where appropriate. It
doesn't need a big memory overhead.
Rather than define a separate xrange_backwards, it is worth noticing
that the built-in 'functions' xrange and enumerate actually seem to be
classes. We could add factory methods, set up to effectively give
alternate constructors, to allow syntax such as...
for i in xrange.backward (10) :
...
for i, j in enumerate.backward (seq) :
David Abrahams
> David Abrahams wrote:
> ...
>> Also, the idea of denying tuples the ability to reverse iterate seems
>> arbitrary and capricious.
>
> Sure, but so is denying them, e.g., non-mutating methods such as
> .index() and .count().
Not IMO. Immutability is a very useful trait.
> At least we're _consistently_ arbitrary and capricious!-)
Not in this case.
Sean Ross
> Your proposed name has an implication for the semantics.
Do the implied semantics of ireverse() differ from those implied by the
names iter_backwards(), reverse_view(), riter(), etc.?
Because, to me, they' re just different words for the same idea.
> I was questioning the feasibility of the implied semantics.
Fine. Take it up with the person who proposed the idea.
> I don't think you're let off the hook by the excuse that "it's only a
name".
I'm let off the "hook" because I was never on it. The responsibility for
answering your questions re: the feasability of this enterprise do not fall
to me, but to the proponent of the enterprise. It's like someone said they
where going to build a bridge from the mainland to Hawaii, and that they
would paint it green. And I said, "What about painting it red?" And you
asked me, "And how to you propose to build this bridge?". My answer is, "Uh,
why don't you ask the fellow over there who brought up the thing in the
first place. I'm just talking paint colours over here." I think your
questions are mis-directed. And, even if they are not, I don't have your
answers, I'm not going to try to come up with them, and I feel no
responsibility to do so.
Sean Ross
implementation. My interest is in the syntax. If this feature can be added
to the language (and I don't care whether it can or cannot be), then I would
prefer to read code like
for i in ireverse(xrange(n)):
than
for i in xrange(n).iter_backwards(n):
And there ends my superficial concerns in this matter. If my proposed syntax
implies more than the other syntax proposals, then I withdraw the
suggestion. I've no interest in arguing about whether those, or any other,
implications can be met. I'm talking sugar. You're talking substance. And I
don't care. If it can be done, let it be done, and give it pretty name. If
it can't be done, then it doesn't much matter what name you give it.
I hope were done here,
Sean
Alex Martelli
> Alex Martelli <al...@aleax.it> writes:
>
>> David Abrahams wrote:
>> ...
>>> Also, the idea of denying tuples the ability to reverse iterate seems
>>> arbitrary and capricious.
>>
>> Sure, but so is denying them, e.g., non-mutating methods such as
>> .index() and .count().
>
> Not IMO. Immutability is a very useful trait.
Sure, very useful indeed -- and why does YO suggest that add such
NON-mutating methods would damage immutability in the LEAST...?
>> At least we're _consistently_ arbitrary and capricious!-)
>
> Not in this case.
Some amplification would be welcome, because the above comment
on immutability's usefulness is totally obscure to me in context.
Alex
Stephen Horne
<sr...@connectmail.carleton.ca> wrote:
>
>"David Eppstein" <epps...@ics.uci.edu> wrote in message
>news:eppstein-AF4213...@news.service.uci.edu...
>> Your proposed name has an implication for the semantics.
>
>Do the implied semantics of ireverse() differ from those implied by the
>names iter_backwards(), reverse_view(), riter(), etc.?
>Because, to me, they' re just different words for the same idea.
So far as I can see, it is having the function rather than the method
that creates the issues - that originated with David Abrahams post (at
least tracing back through this thread - it had been suggested
before).
However, your reply was implicitly supporting the use of a function
rather than a method. If 'ireverse' is imported from 'itertools' then
it cannot be a method of the object being reversed.
I think some wires have been slightly crossed - David Eppstein raised
a good point, but should perhaps have aimed it at David Abrahams.
Stephen Horne
<da...@boost-consulting.com> wrote:
>Alex Martelli <al...@aleax.it> writes:
>
>> David Abrahams wrote:
>> ...
>>> Also, the idea of denying tuples the ability to reverse iterate seems
>>> arbitrary and capricious.
>>
>> Sure, but so is denying them, e.g., non-mutating methods such as
>> .index() and .count().
>
>Not IMO. Immutability is a very useful trait.
Yes - and perfectly consistent with having *NON*-mutating methods such
as .index() and .count() ;-)
I always assumed that these were considered inconsistent with normal
use of tuples (which certainly I rarely need to get 'index' or
'count'-like results from).
Actually, they even seem a little odd in list, to be honest. I'd have
functions, not necessarily even in __builtins__, which work on any
sequence. They just don't seem like everyday operations that should be
built into the object.
Chad Netzer
> > I was questioning the feasibility of the implied semantics.
>
> Fine. Take it up with the person who proposed the idea.
You proposed it (hear me out). You are missing David's point. In
addition to whatever semantics a general reverse iterator might have, he
is saying that the itertools iterators have an implied additional
semantic of not (necessarily) needing to fully expand an iterable in
memory to operate on it, as a general ireverse() would probably have to
do. So, having ireverse potentially use up VAST quantities of memory,
in order to work, doesn't quite fit into the itertools philosophy.
That is the point he is making, and it is a specific comment on your (I
believe; others have probably proposed it as well) suggestion of an
ireverse() in itertools.
Now, my response to David's point is that currently, cycle() may also
require enough extra storage to remember an entire iterated sequence, so
the itertools philosophy is not a hard rule. Still, ireverse() would
imply some real devilry under the covers...
--
Chad Netzer
Sean Ross
news:mailman.106460451...@python.org...
Hi.
Thanks for clearing that up.
Yes, "others have ... proposed it as well". I was merely interested in the
appearance of the code, not the underlying implementation. I was not aware
that the name would imply more than the other suggestions simply because it
was housed in itertools. But then that's not really the point, I suppose:
The main point is the "implied additional semantic", regardless of the
function's location. Yes? OK. Fine. I was not concerned with semantics when
I made the suggestion, only sugar, so I couldn't see where David was coming
from. Sorry for the confusion.
Sean
David Abrahams
> David Abrahams wrote:
>
>> Alex Martelli <al...@aleax.it> writes:
>>
>>> David Abrahams wrote:
>>> ...
>>>> Also, the idea of denying tuples the ability to reverse iterate seems
>>>> arbitrary and capricious.
>>>
>>> Sure, but so is denying them, e.g., non-mutating methods such as
>>> .index() and .count().
>>
>> Not IMO. Immutability is a very useful trait.
>
> Sure, very useful indeed -- and why does YO suggest that add such
> NON-mutating methods would damage immutability in the LEAST...?
Sorry, I misread your reply. I didn't know those were missing.
>>> At least we're _consistently_ arbitrary and capricious!-)
>>
>> Not in this case.
>
> Some amplification would be welcome, because the above comment
> on immutability's usefulness is totally obscure to me in context.
My mistake; you were right as usual.
David Abrahams
> On Fri, 26 Sep 2003 14:51:05 -0400, David Abrahams
> <da...@boost-consulting.com> wrote:
>
>>Alex Martelli <al...@aleax.it> writes:
>>
>>> David Abrahams wrote:
>>> ...
>>>> Also, the idea of denying tuples the ability to reverse iterate seems
>>>> arbitrary and capricious.
>>>
>>> Sure, but so is denying them, e.g., non-mutating methods such as
>>> .index() and .count().
>>
>>Not IMO. Immutability is a very useful trait.
>
> Yes - and perfectly consistent with having *NON*-mutating methods such
> as .index() and .count() ;-)
Right. Oops, I misread.
> I always assumed that these were considered inconsistent with normal
> use of tuples (which certainly I rarely need to get 'index' or
> 'count'-like results from).
I don't see why not.
> Actually, they even seem a little odd in list, to be honest. I'd have
> functions, not necessarily even in __builtins__, which work on any
> sequence. They just don't seem like everyday operations that should be
> built into the object.
It's probably just for optimization purposes.
David Abrahams
> "David Eppstein" <epps...@ics.uci.edu> wrote in message
> news:eppstein-AF4213...@news.service.uci.edu...
>> Your proposed name has an implication for the semantics.
>
> Do the implied semantics of ireverse() differ from those implied by the
> names iter_backwards(), reverse_view(), riter(), etc.?
> Because, to me, they' re just different words for the same idea.
Well, the only other place we have an 'i' prefix on a function name,
AFAIK, it means "inplace". I don't mind riter(); it has a nice
resonance with rbegin() in C++.
Raymond Hettinger
> > It crashes when applied to an infinite iterator.
[Steve Holden]
> Hmm. My little brain is having difficulty imagining anything that won't.
> What am I missing?
The PEP proposes adding a method only to a few objects
that don't have degenerate cases: list, str, unicode, xrange
and possibly tuple.
Raymond Hettinger
Raymond Hettinger
> > > verfiles = os.listdir('/usr/lib/setup')
> > > verfiles = [name for name in verfiles
> > > if name.startswith("slack-version-")]
> >
> > I like your version better and recommend you submit it as a patch.
> > I'll take out the ifilter() comment.
>
> But that goes against the pretty good motto of "if it ain't broke,
> don't touch it."
Between major releases, refactoring is fair game. Using newer
constructs like list comprehensions can make this code more
readable and maintainable.
> Granted, XP-style tests are supposed to
> help out with that, but I don't have a /usr/lib/setup where I
> can test this.
Submit a patch, assign to me, and I'll test it.
> > > > random.shuffle() uses for i in xrange(len(x)-1, 0, -1)
> > > This isn't a use case. The xrange never returns a 0 while
> > > the iter_backwards one will.
> >
> > It is an important use case. The replacement code is:
> >
> > for i in xrange(1,len(x)). iter_backwards()
>
> Ahh, right. Didn't think about that. You should include that
> wording in the PEP.
Okay, done!
> > Whenever the indices have any complexity, the forwards version,
> > followed by .iter_backwards() is, IMHO, much easier to mentally verify.
>
> I agree. When I do need to go backwards I often forgot to get
> both -1s in place. Ie, I'll do (n, 0) instead of (n-1, -1).
> Well, don't forgot much and more, but there's some tension in
> my mind as I worry if I got it correct or not.
That is a key motivation for the pep!
> > Also, I have a preference for creating something that
> > is as robust as possible. Making a builtin function that
> > doesn't apply to all objects; behaves badly with mappings;
> > and crashes with an infinite iterator is not my idea of robust.
>
> The backup which uses __getitem__ and negative offsets
> does work for all objects, behaves exactly as badly as the
> existing iter, and doesn't crash with an infinite iterator, no?
1. Sure, all objects that define __getitem__ in a non-mapping
sense or that don't ignore the index (that was at one time a
common python practice).
2. With infinite iterators, running them to completion causes
the crash. They can be used in other generators, iterators,
or loops that have a break. This is not true with reverse
iterators where the very first call will cause the crash:
itertools.count().next() # this always works
riter(itertools.count()).next() # this always fails
> That suggests a UserList.ListMixin might be useful, but it
> isn't obvious to me that it would be.
You're the fourth person to suggest it.
If you want to know the issues involved, see my post at:
http://groups.google.com/groups?selm=Bebab.9323%24hl4...@nwrdny01.gnilink.net
> Should dicts support riteritems, ritervalues?
Heck no! I only want reverse iteration for lists, strings, and xrange
where forward and reverse orders make some sense.
> I also think that with these changes,
> the PEP is pretty complete and solid.
Thanks for the detailed help getting it to that point.
Raymond
David Eppstein
"Raymond Hettinger" <vze4...@verizon.net> wrote:
Speaking of which: str, int, etc., used to be functions, but they got
changed to be the initializers of types having those names. Why is
xrange still a function rather than the initializer of the xrange type?
The most visible difference of changing xrange to be the type of the
xrange objects would be more information shown in help(xrange), some of
which could be useful if we are talking about adding more methods to the
xrange objects.
Stephen Horne
<epps...@ics.uci.edu> wrote:
>Speaking of which: str, int, etc., used to be functions, but they got
>changed to be the initializers of types having those names. Why is
>xrange still a function rather than the initializer of the xrange type?
"""
Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> xrange
<type 'xrange'>
"""
Looks like a type rather than a function to me, and the help appears
to be very good. Enumerate also seems to be a type with an
initialiser.
Are you using an earlier version?
Alex Martelli
...
>> I always assumed that these were considered inconsistent with normal
>> use of tuples (which certainly I rarely need to get 'index' or
>> 'count'-like results from).
>
> I don't see why not.
The party line (not saying I necessarily agree, mind you -- just relating
it) is that tuples are meant as semantically heterogeneous containers
(semantically in that the items may happen to be the same type, but
their _meaning_ is disparate: e.g., consider the tuples used in modules
time or stat). So it doesn't really make sense to iterate on them, either;
it just happens to be possible because their items are accessed with []
with progressive naturals, and they have a len(). Associating _names_
to the indices makes more sense and indeed such pseudo-tuples are
starting to be supplied by the standard library (though not a general
mechanism for making your own, yet). Much like you can't iterate in
C++ on a std::pair<>, thus you "shouldn't" be able to iterate on a tuple.
"Frozen" versions of lists and dicts might make more sense -- and lose
only the MUTATING methods. I've seen Ruby experts claim that the
idea of freezing an object looks cool but in practice you end up almost
never using it -- they're speaking from experience, I guess, since in Ruby
they've had the possibility of freezing any object for ages. However, my
intuition (not supported by that much specific experience) makes me
yearn for such a feature...
Alex
David Abrahams
> David Abrahams wrote:
> ...
>>> I always assumed that these were considered inconsistent with normal
>>> use of tuples (which certainly I rarely need to get 'index' or
>>> 'count'-like results from).
>>
>> I don't see why not.
>
> The party line (not saying I necessarily agree, mind you -- just relating
> it) is that tuples are meant as semantically heterogeneous containers
> (semantically in that the items may happen to be the same type, but
> their _meaning_ is disparate: e.g., consider the tuples used in modules
> time or stat). So it doesn't really make sense to iterate on them, either;
> it just happens to be possible because their items are accessed with []
> with progressive naturals, and they have a len(). Associating _names_
> to the indices makes more sense and indeed such pseudo-tuples are
> starting to be supplied by the standard library (though not a general
> mechanism for making your own, yet). Much like you can't iterate in
> C++ on a std::pair<>, thus you "shouldn't" be able to iterate on a
> tuple.
Well, (understanding that you don't nececessarily agree with the
above) you can in fact iterate on std::pair<T,T> with the usual C++
iterator protocol, and with a new mixed compile-time/runtime tuple
iterator protocol developed by Doug Gregor, iteration over
heterogeneous tuples is possible too. It certainly is desirable to be
able to do that; it's a real need that has come up in practice.
Alex Martelli
...
> Well, (understanding that you don't nececessarily agree with the
> above) you can in fact iterate on std::pair<T,T> with the usual C++
> iterator protocol,
You mean there's an std::pair::begin etc?! OK, I guess I'm even
rustier on standard C++ than I thought I was -- I could have SWORN
there wasn't. (Std chapter & verse pls? I plan to win bets based
on this tidbit...!-). So I guess the lack of those in gcc is a
breach of the Standard on gcc's part...?
> and with a new mixed compile-time/runtime tuple
> iterator protocol developed by Doug Gregor, iteration over
> heterogeneous tuples is possible too. It certainly is desirable to be
> able to do that; it's a real need that has come up in practice.
So, given an arbitrary struct, with fields (generally) of different types,
you can iterate field by field? That's basically what the "heterogeneous
tuple" is supposed to be equivalent to, in the "party line" I was relating
(although the names to go with the fields are only present in a FEW such
tuples, such as those returned by modules time and stat, as of now; more
general tuples still haven't grown the ability to access fields by name,
in Python).
My partial dissent comes from feeling the need for "frozen/immutable
lists" and the fact that tuples are often used as such (including goofy
immutable representations of dicts, e.g. via tuple(thedict.iteritems()),
and the like). If tuples cannot be thought of as immutable lists, then
(I think) we need "immutable/hashable/frozen" lists by other means.
(I need to say "I think" because, as I quoted, Ruby experts claim that
the "anyobject.freeze" feature they do have isn't actually as useful
as it SEEMS it should be -- though I'm not sure I understand why, yet).
Alex
Stephen Horne
wrote:
>David Abrahams wrote:
> ...
>> Well, (understanding that you don't nececessarily agree with the
>> above) you can in fact iterate on std::pair<T,T> with the usual C++
>> iterator protocol,
>
>You mean there's an std::pair::begin etc?! OK, I guess I'm even
>rustier on standard C++ than I thought I was -- I could have SWORN
>there wasn't. (Std chapter & verse pls? I plan to win bets based
>on this tidbit...!-). So I guess the lack of those in gcc is a
>breach of the Standard on gcc's part...?
Somehow I think David is mistaken here - I cannot believe that
dereferencing an iterator returns a different datatype depending on
which item it happens to point to at runtime in statically typed C++,
and without that ability to dereference the iterator (1) I cannot see
the point of iterating through a pair, and (2) the 'iterator' would
not be a true iterator as C++ iterators have to comply with one of a
set of standard protocols (forward, bidirectional, random etc) which
all include subscripting.
Of course you can iterate through a container that holds std::pair
objects - you do that every time you iterate through an std::map - but
that isn't the same thing.
Stephen Horne
<$$$$$$$$$$$$$$$$$@$$$$$$$$$$$$$$$$$$$$.co.uk> wrote:
>set of standard protocols (forward, bidirectional, random etc) which
>all include subscripting.
Oops - I've got subscripting on the mind just now. I meant
dereferencing of course.
David Abrahams
> David Abrahams wrote:
> ...
>> Well, (understanding that you don't nececessarily agree with the
>> above) you can in fact iterate on std::pair<T,T> with the usual C++
>> iterator protocol,
>
> You mean there's an std::pair::begin etc?!
No, that's an entirely different question. I can build an iterator
which iterates over pair<T,T>.
> OK, I guess I'm even rustier on standard C++ than I thought I was --
> I could have SWORN there wasn't. (Std chapter & verse pls? I plan
> to win bets based on this tidbit...!-). So I guess the lack of
> those in gcc is a breach of the Standard on gcc's part...?
No, nothing like that.
>> and with a new mixed compile-time/runtime tuple iterator protocol
>> developed by Doug Gregor, iteration over heterogeneous tuples is
>> possible too. It certainly is desirable to be able to do that;
>> it's a real need that has come up in practice.
>
> So, given an arbitrary struct, with fields (generally) of different
> types, you can iterate field by field?
Ah, no. A tuple in C++ is a different thing, and much more like a
Python tuple: http://www.boost.org/libs/tuple
> That's basically what the "heterogeneous tuple" is supposed to be
> equivalent to, in the "party line" I was relating
Well, that's just nutty. We can make a much better analogy to a
struct in Python with the usual class hacks. But even then, we can
arrange to iterate the attributes.
> (although the names to go with the fields are only present in a FEW
> such tuples, such as those returned by modules time and stat,
Whaa?? Are they subclassing tuple? Nope, time.struct_time is not
even a tuple. It's just the struct hack with a tuple-like repr()
>>> time.localtime()
(2003, 9, 27, 12, 10, 43, 5, 270, 1)
>>> type(time.localtime())
<type 'time.struct_time'>
>>> time.struct_time.__bases__
(<type 'object'>,)
> as of now; more general tuples still haven't grown the ability to
> access fields by name, in Python).
>
> My partial dissent comes from feeling the need for "frozen/immutable
> lists" and the fact that tuples are often used as such (including goofy
> immutable representations of dicts, e.g. via tuple(thedict.iteritems()),
> and the like). If tuples cannot be thought of as immutable lists, then
> (I think) we need "immutable/hashable/frozen" lists by other means.
I agree completely.
> (I need to say "I think" because, as I quoted, Ruby experts claim that
> the "anyobject.freeze" feature they do have isn't actually as useful
> as it SEEMS it should be -- though I'm not sure I understand why, yet).
Immutability is extremely useful whenever you have inplace operations
like +=, because it guarantees value stability for other references
to the same object.
David Abrahams
> On Sat, 27 Sep 2003 15:22:37 GMT, Alex Martelli <al...@aleax.it>
> wrote:
>
>>David Abrahams wrote:
>> ...
>>> Well, (understanding that you don't nececessarily agree with the
>>> above) you can in fact iterate on std::pair<T,T> with the usual C++
>>> iterator protocol,
>>
>>You mean there's an std::pair::begin etc?! OK, I guess I'm even
>>rustier on standard C++ than I thought I was -- I could have SWORN
>>there wasn't. (Std chapter & verse pls? I plan to win bets based
>>on this tidbit...!-). So I guess the lack of those in gcc is a
>>breach of the Standard on gcc's part...?
>
> Somehow I think David is mistaken here - I cannot believe that
> dereferencing an iterator returns a different datatype depending on
> which item it happens to point to at runtime in statically typed
> C++,
You didn't read carefully enough: I said std::pair<T,T>, not
std::pair<T,U>.
> and without that ability to dereference the iterator (1) I cannot see
> the point of iterating through a pair, and (2) the 'iterator' would
> not be a true iterator as C++ iterators have to comply with one of a
> set of standard protocols (forward, bidirectional, random etc) which
> all include subscripting.
I'm pretty well familiar with those protocols - I've been working on
the C++ standards committee since 1997 and have written several
related proposals, c.f. http://tinyurl.com/ovpe.
> Of course you can iterate through a container that holds std::pair
> objects - you do that every time you iterate through an std::map - but
> that isn't the same thing.
No, definitely not.
John Roth
"David Abrahams" <da...@boost-consulting.com> wrote in message
news:ullsa5...@boost-consulting.com...
> Alex Martelli <al...@aleax.it> writes:
> >
> > So, given an arbitrary struct, with fields (generally) of different
> > types, you can iterate field by field?
>
> Ah, no. A tuple in C++ is a different thing, and much more like a
> Python tuple: http://www.boost.org/libs/tuple
>
> > That's basically what the "heterogeneous tuple" is supposed to be
> > equivalent to, in the "party line" I was relating
>
> Well, that's just nutty. We can make a much better analogy to a
> struct in Python with the usual class hacks. But even then, we can
> arrange to iterate the attributes.
>
> > (although the names to go with the fields are only present in a FEW
> > such tuples, such as those returned by modules time and stat,
>
> Whaa?? Are they subclassing tuple? Nope, time.struct_time is not
> even a tuple. It's just the struct hack with a tuple-like repr()
>
> >>> time.localtime()
> (2003, 9, 27, 12, 10, 43, 5, 270, 1)
> >>> type(time.localtime())
> <type 'time.struct_time'>
> >>> time.struct_time.__bases__
> (<type 'object'>,)
Not exactly. It's an object that can be subscripted like a
sequence, so it looks like a tuple for the most common
use cases. I haven't looked at the code, so I don't
know how far they went in putting in the remainder of
tuple behavior, though.
The subscripting and slicing is the key point here, though.
That's what makes it backward compatible. It's not an
extended tuple, and I doubt if it has most of the tuple
methods.
Frankly, I prefer to call it a data object, rather than
a "struct hack." Calling it a data object means that
it might grow other useful behaviors in the future. I'm
not sure about time, but I could appreciate methods
in stat that apply the access and modify dates to another
file in one operation.
>
> > as of now; more general tuples still haven't grown the ability to
> > access fields by name, in Python).
And they most likely won't. It's easy enough to create
a data object that implements the basic part of the sequence
protocol.
John Roth
Stephen Horne
<da...@boost-consulting.com> wrote:
>Stephen Horne <$$$$$$$$$$$$$$$$$@$$$$$$$$$$$$$$$$$$$$.co.uk> writes:
>> Somehow I think David is mistaken here - I cannot believe that
>> dereferencing an iterator returns a different datatype depending on
>> which item it happens to point to at runtime in statically typed
>> C++,
>
>You didn't read carefully enough: I said std::pair<T,T>, not
>std::pair<T,U>.
>
>> and without that ability to dereference the iterator (1) I cannot see
>> the point of iterating through a pair, and (2) the 'iterator' would
>> not be a true iterator as C++ iterators have to comply with one of a
>> set of standard protocols (forward, bidirectional, random etc) which
>> all include subscripting.
>
>I'm pretty well familiar with those protocols - I've been working on
>the C++ standards committee since 1997 and have written several
>related proposals, c.f. http://tinyurl.com/ovpe.
OK - sorry for that.
I remain surprised that this degree of specialisation occurs, but it's
a case of live and learn I suppose.
Mel Wilson
Tom Anderson <tw...@urchin.earth.li> wrote:
>On Thu, 25 Sep 2003, Steve Holden wrote:
>
>> "Raymond Hettinger" <vze4...@verizon.net> wrote ...
>> > Please comment on the new PEP for reverse iteration methods.
>> [...]
>> > It crashes when applied to an infinite iterator.
>> Hmm. My little brain is having difficulty imagining anything that won't.
>> What am I missing?
>
:> But I wouldn't wait for it, any more than I would
wait for
t = 1 / (1 / 0.0)
I can imagine seeing an InfiniteIteratorException coming up
if I were to call for generalized reverse iteration wrongly.
Regards. Mel.
David Abrahams
>>> and without that ability to dereference the iterator (1) I cannot see
>>> the point of iterating through a pair, and (2) the 'iterator' would
>>> not be a true iterator as C++ iterators have to comply with one of a
>>> set of standard protocols (forward, bidirectional, random etc) which
>>> all include subscripting.
>>
>>I'm pretty well familiar with those protocols - I've been working on
>>the C++ standards committee since 1997 and have written several
>>related proposals, c.f. http://tinyurl.com/ovpe.
>
> OK - sorry for that.
No prob.
> I remain surprised that this degree of specialisation occurs, but
> it's a case of live and learn I suppose.
Sorry, what specialization?
Stephen Horne
Presumably template specialisation - such that the special case of
std::pair<T,T> picks up the iterating functionality that
std::pair<T,U> lacks (begin, end etc). That is what I thought you were
saying.
Or am I still getting this wrong?
David Abrahams
>>Sorry, what specialization?
>
> Presumably template specialisation - such that the special case of
> std::pair<T,T> picks up the iterating functionality that
> std::pair<T,U> lacks (begin, end etc). That is what I thought you were
> saying.
>
> Or am I still getting this wrong?
Yeah, slightly. You don't need a begin() member function in order to
make an iterator. The interface might look like:
std::for_each(pair_iterator<T>(my_pair), pair_iterator<T>(), f);
Decoupling is the way to go, man! :^)
Stephen Horne
Ah - I get it! - std::pair doesn't exactly support iteration itself,
but a support class can be used to add that capability.
You can do this in any language. For instance, did you know that
Python classes supports iterating through the subset of their
attribute that have names beginning with "a", interleaved with
insults? Yes, all you need is to use this generator...
def A_Attrib_Gen (p_Instance) :
for i in dir (p_Instance) :
if i[0] = "a" :
yield i
yield "stupid stupid stupid"
Well, OK, maybe this is a little unfair - this generator isn't exactly
in the library, but with a little luck you see my point.
When you say "you can in fact iterate on std::pair<T,T> with the usual
C++ iterator protocol" it implies to me that std::pair<T,T> provides
the iterator protocol itself - not that some other class provides a
way to support iteration over the pair. After all, there is *always*
some way to support iteration of *anything*.
But maybe I'm just being overliteral.
David Abrahams
> On Sun, 28 Sep 2003 11:22:11 -0400, David Abrahams
> <da...@boost-consulting.com> wrote:
>
>>Stephen Horne <$$$$$$$$$$$$$$$$$@$$$$$$$$$$$$$$$$$$$$.co.uk> writes:
>>
>>>>Sorry, what specialization?
>>>
>>> Presumably template specialisation - such that the special case of
>>> std::pair<T,T> picks up the iterating functionality that
>>> std::pair<T,U> lacks (begin, end etc). That is what I thought you were
>>> saying.
>>>
>>> Or am I still getting this wrong?
>>
>>Yeah, slightly. You don't need a begin() member function in order to
>>make an iterator. The interface might look like:
>>
>> std::for_each(pair_iterator<T>(my_pair), pair_iterator<T>(), f);
>>
>>Decoupling is the way to go, man! :^)
>
> Ah - I get it! - std::pair doesn't exactly support iteration itself,
> but a support class can be used to add that capability.
>
> You can do this in any language.
Yes; I never implied otherwise.
> For instance, did you know that Python classes supports iterating
> through the subset of their attribute that have names beginning with
> "a", interleaved with insults?
I never, anywhere, said "std::pair supports..."
> Yes, all you need is to use this
> generator...
>
> def A_Attrib_Gen (p_Instance) :
> for i in dir (p_Instance) :
> if i[0] = "a" :
> yield i
> yield "stupid stupid stupid"
>
> Well, OK, maybe this is a little unfair - this generator isn't exactly
> in the library, but with a little luck you see my point.
Your sarcasm is briefly amusing, but unwarranted.
> When you say "you can in fact iterate on std::pair<T,T> with the usual
> C++ iterator protocol" it implies to me that std::pair<T,T> provides
> the iterator protocol itself
Hey, it's not my fault you read what you want to see into what I
posted.
> - not that some other class provides a
> way to support iteration over the pair. After all, there is *always*
> some way to support iteration of *anything*.
>
> But maybe I'm just being overliteral.
No, you just don't know what "the usual iterator protocol" means. It
has to do with the protocol for using iterators, and begin()/end() are
nowhere in the iterator requirements. They're part of the container
requirements. There are many iterators that have nothing to do with
containers and have never been passed through a begin()/end().
Pointers to built-in arrays are one obvious example.
This whole discussion started out talking about what the Python
protocol for manipulating iterators should be. The fact that
iterators must themselves have an __iter__ method in Python may just
be making this more confusing than it needs to be.
Stephen Horne
It was my intention to explain the origins of my misunderstanding,
though certainly in a slightly less than serious way and with a big
dollop of its-not-all-my-fault-you-know. I tend to screw this kind of
thing up, and it seems I have done so again here.
Everything you have said in your last reply was at least 95% valid,
but there is a point I'd still like to make...
>I never, anywhere, said "std::pair supports..."
...
>> When you say "you can in fact iterate on std::pair<T,T> with the usual
>> C++ iterator protocol" it implies to me that std::pair<T,T> provides
>> the iterator protocol itself
>
>Hey, it's not my fault you read what you want to see into what I
>posted.
...
>> But maybe I'm just being overliteral.
>
>No, you just don't know what "the usual iterator protocol" means.
I never said, anywhere, that you said that "std::pair supports..." if
we are being that pedantic, but in the life of this thread neither of
us really has been that pedantic. I quoted your exact words for the
crucial point, which were...
"you can in fact iterate on std::pair<T,T> with the usual C++ iterator
protocol"
Pedantically speaking, this is imprecise language. Most significantly,
there is no such thing as "the usual C++ iterator protocol". An
iterator may implement any one of five different iterator protocols,
and these protocols have surprisingly little in common. All require an
operator++ and all require a dereference-style operator* and that is
it.
Actually, even the operator* isn't as common as it seems - input
iterators only support read access, output ones only write access - so
the practical commonality is limited to the operator++.
But then you weren't writing a standards document, and I wasn't
reading it as a standards document.
To most C++ programmers most of the time, the words "the usual C++
iterator protocol" don't have the pedantic meaning set by the C++
standard - they are not just about having an operator++. They have a
somewhat more pragmatic meaning, which includes the usual means of
obtaining iterators from containers. std::pair *is* a container in the
general sense of containing other objects, even though in the C++
standards document context it is not formally considered a container.
Of course there is room for misinterpretation in virtually any piece
of writing - criticism of your choice of words was certainly
intentional, but meant to be lighthearted.
But if you really believe that I "read what [I] want to see into what
[you] posted" then I'm afraid you're wrong. I saw an implication which
you never intended, but that was right at the start when I had no
reason to "want to see" anything in particular.
I do have a certain history in this kind of
minor-misunderstanding-gets-overblown storyline, as Alex Martelli I
think can confirm if he's still following the thread. Actually, he'll
probably say you should consider yourself lucky that it only got this
bad ;-)
Anyway, I'm in no doubt that I'm primarily (probably entirely)
responsible for the 'overblown' part especially with my previous post.
I appologise for that.
>This whole discussion started out talking about what the Python
>protocol for manipulating iterators should be. The fact that
>iterators must themselves have an __iter__ method in Python may just
>be making this more confusing than it needs to be.
We may think of '__iter__' as part of the iterator protocol but,
pedantically speaking, it is no more part of the Python iterator
protocol than 'begin' is part of the C++ iterator protocol.
Iterators don't have to have an '__iter__' method in Python. Iterators
only have to have a 'next' method. It is the iterable object that
implements '__iter__'. And, as with getting C++ iterators from 'begin'
etc, even that is a convention rather than a requirement.
Sometimes iterators do have an '__iter__' method, but that normally
just returns self - it's a quick-fix convenience for when iterators
end up in a context where an iterable object is expected.
I guess our mindsets aren't so different that we can't make similar
mistakes ;-)
BTW - there's no reason why an iterator can't have additional methods
beyond those required for the iterator protocol. So a container class
could, for instance, support the following...
for i in iter (container).range (begin, end).reverse () :
...
Simply by defining its iterator class as something like...
class Iter (object) :
def __init__ (self, p_Src) :
"Keep ref to container to iterate, plus other setup stuff."
self.Src = p_Src
...
def __iter__ (self) :
"Allow iterator to behave as iterable for convenience"
return self
def range (self, p_Begin, p_End) :
"Set restriction on range to be iterated"
...
return self
def reverse (self) :
"Set reverse-order iteration mode"
...
return self
def next (self) :
"Get next item"
...
Which reminds me of Paul Foleys post in "Thoughts on PEP284",
Message-ID: <m2eky8d...@mycroft.actrix.gen.nz>, about Lisp-like
for-loops - though actually it's really just a minor variation of what
Raymond suggested right from the start.
Hmmm - possibly this suggestion should be made closer to the root.
Stephen Horne
<vze4...@verizon.net> wrote:
>Please comment on the new PEP for reverse iteration methods.
>Basically, the idea looks like this:
I have one last suggestion to make on this. Instead of adding a method
to the container, possibly it should be added to the iterator.
If the iterator class was roughly equivalent to the following...
class Iter (object) :
def __init__ (self, p_Src) :
"Keep ref to container to iterate, plus other setup stuff."
self.Src = p_Src
...
def __iter__ (self) :
"Allow iterator to behave as iterable for convenience"
return self
def reverse (self) :
"Set reverse-order iteration mode"
...
return self
def next (self) :
"Get next item"
...
We could write...
for i in iter (seq).reverse () :
Possible advantages being...
1. The need for/existence of an iterator is made explicit by the
already familiar 'iter' call.
2. Because the 'reverse' method is applied to the iterator rather
than the container, the existing spelling can be used without
worries about iterating over strings (or user classes) that
already have a 'reverse' method.
3. It's an extensible approach (other 'iterator modifiers' could be
defined in much the same way, e.g. range restriction) yet at the
same time both simple and lightweight.
The xrange and enumerate classes would probably also adopt the
'reverse' spelling for consistency...
for i in xrange(10).reverse() :
...
David Abrahams
I wasn't even going to post this to the list, because it's so full of
static and other unpythonic stuff. But then, Stephen hides his email
address, and there is one real Python-related issue at the bottom,
so...
Stephen Horne <$$$$$$$$$$$$$$$$$@$$$$$$$$$$$$$$$$$$$$.co.uk> writes:
> It was my intention to explain the origins of my misunderstanding,
> though certainly in a slightly less than serious way and with a big
> dollop of its-not-all-my-fault-you-know.
I didn't need to have an assignment of blame in order to find closure
with this thread. I was just trying to be helpful.
> I tend to screw this kind of thing up, and it seems I have done so
> again here.
Then I can't understand why you continue to charge about in the china
shop....
> I never said, anywhere, that you said that "std::pair supports..." if
> we are being that pedantic, but in the life of this thread neither of
> us really has been that pedantic.
Seems like you've started.
> I quoted your exact words for the crucial point, which were...
>
> "you can in fact iterate on std::pair<T,T> with the usual C++ iterator
> protocol"
>
> Pedantically speaking, this is imprecise language.
No, there is a common subset of all iterator requirements and that's
what I was referring to.
> Most significantly, there is no such thing as "the usual C++
> iterator protocol". An iterator may implement any one of five
> different iterator protocols, and these protocols have surprisingly
> little in common. All require an operator++ and all require a
> dereference-style operator* and that is it.
And that is the usual iterator protocol. I happen to know exactly
what those protocols have in common.
> Actually, even the operator* isn't as common as it seems - input
> iterators only support read access, output ones only write access
Actually it's subtler than that. You can have input/output iterators
which support random access which aren't random access iterators
because they don't support lvalue access. So what, though?
> - so the practical commonality is limited to the operator++.
You need operator* regardless.
> But then you weren't writing a standards document, and I wasn't
> reading it as a standards document.
That's why I didn't understand why you were giving me such a hard
time. Because it's informal speech I'm supposed to do quadruple duty
to make sure you haven't misinterpreted me? I really was going out
of my way to explain this stuff to you politely.
> To most C++ programmers most of the time, the words "the usual C++
> iterator protocol" don't have the pedantic meaning set by the C++
> standard - they are not just about having an operator++
And operator*.
> They have a somewhat more pragmatic meaning, which includes the
> usual means of obtaining iterators from containers. std::pair *is* a
> container in the general sense of containing other objects, even
> though in the C++ standards document context it is not formally
> considered a container.
Then so is
struct { int x, y; };
Also, so is char[4], and unsigned long is a container of at least 32
bits.
This is getting ridiculous. It seems like you want to have a pedantic
debate about terms whose meaning is going to be defined by your
completely informal and subjective interpretation.
> Of course there is room for misinterpretation in virtually any piece
> of writing - criticism of your choice of words was certainly
> intentional, but meant to be lighthearted.
>
> But if you really believe that I "read what [I] want to see into what
> [you] posted" then I'm afraid you're wrong. I saw an implication which
> you never intended, but that was right at the start when I had no
> reason to "want to see" anything in particular.
>
> I do have a certain history in this kind of
> minor-misunderstanding-gets-overblown storyline, as Alex Martelli I
> think can confirm if he's still following the thread. Actually,
> he'll probably say you should consider yourself lucky that it only
> got this bad ;-)
>
> Anyway, I'm in no doubt that I'm primarily (probably entirely)
> responsible for the 'overblown' part especially with my previous post.
> I appologise for that.
Thanks, I think. This is somewhat of a backhanded apology, but I'll
take it. [If you had just stopped with the last message, I wouldn't
even feel it mattered].
>>This whole discussion started out talking about what the Python
>>protocol for manipulating iterators should be. The fact that
>>iterators must themselves have an __iter__ method in Python may just
>>be making this more confusing than it needs to be.
>
> We may think of '__iter__' as part of the iterator protocol but,
> pedantically speaking, it is no more part of the Python iterator
> protocol than 'begin' is part of the C++ iterator protocol.
>
> Iterators don't have to have an '__iter__' method in Python. Iterators
> only have to have a 'next' method. It is the iterable object that
> implements '__iter__'. And, as with getting C++ iterators from 'begin'
> etc, even that is a convention rather than a requirement.
Sorry,
http://www.python.org/doc/current/lib/typeiter.html#l2h-149
Read it twice, carefully.
> Sometimes iterators do have an '__iter__' method, but that normally
> just returns self
Always.
> - it's a quick-fix convenience for when iterators
> end up in a context where an iterable object is expected.
>
> I guess our mindsets aren't so different that we can't make similar
> mistakes ;-)
I'm afraid not. I never, ever make mistakes in the first place ;->
infallib-ly y'rs
Stephen Horne
<da...@boost-consulting.com> wrote:
>That's why I didn't understand why you were giving me such a hard
>time. Because it's informal speech I'm supposed to do quadruple duty
>to make sure you haven't misinterpreted me? I really was going out
>of my way to explain this stuff to you politely.
Nope - occasional misreadings are a fact of life and shouldn't be a
big deal. But when you said "I never, anywhere, said "std::pair
supports..."" that seemed very pedantic to me, as the issue wasn't
your precise words but the meaning behind them. And when you said
"Hey, it's not my fault you read what you want to see into what I
posted." you seemed to be claiming that there was no room for
reasonable misinterpretation in your original words and that I was
100% to blame for the misinterpretation.
The whole point of my pedanticism was to point out that your original
statement was informal and subjective and that, while that was
certainly wholy appropriate, there was room for accidental
misinterpretation. IIRC I was not even the first to misinterpret - I
simply replied to Alex Martelli who had already misinterpreted your
words the same way that I did. So being made a scapegoat seemed
unfair.
And now I seem to be whining about it far to much, but really I just
want this point to be understood - and I wish I had been clear about
it in my last post.
>Then so is
>
> struct { int x, y; };
>
>Also, so is char[4], and unsigned long is a container of at least 32
>bits.
Not in the pedantic 'C++ standard definition' sense of course, but the
C++ standard has taken an existing word and tied a more restrictive
meaning to it. In the wider world the original unrestricted meaning is
still valid.
To me both structs and char arrays are containers in the
less-restrictive sense. After all, if a C programmer talks about
containers what is he referring to? And in C# even a fixed-size array
is implemented using a library class.
An unsigned long, no - that is a single atomic item. But then again,
IIRC, in the pre-C++ days when I first learned programming, 'container
for a value' was one common way of explaining the concept of a
variable ;-)
>Thanks, I think. This is somewhat of a backhanded apology, but I'll
>take it. [If you had just stopped with the last message, I wouldn't
>even feel it mattered].
I felt it mattered because I felt that I was being held soley and
entirely responsible for a mistake that I felt wasn't unreasonable.
I appologise for the misinterpretation because I did misinterpret, and
I particularly appologise for the overblowing because it arises out of
my oversensitivity to certain things you said - which I certainly have
no right to given the sarcastic tone of my earlier post, and in any
case I have no doubt your words resulted from frustration.
I do not, however, accept that I am enirely and soley to responsible
for the misinterpretation.
My persistence in sticking to this may seem bizarre and petty, but
there are reasons for my being bizarre and petty over such things.
Lets just call it 'desperate defending of what little self esteem I
have left' - and yes, it does tend to be counterproductive :-(
>http://www.python.org/doc/current/lib/typeiter.html#l2h-149
>
>Read it twice, carefully.
OK - you are right and I am feeling a right fool. Sorry.
>I'm afraid not. I never, ever make mistakes in the first place ;->
>
>infallib-ly y'rs
:-)