recognizing empty iterators
Michele Simionato
I realized that I don't know a satisfactory way to check if an
iterator is empty. In other words I am looking for an
"isempty" function to use in "if" statements such as
if isempty(iterator):
do_something()
without side effects. Here are some unsatisfactory ways of implementing
"isempty":
#1: converting to a list or tuple
def isempty(iterator):
return not list(iterator)
Easy, but one has to create the entire list, thus defecting the basic
purpose of the iterator, i.e. lazy evaluation.
#2: checking for StopIteration
def isempty(iterator):
try:
iterator.next()
except StopIteration:
return True
else:
return False
This works, for instance
print isempty(iter([]))
gives True and
it=iter([1,2,3])
print isempty(it)
gives False. However, there is a side effect: after the check, the
iterator has advanced of one step and now "it.next()" gives 2, not 1.
In order this to work without side effects, I should be able to restart
the iterator from the beginning, but I don't know how to do that.
Is it possible?
#3: comparing with the empty iterator
emptyiterator=iter([])
it=iter([])
if it == emptyiterator: print 'Ok!'
This simply doesn't work.
I wonder if the itertools module should contain a function to check for
empty iterators, thus simplifying my life ;) Of course, I may well be
missing something obvious, if so, please enlighten me.
Michele
Gonçalo Rodrigues
wrote:
I don't think you're missing anything, the iterator protocol is just
not designed to answer that question. The best that can be got is your
example #2.
I always make sure to add __nonzero__ to any iterator I code - and
where it makes sense, of course.
>
> Michele
With my best regards,
G. Rodrigues
Chris Perkins
> I realized that I don't know a satisfactory way to check if an
> iterator is empty. In other words I am looking for an
> "isempty" function to use in "if" statements such as
>
> if isempty(iterator):
> do_something()
>
> without side effects.
>
In general, it is impossible to test whether an iterator is exhausted.
For example, how would you test the following:
import random
def am_i_empty():
while random.choice((True,False)):
yield 42
return
So you have to call "next" - there's no way around it. You just need a
wrapper to cache the value:
class checkable_iter:
def __init__(self,it):
self.it = iter(it)
self.cached = []
def isempty(self):
if self.cached:
return False
try:
self.cached.append(self.it.next())
return False
except StopIteration:
return True
def next(self):
if self.cached:
return self.cached.pop()
return self.it.next()
>>> it = checkable_iter(am_i_empty())
>>> it.isempty()
False
>>> it.next()
42
>>> it.isempty()
True
>>> it.next()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
StopIteration
Chris
MBR
at most one object ahead. (Would only need to cache when IsEmpty was
asked.) Of course, whether this counts as a side-effect depends on
your interator and its use...
mi...@pitt.edu (Michele Simionato) wrote in message news:<2259b0e2.03072...@posting.google.com>...
Bengt Richter
If iterators had a .peek() method for 1-item lookahead, maybe that would
be easy to implement. If empty, it could raise StopIteration, and otherwise
return what next() would return, without disturbing the state w.r.t. next().
Alternatively, it could return [nextitem] vs [] if at the end
(without raising StopIteration).
Regards,
Bengt Richter
John Roth
"Bengt Richter" <bo...@oz.net> wrote in message
news:bfhvnk$hvd$0...@216.39.172.122...
I think that's two separate functions. isEmpty() is not the same
thing as willBeEmptyOnNextCall().
That said, there are places where a 1 item lookahead can be quite
valuable, and lack of a sliding window is the last major thing I'd
like to see addressed in the for loop.
John Roth
>
> Regards,
> Bengt Richter
Steven Taschuk
> [...] In other words I am looking for an
> "isempty" function to use in "if" statements such as
>
> if isempty(iterator):
> do_something()
>
I don't have anything substantial to add to others' posts, but I
wonder: under what circumstances do you want to do this?
--
Steven Taschuk 7\ 7'Z {&~ .
stas...@telusplanet.net Y r --/hG-
(__/ )_ 1^1`
Delaney, Timothy C (Timothy)
>
> If iterators had a .peek() method for 1-item lookahead, maybe
> that would
> be easy to implement. If empty, it could raise StopIteration,
> and otherwise
> return what next() would return, without disturbing the state
> w.r.t. next().
So create an iterator that has a `peek` method, and wrap all iterators in that. Easy!
Tim Delaney
Michele Simionato
> Quoth Michele Simionato:
> > [...] In other words I am looking for an
> > "isempty" function to use in "if" statements such as
> >
> > if isempty(iterator):
> > do_something()
> >
> > without side effects. [...]
>
> I don't have anything substantial to add to others' posts, but I
> wonder: under what circumstances do you want to do this?
I wanted to check the output of ifilter or imap; at the end I solved
my problem in another way, nevertheless I am surprised there is no
way to check for an empty iterator in current Python, it seems to be
a quite legitimate question, isn't it?
Michele
Daniel Dittmar
> I wanted to check the output of ifilter or imap; at the end I solved
> my problem in another way, nevertheless I am surprised there is no
> way to check for an empty iterator in current Python, it seems to be
> a quite legitimate question, isn't it?
It would make writing iterators more difficult because you'd have to know
that there is a first element before the first call to .next ().
It shouldn't be too difficult to write an iterator wrapper class that does
exactly what you want (not tested):
class IteratorWrapper:
def __init__ (self, iterArg):
iterArg = iter (iterArg)
try:
self.firstElement = iterArg.next ()
self.isEmpty = false
self.next = self.returnFirstElement
self.baseIter = iterArg
except StopIteration:
self.isEmpty = true
self.next = self.throwStopIteration
def returnFirstElement (self):
self.next = self.baseIter.next
return self.firstElement
def throwStopIteration (self):
throw StopIteration
Daniel
Terry Reedy
"Michele Simionato" <mi...@pitt.edu> wrote in message
news:2259b0e2.03072...@posting.google.com...
> I wanted to check the output of ifilter or imap; at the end I solved
> my problem in another way, nevertheless I am surprised there is no
> way to check for an empty iterator in current Python, it seems to be
> a quite legitimate question, isn't it?
As I understand, the iterator protocol was primarily designed to be
the *minimum* necessary to work (behind the scene) in
for item in iterable:
<statements>
statements, replacing and improving upon the getitem interface (kept
only for use by old code and likely to disappear in 3.0 ). It works
quite well for this designed purpose. In this context one usually
does not even have a handle on the iterator and thereor no means of
calling a precheck method.
Iterators were secondarily intended for the equivalent
iterator = iter(iterable)
try:
while 1:
item = iterator.next()
<statements>
except StopIteration: pass
Here the iterator is visibly bound, but the basic idea of 'process
each item' is the same. In any case, it is the stark simplicity of
the protocol that makes possible the current version of generators and
the neat trick of transfering the function body to the .next method.
One who wants to add more to the interface should, I think, expect to
add code somewhere to make the added function come true.
Terry J. Reedy
Steven Taschuk
> Steven Taschuk <stas...@telusplanet.net> wrote in message news:<mailman.1058835583...@python.org>...
> > I don't have anything substantial to add to others' posts, but I
> > wonder: under what circumstances do you want to do this?
>
Ah. And then, if there were elements in the iterator, I assume
you would then process them somehow, right? Why not just do that
with a for loop, say, and check afterwards whether anything has
been done? A silly example:
processed = 0
for x in iterator:
process(x)
processed = processed + 1
if processed:
# ...
> [...] at the end I solved
> my problem in another way, nevertheless I am surprised there is no
> way to check for an empty iterator in current Python, it seems to be
> a quite legitimate question, isn't it?
Absolutely. I was just curious what the original problem was.
(I haven't read the iterators PEP in a while, but it seems to me
the iterator protocol is deliberately minimalistic. Makes it
easier to write them; and if you do frequently need lookahead,
writing a wrapper as others have suggested seems easy enough.)
--
Steven Taschuk "The world will end if you get this wrong."
stas...@telusplanet.net -- "Typesetting Mathematics -- User's Guide",
Brian Kernighan and Lorrinda Cherry
Bengt Richter
>Quoth Michele Simionato:
>> Steven Taschuk <stas...@telusplanet.net> wrote in message news:<mailman.1058835583...@python.org>...
> [...]
>> > I don't have anything substantial to add to others' posts, but I
>> > wonder: under what circumstances do you want to do this?
>>
>> I wanted to check the output of ifilter or imap; [...]
>
>Ah. And then, if there were elements in the iterator, I assume
>you would then process them somehow, right? Why not just do that
>with a for loop, say, and check afterwards whether anything has
>been done? A silly example:
>
> processed = 0
> for x in iterator:
> process(x)
> processed = processed + 1
> if processed:
> # ...
>
You can avoid flag bumping in the loop:
>>> x = object(); idx=id(x)
>>> for x in iter(''): print x,
...
>>> id(x)==idx
1
>>> for x in iter('abcd'): print x,
...
a b c d
>>> id(x)==idx
0
Regards,
Bengt Richter
Alan Kennedy
> It shouldn't be too difficult to write an iterator wrapper class
> that does exactly what you want (not tested):
>
> class IteratorWrapper:
> def __init__ (self, iterArg):
> iterArg = iter (iterArg)
> try:
> self.firstElement = iterArg.next ()
> self.isEmpty = false
> self.next = self.returnFirstElement
> self.baseIter = iterArg
> except StopIteration:
> self.isEmpty = true
> self.next = self.throwStopIteration
>
> def returnFirstElement (self):
> self.next = self.baseIter.next
> return self.firstElement
>
> def throwStopIteration (self):
> throw StopIteration
^^^^^
Ahem, cough, I think, em, maybe you mean "raise"?
That java stuff polluting your mind, eh? ;-)
As an aside, I've got to hand it to Java for exception handling. Java
was the language that taught me how to do proper and robust exception
handling, by "rapping my knuckles" continually, and actually making me
think about the exception hierarchies I was generating and processing.
You can't just randomly define and generate new exceptions: you have
to change your method definition to declare that it can throw the new
exception. Which means you either have to handle the exception in the
calling method, or change *that* method signature to permit
re-throwing of the exception, and so on up the hierarchy.
It's not very pythonic though.
regards,
--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan
Michele Simionato
I see that the source of my problem was the following: whereas an
empty list or an empty tuple has a truth value of False, an empty
iterator has a truth value of True. It is easy to fix, anyway, with
a function which works as "iter", but return an empty list (or tuple)
when the iterator is empty:
import itertools
def iter_(seq):
i=iter(seq)
try:
first=i.next()
except StopIteration:
return []
return itertools.chain([first],i)
Maybe not very elegant, but it works:
it=iter_('xyz')
for c in it:
print c
if iter_(it):
print "non empty"
else:
print "is empty"
BTW, the fact that an empty iterator is True is somewhat strange,
considering that [],() and "" are False; on the other hand, generic
objects are True, so the point can be debated ...
Michele
Delaney, Timothy C (Timothy)
>
> BTW, the fact that an empty iterator is True is somewhat strange,
> considering that [],() and "" are False; on the other hand, generic
> objects are True, so the point can be debated ...
It is impossible to know, in the general case, if an iterator is empty. The primary reason for this is that many iterators are destructive e.g. file iterators. A wrapper iterator can be used around a destructive iterator that caches information, but that's about the best you can do unless the iterator is resettable.
An iterator may in fact be conceptually empty at one instant, then the next instant not be empty...
All it is possible to know is if an iterator has been exhausted *for now* - and that is if it raises StopIteration. Technically this does not mean that the iterator is permanently exhausted - it could start returning values at a later date, but this would be very confusing to people ;)
Tim Delaney
Raymond Hettinger
> BTW, the fact that an empty iterator is True is somewhat strange,
> considering that [],() and "" are False; on the other hand, generic
> objects are True, so the point can be debated ...
Requiring iterators to know whether they are empty *before* being
called would be like from requiring them to know their length in
advance (after all, the falsity of [] and () stems from their __len__()
methods returning zero). Non-sequential access and the absence
of a __len__() method are what give iterators their character. IOW,
iterators are *supposed* to not be the same as sequences.
Viva la difference.
[Timothy Delaney]
> It is impossible to know, in the general case, if an iterator is empty.
> The primary reason for this is that many iterators are destructive
> e.g. file iterators. A wrapper iterator can be used around a
> destructive iterator that caches information, but that's about the
> best you can do unless the iterator is resettable.
> An iterator may in fact be conceptually empty at one instant, then
> the next instant not be empty...
> All it is possible to know is if an iterator has been exhausted
> *for now* - and that is if it raises StopIteration. Technically this
> does not mean that the iterator is permanently exhausted - it could
> start returning values at a later date, but this would be very
> confusing to people ;)
Yes, it was. So, Guido decreed that once a iterator has returned
StopIteration, it will continue to do so on subsequent calls to
it.next(). Promptly, all of Python's iterators were modified to
comply. IOW, iterators can and do stay "permanently exhausted".
However, Tim is essentially correct in focusing on the "just-in-time"
nature of iterators. Part of the core concept for iterators is to *not*
know what lies ahead.
Raymond Hettinger
Delaney, Timothy C (Timothy)
>
> Yes, it was. So, Guido decreed that once a iterator has returned
> StopIteration, it will continue to do so on subsequent calls to
> it.next(). Promptly, all of Python's iterators were modified to
> comply. IOW, iterators can and do stay "permanently exhausted".
Yes - these are the rules, and all the standard iterators conform to them. This is a Good Thing(TM).
A non-standard iterator *may* break the rules (as with just about anything in Python). That would, however, be a Bad Thing(TM).
BTW I was there during those discussions (which is why the thought popped into my mind ...). I can't remember off the top of my head though if iter() is guaranteed to return an iterator that once exhausted will be permanently exhausted (which would probably require a proxy). I'm pretty certain though the the resolution was "the standard iterators will do the Right Thing(TM) and it's up to people to play nice with their own iterators".
> However, Tim is essentially correct in focusing on the "just-in-time"
> nature of iterators. Part of the core concept for iterators
> is to *not* know what lies ahead.
Why, thank you sir ;)
Tim Delaney