A quirk/gotcha of for i, x in enumerate(seq) when seq is empty

[Alex Willmer]

This week I was slightly surprised by a behaviour that I've not
considered before. I've long used

for i, x in enumerate(seq):
# do stuff

as a standard looping-with-index construct. In Python for loops don't
create a scope, so the loop variables are available afterward. I've
sometimes used this to print or return a record count e.g.

for i, x in enumerate(seq):
# do stuff
print 'Processed %i records' % i+1

However as I found out, if seq is empty then i and x are never
created. The above code will raise NameError. So if a record count is
needed, and the loop is not guaranteed to execute the following seems
more correct:

i = 0
for x in seq:
# do stuff
i += 1
print 'Processed %i records' % i

Just thought it worth mentioning, curious to hear other options/
improvements/corrections.

[Steven D'Aprano]

On Thu, 23 Feb 2012 16:30:09 -0800, Alex Willmer wrote:

> This week I was slightly surprised by a behaviour that I've not
> considered before. I've long used
>
> for i, x in enumerate(seq):
> # do stuff
>
> as a standard looping-with-index construct. In Python for loops don't
> create a scope, so the loop variables are available afterward. I've
> sometimes used this to print or return a record count e.g.
>
> for i, x in enumerate(seq):
> # do stuff
> print 'Processed %i records' % i+1
>
> However as I found out, if seq is empty then i and x are never created.

This has nothing to do with enumerate. It applies to for loops in
general: the loop variable is not initialised if the loop never runs.
What value should it take? Zero? Minus one? The empty string? None?
Whatever answer Python choose would be almost always wrong, so it refuses
to guess.

> The above code will raise NameError. So if a record count is needed, and
> the loop is not guaranteed to execute the following seems more correct:
>
> i = 0
> for x in seq:
> # do stuff
> i += 1
> print 'Processed %i records' % i

What fixes the problem is not avoiding enumerate, or performing the
increments in slow Python instead of fast C, but that you initialise the
loop variable you care about before the loop in case it doesn't run.

i = 0
for i,x in enumerate(seq):
# do stuff

is all you need: the addition of one extra line, to initialise the loop
variable i (and, if you need it, x) before hand.




--
Steven

[Paul Rubin]

Alex Willmer <al...@moreati.org.uk> writes:
> i = 0
> for x in seq:
> # do stuff
> i += 1
> print 'Processed %i records' % i
>
> Just thought it worth mentioning, curious to hear other options/
> improvements/corrections.

Stephen gave an alternate patch, but you are right, it is a pitfall that
can be easy to miss in simple testing.

A more "functional programming" approach might be:

def do_stuff(x): ...

n_records = sum(1 for _ in imap(do_stuff, seq))

[Ethan Furman]

Actually,

i = -1

or his reporting will be wrong.

~Ethan~

[Mark Lawrence]

On 24/02/2012 03:49, Ethan Furman wrote:

> Actually,
>
> i = -1
>
> or his reporting will be wrong.
>
> ~Ethan~

Methinks an off by one error :)

--
Cheers.

Mark Lawrence.

[Peter Otten]

Ethan Furman wrote:

> Steven D'Aprano wrote:

> Actually,
>
> i = -1
>
> or his reporting will be wrong.

Yes, either

i = -1

for i, x in enumerate(seq):

...
print "%d records" % (i+1)

or

i = 0
for i, x in enumerate(seq, 1):
...
print "%d records" % i

[Rick Johnson]

On Feb 23, 6:30 pm, Alex Willmer <a...@moreati.org.uk> wrote:
> [...]

> as a standard looping-with-index construct. In Python for loops don't
> create a scope, so the loop variables are available afterward. I've
> sometimes used this to print or return a record count e.g.
>
> for i, x in enumerate(seq):
>    # do stuff
> print 'Processed %i records' % i+1

You could employ the "else clause" of "for loops" to your advantage;
(psst: which coincidentally are working pro-bono in this down
economy!)

>>> for x in []:
... print x
... else:
... print 'Empty Iterable'
Empty Iterable

>>> for i,o in enumerate([]):
... print i, o
... else:
... print 'Empty Iterable'
Empty Iterable

[Peter Otten]

Rick Johnson wrote:

> On Feb 23, 6:30 pm, Alex Willmer <a...@moreati.org.uk> wrote:
>> [...]
>> as a standard looping-with-index construct. In Python for loops don't
>> create a scope, so the loop variables are available afterward. I've
>> sometimes used this to print or return a record count e.g.
>>
>> for i, x in enumerate(seq):
>> # do stuff
>> print 'Processed %i records' % i+1
>
> You could employ the "else clause" of "for loops" to your advantage;

>>>> for x in []:
> ... print x
> ... else:
> ... print 'Empty Iterable'
> Empty Iterable
>
>>>> for i,o in enumerate([]):
> ... print i, o
> ... else:
> ... print 'Empty Iterable'
> Empty Iterable

No:


>>> for i in []:
... pass
... else:
... print "else"
...
else
>>> for i in [42]:
... pass
... else:
... print "else"
...
else
>>> for i in [42]:
... break
... else:
... print "else"
...
>>>

The code in the else suite executes only when the for loop is left via
break. A non-empty iterable is required but not sufficient.

[Peter Otten]

Peter Otten wrote:

> The code in the else suite executes only when the for loop is left via
> break.

Oops, the following statement is nonsense:

> A non-empty iterable is required but not sufficient.

Let me try again:

A non-empty iterable is required but not sufficient to *skip* the else-suite
of a for loop.

[Steven D'Aprano]

On Fri, 24 Feb 2012 13:44:15 +0100, Peter Otten wrote:

>>>> for i in []:
> ... pass
> ... else:
> ... print "else"
> ...
> else
>>>> for i in [42]:
> ... pass
> ... else:
> ... print "else"
> ...
> else
>>>> for i in [42]:
> ... break
> ... else:
> ... print "else"
> ...
>>>>
>>>>
> The code in the else suite executes only when the for loop is left via
> break. A non-empty iterable is required but not sufficient.

You have a typo there. As your examples show, the code in the else suite
executes only when the for loop is NOT left via break (or return, or an
exception). The else suite executes regardless of whether the iterable is
empty or not.


for...else is a very useful construct, but the name is misleading. It
took me a long time to stop thinking that the else clause executes when
the for loop was empty.

In Python 4000, I think for loops should be spelled:

for name in iterable:
# for block
then:
# only if not exited with break
else:
# only if iterable is empty

and likewise for while loops.

Unfortunately we can't do the same now, due to the backward-incompatible
change in behaviour for "else".



--
Steven

[Arnaud Delobelle]

On 24 February 2012 14:54, Steven D'Aprano

<steve+comp....@pearwood.info> wrote:

> for...else is a very useful construct, but the name is misleading. It
> took me a long time to stop thinking that the else clause executes when
> the for loop was empty.

This is why I think we should call this construct "for / break / else"
rather than "for / else".

--
Arnaud

[Peter Otten]

Steven D'Aprano wrote:

>> The code in the else suite executes only when the for loop is left via
>> break. A non-empty iterable is required but not sufficient.
>
> You have a typo there. As your examples show, the code in the else suite
> executes only when the for loop is NOT left via break (or return, or an
> exception). The else suite executes regardless of whether the iterable is
> empty or not.

Yup, sorry for the confusion.

[Rick Johnson]

On Feb 24, 8:54 am, Steven D'Aprano <steve

+comp.lang.pyt...@pearwood.info> wrote:

> for...else is a very useful construct, but the name is misleading. It
> took me a long time to stop thinking that the else clause executes when
> the for loop was empty.

Agreed. This is a major stumbling block for neophytes.

> In Python 4000, I think for loops should be spelled:
>
> for name in iterable:
>     # for block
> then:
>     # only if not exited with break
> else:
>     # only if iterable is empty
>
> and likewise for while loops.

I like this syntax better than the current syntax, however, it is
STILL far too confusing!

> for name in iterable:
>     # for block

this part is okay

> then:
>     # only if not exited with break

I only know how the "then" clause works if you include that comment
each and every time!

> else:
>     # only if iterable is empty

Again. I need more info before this code becomes intuitive. Too much
guessing is required. Not to mention that the try/except/else suite
treats "else" differently.

try:
do_this()
except EXCEPTION:
recover()
else NO_EXCEPTION:
okay_do_this_also().

for x in iterable:
do_this()
except EXCEPTION:
recover()
else NO_EXCEPTION:
do_this_also()

while LOOPING:
do_this()
except EXCEPTION:
break or recover()
else NO_EXCEPTION:
do_this_also()

In this manner "else" will behave consistently between exception
handling and looping.

But this whole idea of using an else clause is ridiculous anyway
because all you've done is to "break up" the code visually. Not to
mention; breaking the cognitive flow of a reader!

try:
do_this()
okay_do_this_also()
what_the_heck.do_this_too()
except EXCEPTION:
recover()
finally:
always_do_this()

Loop syntax can drop the "else" and adopt "then/finally" -- if you
think we even need a finally!?!?

for x in iterable:
do_this()
except EXCEPTION:
recover()
then:
okay_do_this_also()
what_the_heck.do_this_too()
finally:
always_do_this()

while LOOPING:
do_this()
except EXCEPTION:
recover()
then:
okay_do_this_also()
what_the_heck.do_this_too()
finally:
always_do_this()

[Chris Angelico]

On Wed, Feb 29, 2012 at 9:56 AM, Rick Johnson
<rantingri...@gmail.com> wrote:
> On Feb 24, 8:54 am, Steven D'Aprano <steve
> +comp.lang.pyt...@pearwood.info> wrote:
>
>> In Python 4000, I think for loops should be spelled:
>>
>> for name in iterable:
>>     # for block
>> then:
>>     # only if not exited with break
>> else:
>>     # only if iterable is empty
>>
>> and likewise for while loops.
>
> I like this syntax better than the current syntax, however, it is
> STILL far too confusing!

Absolutely, it's FAR too confusing. Every syntactic structure should
have the addition of a "foo:" suite, which will run when the
programmer expects it to and no other time. This would solve a LOT of
problems.

ChrisA

[Steven D'Aprano]

On Wed, 29 Feb 2012 10:24:18 +1100, Chris Angelico wrote:

> Every syntactic structure should
> have the addition of a "foo:" suite, which will run when the programmer
> expects it to and no other time. This would solve a LOT of problems.

Indeed, when I design my killer language, the identifiers "foo"
and "bar" will be reserved words, never used, and not even
mentioned in the reference manual. Any program using one will
simply dump core without comment. Multitudes will rejoice.
-- Tim Peters, 29 Apr 1998



--
Steven