[Python-ideas] Revised**11 PEP on Yield-From
Jacob Holm
> Draft 12 of the PEP.
>
> Fixed a bug in the expansion (didn't handle
> StopIteration raised by throw).
>
Just so you know, I now agree that a long expansion with multiple
"try...except StopIteration" blocks is the right thing to do. There are
only two cases I can see where it makes a difference compared to what I
suggested:
1. Throwing StopIteration to an iterator without a throw() method. I
would prefer treating this case *exactly* as if the iterator had a
trivial throw method: "def throw(self, et, ev=None, tb=None):
raise et, ev, tb". Meaning that the StopIteration *should* be
caught by yield-from. Treating it like this makes it easier to
write wrappers that don't accidentally change the semantics of an
obscure corner case. Principle of least surprise and all that...
It is easy enough to change the expansion to do this by expanding
the try block around the throw() call or by actually using such a
trivial throw method as a fallback. Alternatively, the expansion
can be rewritten using functools.partial as in the bottom of this
mail. It has identical semantics to draft 12 of the PEP, except
for the handling of the missing throw method. I actually like that
version because it is careful about what exceptions to catch, but
still only has one "try...except StopIteration". YMMV.
2. Calling an iterator.close() that raises a StopIteration.
Arguably, such a close() is an error, so getting an exception in
the caller is better than swallowing it and turning it into a
normal return. Especially since we only called close() as part of
handling GeneratorExit in the first place.
An unrelated question... What should happen with an iterator that has a
throw or close attribute that just happens to have the value None?
Should that be treated as an error because None is not callable, or
should it be treated as if the attribute wasn't there? The expansion
handles it as if the attribute wasn't there, but IIRC your patch will
raise a TypeError trying to call None.
> Removed paragraph about StopIteration left over
> from an earlier version.
>
> Added some discussion about rejected ideas.
>
Looks good, except...
> Suggestion: If ``close()`` is not to return a value, then raise an
> exception if StopIteration with a non-None value occurs.
>
> Resolution: No clear reason to do so. Ignoring a return value is not
> considered an error anywhere else in Python.
>
I may have been unclear about why I thought this should raise a
RuntimeError. As I see it there are only two code patterns in a
generator that would have close() catch a StopIteration with a non-None
value.
* An explicit catch of GeneratorExit followed by "return Value".
This is harmless and potentially useful, although probably an
abuse of GeneratorExit (that was one of the early arguments for
not returning a value from close). Not raising a RuntimeError in
close makes it simpler to share a code path between the common and
the forced exit.
* An implicit catch of GeneratorExit, followed by "return Value".
By an "implicit catch", I mean either a catch of "BaseException"
or a "finally" clause. In both cases, "return Value" will hide
the original exception and that is almost certainly a bug.
Raising a RuntimeError would let you discover this bug early.
The question now is whether it is better to catch n00b errors or to
allow careful programmers a bit more freedom in how they structure their
code. When I started writing this mail I was leaning towards catching
errors, but I have now changed my mind. I think giving more power to
experienced users is more important.
Best regards
- Jacob
------------------------------------------------------------------------
_i = iter(EXPR)
_p = partial(next, _i)
while 1:
try:
_y = _p()
except StopIteration as _e:
_r = _e.value
break
try:
_s = yield _y
except GeneratorExit:
_m = getattr(_i, 'close', None)
if _m is not None:
_m()
raise
except:
_m = getattr(_i, 'throw', None)
if _m is None:
def _m(et, ev, tb):
raise et, ev, tb
_p = partial(_m, *sys.exc_info())
else:
if _s is None:
_p = partial(next, _i)
else:
_p = partial(_i.send, _s)
RESULT = _r
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas
Aahz
given for motivating examples that were posted in the thread, but I don't
see any of them either in the PEP itself nor at your URL with examples.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur." --Red Adair
Jacob Holm
pointing that out). I hope this is better...
Greg Ewing wrote:
> Draft 12 of the PEP.
>
> Fixed a bug in the expansion (didn't handle StopIteration raised by
> throw).
>
Just so you know, I now agree that a long expansion with multiple
"try...except StopIteration" blocks is the right thing to do. There
are only two cases I can see where it makes a difference compared to
what I suggested:
1. Throwing StopIteration to an iterator without a throw() method.
I would prefer treating this case *exactly* as if the iterator
had a trivial throw method:
def throw(self, et, ev=None, tb=None):
raise et, ev, tb
In other words, I think the StopIteration *should* be caught by
yield-from.
Treating it like this makes it easier to write wrappers that
don't accidentally change the semantics of an obscure corner
case. Principle of least surprise and all that...
It is easy enough to change the expansion to do this by expanding
the try block around the throw() call or by actually using such a
trivial throw method as a fallback. Alternatively, the expansion
can be rewritten using functools.partial as in the bottom of this
mail. It has identical semantics to draft 12 of the PEP, except
for the handling of the missing throw method. I actually like
that version because it is careful about what exceptions to
catch, but still only has one "try...except StopIteration". YMMV.
2. Calling an iterator.close() that raises a StopIteration.
Arguably, such a close() is an error, so getting an exception in
the caller is better than swallowing it and turning it into a
normal return. Especially since we only called close() as part
of handling GeneratorExit in the first place.
An unrelated question... What should happen with an iterator that has
a throw or close attribute that just happens to have the value None?
Should that be treated as an error because None is not callable, or
should it be treated as if the attribute wasn't there? The expansion
handles it as if the attribute wasn't there, but IIRC your patch will
raise a TypeError trying to call None.
> Removed paragraph about StopIteration left over from an earlier
> version.
>
> Added some discussion about rejected ideas.
>
Looks good, except...
Best regards
- Jacob
------------------------------------------------------------------------
Greg Ewing
> 1. Throwing StopIteration to an iterator without a throw() method.
Guido seems happy not to care what happens if you
throw StopIteration in, so I'm happy to do so as
well -- it saves considerable complication.
> 2. Calling an iterator.close() that raises a StopIteration.
> Arguably, such a close() is an error, so getting an exception in
> the caller is better than swallowing it
There are plenty of other ways to get strange
results by raising StopIteration in places where
you shouldn't, so I'm not worried about this either.
> What should happen with an iterator that has a
> throw or close attribute that just happens to have the value None?
> The expansion handles it as if the attribute wasn't there
That's a good point -- I hadn't intended that.
> * An implicit catch of GeneratorExit, followed by "return Value".
> By an "implicit catch", I mean either a catch of "BaseException"
> or a "finally" clause. In both cases, "return Value" will hide
> the original exception and that is almost certainly a bug.
Doing either of those things *anywhere* is likely to hide
a bug. I don't see a strong reason to single out this
particular case and try to detect it.
> I have now changed my mind. I think giving more power to
> experienced users is more important.
My reasoning is more along the lines that it's not worth
the bother of trying to detect this error, even if it's
an error at all, which isn't entirely certain.
--
Greg
Jacob Holm
> Jacob Holm wrote:
>
>> 1. Throwing StopIteration to an iterator without a throw() method.
>
> Guido seems happy not to care what happens if you
> throw StopIteration in, so I'm happy to do so as
> well -- it saves considerable complication.
I was about to reiterate how treating a missing throw like I suggested
would make my wrappers simpler, but realized that it actually makes very
little difference.
>
> > What should happen with an iterator that has a
>> throw or close attribute that just happens to have the value None?
> > The expansion handles it as if the attribute wasn't there
>
> That's a good point -- I hadn't intended that.
Ok.
Note that treating None as missing might actually be useful by making it
easier to "hide" the throw() or close() method of a base class from
yield-from. IIRC there is a precedent for treating None this way in the
handling of hash().
Even without handling None this way it is still possible to hide the
base class methods by creating a property that raises AttributeError, so
this is not all that important.
I just think that using None is slightly cleaner for this use.
Cheers
- Jacob
Greg Ewing
> Note that treating None as missing might actually be useful by making it
> easier to "hide" the throw() or close() method of a base class from
> yield-from. IIRC there is a precedent for treating None this way in the
> handling of hash().
In the case of hash() there's a specific reason for wanting
to be able to hide the base method. You'd have to justify
wanting to be able to do the same for send() and throw()
in particular.
--
Greg
Jacob Holm
> Jacob Holm wrote:
>
>> Note that treating None as missing might actually be useful by making
>> it easier to "hide" the throw() or close() method of a base class from
>> yield-from. IIRC there is a precedent for treating None this way in
>> the handling of hash().
>
> In the case of hash() there's a specific reason for wanting
> to be able to hide the base method. You'd have to justify
> wanting to be able to do the same for send() and throw()
> in particular.
>
I don't care much either way. Providing a property that raises
AttributeError is almost as easy as using None if you really need it.
- Jacob
Scott David Daniels
> Draft 12 of the PEP....
In the ensuing discussion, I replied to one of Jacob Holm's points
by making a comment that was really meant to apply to the PEP itself,
rather than his suggestion. I replied at the place where it became
obvious to me how I felt the PEP could become clearer. In an off-line
exchange, I found he thought I was talking simply about his suggested
change, so I'll restate my case here. I do so not for emphasis, but
rather to attach it to the proper context. This is, of course,
Greg's decision for readability. I share Jacob's fear of bike-shed
discussions on variable names (I am not necessarily in love with
the names I've chosen here).
Now that we passed the magic three or four threshold, is
it not easier to read if we pick some better names?
Instead of:
> _i = iter(EXPR)
> try:
> _y = next(_i)
> except StopIteration as _e:
> _r = _e.value
> else:
> while 1:
> try:
> _s = yield _y
> except GeneratorExit:
> _m = getattr(_i, 'close', None)
> if _m is not None:
> _m()
> raise
> except:
> _m = getattr(_i, 'throw', None)
> if _m is not None:
> try:
> _y = _m(*sys.exc_info())
> except StopIteration as _e:
> _r = _e.value
> break
> else:
> raise
> else:
> try:
> if _s is None:
> _y = next(_i)
> else:
> _y = _i.send(_s)
> except StopIteration as _e:
> _r = _e.value
> break
> RESULT = _r
we could use:
_iterator = iter(EXPR)
try:
_out = next(_iterator)
except StopIteration as _error:
_result = _error.value
else:
while 1:
try:
_inner = yield _out
except GeneratorExit:
_close = getattr(_i, 'close', None)
if _close is not None:
_close()
raise
except:
_throw = getattr(_iterator, 'throw', None)
if _throw is not None:
try:
_out = _throw(*sys.exc_info())
except StopIteration as _error:
_result = _error.value
break
else:
raise
else:
try:
if _inner is None:
_out = next(_iterator)
else:
_out = _i.send(_inner)
except StopIteration as _error:
_result = _error.value
break
RESULT = _result
--Scott David Daniels
Scott....@Acm.Org