Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

[Python-ideas] PEP-3151 pattern-matching

63 views
Skip to first unread message

Devin Jeanpierre

unread,
Apr 7, 2011, 2:59:12 AM4/7/11
to python...@python.org
Hello,

PEP-3151 < http://www.python.org/dev/peps/pep-3151/ > mentions a
really weird syntax for pattern-matching. I was hoping I could suggest
an alternative that's both more concise, and possible to implement
without doing something drastic like changing existing syntax or
semantics.

The PEP offers the pattern-matching syntax:

> except IOError as e if e.errno == errno.ENOENT: ...

I'd instead suggest something along the lines of

> except io_error(errno.ENOENT): ...

Implementing this is fairly straightforward, I've included it in the
bottom of this email. Raising an exception becomes `raise
io_error(errno.ENOENT)(msg)`. Some notational sugar can be implemented
(for example, `raise io_error(errno.ENOENT, msg)` could also be made
to work). I'm not fussed about the details.

I personally prefer keeping the errnos as a big part of handling
exceptions, they're common across OSes and involve less research /
memorization for those that are already aware of errno. I guess what
I'd like to see is the same exception hierarchy proposed by the PEP,
but with the addition of allowing errnos to be specified by
pattern-matching, so that errors not covered by the hierarchy, or more
specific than the hierarchy, can be concisely caught. However, I'm not
really well-versed in the pros and cons for all of this.

Above all, I'd like for the pattern matching alternative to be a bit
more reasonable. It doesn't have to be verbose and it doesn't have to
involve new syntax. Apologies for any mistakes in the code, they are
my own.

Here's the code:


# evil global state or something
error_classes = {}

def io_error(errno_, msg=None): # or something, you get the idea
try:
cls = error_classes[errno_]
except LookupError:
class IOErrorWithErrno(IOError):
errno = errno_

cls = error_classes[errno_] = IOErrorWithErrno

return error_classes[errno_]


# example of usage
import errno
try:
raise io_error(errno.ENOENT)("<Error message>")
except io_error(errno.ENOENT):
print("success")


Thanks for your time!
Devin Jeanpierre
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas

M.-A. Lemburg

unread,
Apr 7, 2011, 4:43:25 AM4/7/11
to Devin Jeanpierre, python...@python.org
Devin Jeanpierre wrote:
> Hello,
>
> PEP-3151 < http://www.python.org/dev/peps/pep-3151/ > mentions a
> really weird syntax for pattern-matching. I was hoping I could suggest
> an alternative that's both more concise, and possible to implement
> without doing something drastic like changing existing syntax or
> semantics.
>
> The PEP offers the pattern-matching syntax:
>
>> except IOError as e if e.errno == errno.ENOENT: ...
>
> I'd instead suggest something along the lines of
>
>> except io_error(errno.ENOENT): ...
>
> Implementing this is fairly straightforward, I've included it in the
> bottom of this email. Raising an exception becomes `raise
> io_error(errno.ENOENT)(msg)`. Some notational sugar can be implemented
> (for example, `raise io_error(errno.ENOENT, msg)` could also be made
> to work). I'm not fussed about the details.

The major difference between your proposal and the one in the
PEP is that in your case, an error object has to be created
or looked up via a function call regardless of whether the
case matches or not.

You typically don't want that to happen in tight loops and
except-clauses are designed to be cheap if they don't match.

> I personally prefer keeping the errnos as a big part of handling
> exceptions, they're common across OSes and involve less research /
> memorization for those that are already aware of errno. I guess what
> I'd like to see is the same exception hierarchy proposed by the PEP,
> but with the addition of allowing errnos to be specified by
> pattern-matching, so that errors not covered by the hierarchy, or more
> specific than the hierarchy, can be concisely caught. However, I'm not
> really well-versed in the pros and cons for all of this.

The main problem with the PEP is the proposal to flatten existing
exception class hierarchies, e.g. making socket.error the same
as IOError.

This introduces very subtle compatibility problems with code that
uses the flattened exception classes in separate except branches, e.g.

try:
...
except socket.error:
...
except IOError:
...

With the PEP implemented, the above code would never get
to execute the IOError branch and instead divert all error
handling to the socket.error branch which may well not be
aware of all the extra cases it now has to handle.

I think that this part of the PEP should not be accepted.

The addition of new IOError subclasses for common errno cases
is useful.

It would be even more useful, if there were a way to catch
standard IOErrors with those errnos using those same classes,
so that the following becomes possible:

try:
...
raise IOError(errno.EPERM, "permission denied")
except PermissionError:
...

and works as one would expect, that is, catch the EPERM error.

Without such support, you'd still have to write:

try:
...
raise IOError(errno.EPERM, "permission denied")
except PermissionError:
...EPERM handling code...
except IOError as error:
if error.errno == errno.EPERM:
...EPERM handling code...

duplicating the error handling code.

Perhaps the IOError constructor could be made to switch
the class of the generated object based on the errno
attribute passed to the constructor.

That way no new syntax would be necessary at all.

> Above all, I'd like for the pattern matching alternative to be a bit
> more reasonable. It doesn't have to be verbose and it doesn't have to
> involve new syntax. Apologies for any mistakes in the code, they are
> my own.
>
> Here's the code:
>
>
> # evil global state or something
> error_classes = {}
>
> def io_error(errno_, msg=None): # or something, you get the idea
> try:
> cls = error_classes[errno_]
> except LookupError:
> class IOErrorWithErrno(IOError):
> errno = errno_
>
> cls = error_classes[errno_] = IOErrorWithErrno
>
> return error_classes[errno_]
>
>
> # example of usage
> import errno
> try:
> raise io_error(errno.ENOENT)("<Error message>")
> except io_error(errno.ENOENT):
> print("success")
>
>
> Thanks for your time!
> Devin Jeanpierre

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Apr 07 2011)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

Antoine Pitrou

unread,
Apr 7, 2011, 5:17:16 AM4/7/11
to python...@python.org
On Thu, 07 Apr 2011 10:43:25 +0200
"M.-A. Lemburg" <m...@egenix.com> wrote:
>
> It would be even more useful, if there were a way to catch
> standard IOErrors with those errnos using those same classes,
> so that the following becomes possible:
>
> try:
> ...
> raise IOError(errno.EPERM, "permission denied")
> except PermissionError:
> ...
>
> and works as one would expect, that is, catch the EPERM error.

> Perhaps the IOError constructor could be made to switch


> the class of the generated object based on the errno
> attribute passed to the constructor.

Nice suggestion. I'll try to see if that is possible.

Regards

Antoine.

Nick Coghlan

unread,
Apr 7, 2011, 7:48:01 AM4/7/11
to Antoine Pitrou, python...@python.org
On Thu, Apr 7, 2011 at 7:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
>> Perhaps the IOError constructor could be made to switch
>> the class of the generated object based on the errno
>> attribute passed to the constructor.
>
> Nice suggestion. I'll try to see if that is possible.

It should be possible with appropriate fancy footwork in __new__. You
do need to be careful to avoid calling __init__ on the created object
twice.

I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

Michael Foord

unread,
Apr 7, 2011, 9:32:38 AM4/7/11
to Nick Coghlan, Antoine Pitrou, python...@python.org
On 7 April 2011 12:48, Nick Coghlan <ncog...@gmail.com> wrote:
On Thu, Apr 7, 2011 at 7:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
>> Perhaps the IOError constructor could be made to switch
>> the class of the generated object based on the errno
>> attribute passed to the constructor.
>
> Nice suggestion. I'll try to see if that is possible.

It should be possible with appropriate fancy footwork in __new__. You
do need to be careful to avoid calling __init__ on the created object
twice.

I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)


__init__ is called for you on construction so long as the object returned by __new__ is an instance of the type being constructed or a subclass.

So if what you want __new__ to return *is* a subclass, then you create it with subclass.__new__(...) and not subclass(...) (the latter would call __init__ which would then be called again after __new__ returns).

If what you're returning *isn't* a subclass (which is best avoided anyway) then you can construct it with otherclass(...) as __init__ won't be called for you.

All the best,

Michael Foord
 

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Michael Foord

unread,
Apr 7, 2011, 9:33:16 AM4/7/11
to Nick Coghlan, Antoine Pitrou, python...@python.org
On 7 April 2011 14:32, Michael Foord <fuzz...@gmail.com> wrote:


On 7 April 2011 12:48, Nick Coghlan <ncog...@gmail.com> wrote:
On Thu, Apr 7, 2011 at 7:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
>> Perhaps the IOError constructor could be made to switch
>> the class of the generated object based on the errno
>> attribute passed to the constructor.
>
> Nice suggestion. I'll try to see if that is possible.

It should be possible with appropriate fancy footwork in __new__. You
do need to be careful to avoid calling __init__ on the created object
twice.

I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)


__init__ is called for you on construction so long as the object returned by __new__ is an instance of the type being constructed or a subclass.

So if what you want __new__ to return *is* a subclass, then you create it with subclass.__new__(...) and not subclass(...) (the latter would call __init__ which would then be called again after __new__ returns).

If what you're returning *isn't* a subclass (which is best avoided anyway) then you can construct it with otherclass(...) as __init__ won't be called for you.


Those are the pure python rules anyway (which you were probably aware of), no idea if it is different in C. :-)

Michael
 
All the best,

Michael Foord
 

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas



--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

Michael Foord

unread,
Apr 7, 2011, 9:48:42 AM4/7/11
to Nick Coghlan, Antoine Pitrou, python...@python.org
On 7 April 2011 14:32, Michael Foord <fuzz...@gmail.com> wrote:


On 7 April 2011 12:48, Nick Coghlan <ncog...@gmail.com> wrote:
On Thu, Apr 7, 2011 at 7:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
>> Perhaps the IOError constructor could be made to switch
>> the class of the generated object based on the errno
>> attribute passed to the constructor.
>
> Nice suggestion. I'll try to see if that is possible.

It should be possible with appropriate fancy footwork in __new__. You
do need to be careful to avoid calling __init__ on the created object
twice.

I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)


__init__ is called for you on construction so long as the object returned by __new__ is an instance of the type being constructed or a subclass.

So if what you want __new__ to return *is* a subclass, then you create it with subclass.__new__(...)

Hmmm... that would rely on subclass.__new__ both existing *and* not calling up to its parent __new__ or you will have infinite recursion.

Probably what you have to do is call object.__new__(subclass, ...) and knowing / praying that subclass.__new__ doesn't do anything important...

Nick Coghlan

unread,
Apr 7, 2011, 10:36:39 AM4/7/11
to Michael Foord, Antoine Pitrou, python...@python.org
On Thu, Apr 7, 2011 at 11:48 PM, Michael Foord <fuzz...@gmail.com> wrote:
> Hmmm... that would rely on subclass.__new__ both existing *and* not calling
> up to its parent __new__ or you will have infinite recursion.
> Probably what you have to do is call object.__new__(subclass, ...) and
> knowing / praying that subclass.__new__ doesn't do anything important...

That's the dance I'm trying to remember. You make it work by playing
identity checking games with the cls argument, but it's been
absolutely ages since I read about it and experimented with it.

I think it's something like:

def __new__(cls, *args, **kwds):
if cls is ThisClass:
# Do fancy footwork to implicitly create an appropriate subclass instead
# via subclass.__new__
obj = cls._create_instance(*args, **kwds)
else:
# Don't do anything tricky for subclasses, that's their problem
obj = object.__new__(*args, **kwds)
return obj

Subclasses then have the option of passing the parent class as "cls"
if they want to invoke the fancy footwork, or themselves if they
don't.

Michael Foord

unread,
Apr 7, 2011, 11:53:23 AM4/7/11
to Nick Coghlan, Antoine Pitrou, python...@python.org
On 7 April 2011 15:36, Nick Coghlan <ncog...@gmail.com> wrote:
On Thu, Apr 7, 2011 at 11:48 PM, Michael Foord <fuzz...@gmail.com> wrote:
> Hmmm... that would rely on subclass.__new__ both existing *and* not calling
> up to its parent __new__ or you will have infinite recursion.
> Probably what you have to do is call object.__new__(subclass, ...) and
> knowing / praying that subclass.__new__ doesn't do anything important...

That's the dance I'm trying to remember. You make it work by playing
identity checking games with the cls argument, but it's been
absolutely ages since I read about it and experimented with it.

I think it's something like:

def __new__(cls, *args, **kwds):
 if cls is ThisClass:
   # Do fancy footwork to implicitly create an appropriate subclass instead
   # via subclass.__new__
   obj = cls._create_instance(*args, **kwds)
 else:
   # Don't do anything tricky for subclasses, that's their problem
   obj = object.__new__(*args, **kwds)
 return obj

Subclasses then have the option of passing the parent class as "cls"
if they want to invoke the fancy footwork, or themselves if they
don't.


Nice solution. You should write it up on your blog. It lets you call subclass.__new__, to return instances of subclasses, without having to worry about whether or not subclass.__new__ is going to upcall.

Michael
 
Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

Devin Jeanpierre

unread,
Apr 7, 2011, 1:44:10 PM4/7/11
to python...@python.org
On Thu, Apr 7, 2011 at 4:43 AM, M.-A. Lemburg <m...@egenix.com> wrote:
> You typically don't want that to happen in tight loops and
> except-clauses are designed to be cheap if they don't match.

Ah, well, you can always except IOError directly if performance is important.

Also, sorry, I left an error in my code. `msg=None` should be omitted.

Nonetheless it appears my idea has been discarded by now. Doesn't
matter too much, just wanted to get it off my chest. Thanks for the
time, everyone!

Devin Jeanpierre

Nick Coghlan

unread,
Apr 7, 2011, 7:28:31 PM4/7/11
to M.-A. Lemburg, python...@python.org
On Thu, Apr 7, 2011 at 6:43 PM, M.-A. Lemburg <m...@egenix.com> wrote:
> The main problem with the PEP is the proposal to flatten existing
> exception class hierarchies, e.g. making socket.error the same
> as IOError.
>
> This introduces very subtle compatibility problems with code that
> uses the flattened exception classes in separate except branches, e.g.
>
> try:
>  ...
> except socket.error:
>  ...
> except IOError:
>  ...
>
> With the PEP implemented, the above code would never get
> to execute the IOError branch and instead divert all error
> handling to the socket.error branch which may well not be
> aware of all the extra cases it now has to handle.
>
> I think that this part of the PEP should not be accepted.

I think EnvironmentError, WindowsError, VMSError, OSError, mmap.error
and select.error should definitely all be merged with IOError, as they
aren't used consistently enough to make handling them differently
reliable even in current code.

socket.error is the one case where that argument isn't quite as
strong, since the socket module is reasonably consistent about raising
it over a base IOError.

I think it would be useful to look at which errno values may currently
be associated with socket.error, and merge it with the new exception
at that level in the type heirarchy.

For example, taking the draft heirarchy at
http://www.python.org/dev/peps/pep-3151/#new-exception-classes, then
the merger that makes the most sense might be "socket.error =
ConnectionError".

Alternatively, and if necessary, socket.error could be grandfathered
in to cover an appropriate subset of errno values via multiple
inheritance: class error(ConnectionError, TimeoutError): pass

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

M.-A. Lemburg

unread,
Apr 8, 2011, 4:59:16 AM4/8/11
to Nick Coghlan, python...@python.org
Nick Coghlan wrote:
> On Thu, Apr 7, 2011 at 6:43 PM, M.-A. Lemburg <m...@egenix.com> wrote:
>> The main problem with the PEP is the proposal to flatten existing
>> exception class hierarchies, e.g. making socket.error the same
>> as IOError.
>>
>> This introduces very subtle compatibility problems with code that
>> uses the flattened exception classes in separate except branches, e.g.
>>
>> try:
>> ...
>> except socket.error:
>> ...
>> except IOError:
>> ...
>>
>> With the PEP implemented, the above code would never get
>> to execute the IOError branch and instead divert all error
>> handling to the socket.error branch which may well not be
>> aware of all the extra cases it now has to handle.
>>
>> I think that this part of the PEP should not be accepted.
>
> I think EnvironmentError, WindowsError, VMSError, OSError, mmap.error
> and select.error should definitely all be merged with IOError, as they
> aren't used consistently enough to make handling them differently
> reliable even in current code.

Their use may be inconsistent in a few places, but those cases
are still well-defined by the implementation, so code relying
on that well-defined behavior will break in subtle ways.

Example:

If code catches select.error (which is only raised for select.select()
and poll_object.poll() calls), because the try...except is expecting
a possible error from the select call used in the try...except
block, it will most likely do the wrong thing for some IOError
raised by a logging file I/O call in that same block.

Moreover, catching the exception at the wrong level, will
prevent the IOError from bubbling up the call chain and can
effectively mask potential errors at lower levels.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Apr 08 2011)


>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

Nick Coghlan

unread,
Apr 8, 2011, 5:27:22 AM4/8/11
to M.-A. Lemburg, python...@python.org
On Fri, Apr 8, 2011 at 6:59 PM, M.-A. Lemburg <m...@egenix.com> wrote:
> Their use may be inconsistent in a few places, but those cases
> are still well-defined by the implementation, so code relying
> on that well-defined behavior will break in subtle ways.

The phrases "defined by the implementation" and "well-defined" do not
belong in the same sentence.

> Example:
>
> If code catches select.error (which is only raised for select.select()
> and poll_object.poll() calls), because the try...except is expecting
> a possible error from the select call used in the try...except
> block, it will most likely do the wrong thing for some IOError
> raised by a logging file I/O call in that same block.

A deprecation period for the merged exceptions would be advisable.
That's an argument in favouring of figuring out how to implement
correct deprecation warnings for the merges, not an argument in favour
of not doing them.

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

M.-A. Lemburg

unread,
Apr 8, 2011, 5:53:42 AM4/8/11
to Nick Coghlan, python...@python.org
Nick Coghlan wrote:
> On Fri, Apr 8, 2011 at 6:59 PM, M.-A. Lemburg <m...@egenix.com> wrote:
>> Their use may be inconsistent in a few places, but those cases
>> are still well-defined by the implementation, so code relying
>> on that well-defined behavior will break in subtle ways.
>
> The phrases "defined by the implementation" and "well-defined" do not
> belong in the same sentence.

I agree that exceptions being raised by certain APIs
should probably be documented, but in the absence of such documentation,
the implementation is still the ultimate source of wisdom and
given that it's written down in C and not some legal code, I
think "well-defined" is an appropriate term ;-)

>> Example:
>>
>> If code catches select.error (which is only raised for select.select()
>> and poll_object.poll() calls), because the try...except is expecting
>> a possible error from the select call used in the try...except
>> block, it will most likely do the wrong thing for some IOError
>> raised by a logging file I/O call in that same block.
>
> A deprecation period for the merged exceptions would be advisable.
> That's an argument in favouring of figuring out how to implement
> correct deprecation warnings for the merges, not an argument in favour
> of not doing them.

Indeed, getting deprecations right is yet another aspect to consider,
but not the one the example was supposed to explain.

I don't think that such major changes in class hierarchy can be
implemented in a minor Python release.

Note that select.error currently does not inherit from IOError,
so "except IOError" won't catch select errors.

Renaming the exceptions would be something we could do for a
minor release and then have deprecations hint the user to
the naming change.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Apr 08 2011)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

Mike Graham

unread,
Apr 8, 2011, 10:04:50 AM4/8/11
to Devin Jeanpierre, Python-Ideas

I don't see how either solution is better than continuing to do what
we have right now. The PEP's idea introduces new syntax for a problem
that is currently solved in two lines. Your suggestion makes a new
pattern within the Python exception world, and further helps make the
exception hierarchy a little harder to understand again.

Neither of these seem justified for a rare case (this sort of patter
is fairly rare, most notably this one example) when there's nothing
that awful about the current solution.

Mike

Nick Coghlan

unread,
Apr 8, 2011, 10:13:41 AM4/8/11
to mikeg...@gmail.com, Python-Ideas
On Sat, Apr 9, 2011 at 12:04 AM, Mike Graham <mikeg...@gmail.com> wrote:
> Neither of these seem justified for a rare case (this sort of patter
> is fairly rare, most notably this one example) when there's nothing
> that awful about the current solution.

Actually, there is an awful lot of code in the wild that gives
incorrect and wildly misleading error messages *because* correctly
checking the errno attribute is so rare. PEP 3151 would improve the
quality of information provided by a lot of tracebacks and error
messages without many of those developers needing to even set finger
to keyboard.

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

Mike Graham

unread,
Apr 8, 2011, 10:31:41 AM4/8/11
to Nick Coghlan, Python-Ideas
On Fri, Apr 8, 2011 at 10:13 AM, Nick Coghlan <ncog...@gmail.com> wrote:
> On Sat, Apr 9, 2011 at 12:04 AM, Mike Graham <mikeg...@gmail.com> wrote:
>> Neither of these seem justified for a rare case (this sort of patter
>> is fairly rare, most notably this one example) when there's nothing
>> that awful about the current solution.
>
> Actually, there is an awful lot of code in the wild that gives
> incorrect and wildly misleading error messages *because* correctly
> checking the errno attribute is so rare. PEP 3151 would improve the
> quality of information provided by a lot of tracebacks and error
> messages without many of those developers needing to even set finger
> to keyboard.

But Nick, that's different from what I'm saying is rare.

I'm saying that the situation where we need an if->raise on a constant
parameter is rare (this being almost the only common case). The issue
of whether people handle that case being rare is separate. Neither the
"except Foo as e if f(e):" syntax nor the "except foo(some_errno):"
pattern propose something that stops people from doing stupid stuff
like "except IOError" with no check.

Mike

Guido van Rossum

unread,
Apr 8, 2011, 1:11:34 PM4/8/11
to python...@python.org
With apologies for not reading the PEP or this thread in full, some comments:

- I really like the syntax "except <exc> [as <var>] [if <test>]:".
This addresses a pretty common use case in my experience. I don't care
for the alternate proposed syntax that started this thread. I'm not
sure that the 'if' subclause makes sense without the 'as' subclause,
since most likely you'd want to refer to the caught exception. I note
that it is more powerful than putting "if not <test>: raise" in the
body of the except-clause, because it will allow subsequent except
clauses to match still. I also note that it is a much "cleaner" change
than (again) reorganizing the exception hierarchy, since there is no
backward compatibility to consider.

- Regarding restructuring the exception tree, I don't think this needs
to wait for Python 4. (We've done it several times during Python 2.)
But it is very tricky to do it without breaking existing code, and I'm
not sure that deprecations will help that much. E.g. if two exceptions
A and B are currently not related via inheritance, and we were to make
A a subclass of B, then in code of the form try: ... except B: ...
except A: ... the except A clause would become unreachable. Maybe it's
easier to propose small changes instead of trying to refactor a huge
section of the exception tree?

- Quite independently, I think it is time that we started documenting
which exceptions are raised from standard APIs. The trick is to avoid
overdocumenting: there is no point in documenting that everything can
raise MemoryError or that passing in wildly invalid arguments can
raise TypeError or AttributeError (and pretty much anything else). But
the variety of exceptions that may be raised by something like
urlopen, depending on which layer of the network and I/O stacks
detects a problem are worth documenting. Maybe this example could even
be a starting point for a rationalization of some of the exceptions
involved. (And if we find there are legitimate situations where a
networking or I/O problem raises a "generic" error such as TypeError
or AttributeError, those are probably bugs that should be fixed in the
code.)

- Lastly, there is a bunch of standard exception types that are
usually caused by bad code (as opposed to bad data or an environmental
problem). This would include NameError, TypeError, AttributeError, but
not KeyError, IndexError, ValueError. Perhaps it makes sense to
introduce a common base class for these "bad code" exceptions? Unlike
the other exception types, I think that the set of "bad code"
exceptions is pretty much static; a new library module is not going to
define raise a new kind of "bad code" exception. (But it may well
define a new kind of "bad data" exception, and I don't think we need a
common base class for all "bad data" or "bad state" exceptions.)

--
--Guido van Rossum (python.org/~guido)

Mike Graham

unread,
Apr 8, 2011, 1:52:18 PM4/8/11
to Guido van Rossum, Python-Ideas
On Fri, Apr 8, 2011 at 1:11 PM, Guido van Rossum <gu...@python.org> wrote:
> With apologies for not reading the PEP or this thread in full, some comments:
>
> - I really like the syntax "except <exc> [as <var>] [if <test>]:".
> This addresses a pretty common use case in my experience. I don't care
> for the alternate proposed syntax that started this thread. I'm not
> sure that the 'if' subclause makes sense without the 'as' subclause,
> since most likely you'd want to refer to the caught exception. I note
> that it is more powerful than putting "if not <test>: raise" in the
> body of the except-clause, because it will allow subsequent except
> clauses to match still. I also note that it is a much "cleaner" change
> than (again) reorganizing the exception hierarchy, since there is no
> backward compatibility to consider.

Of course, multiple cases can currently be handled by an if statement

except Foo as e:
if e.bar == baz:
...
elif e.bar == qux:
...
else:
raise

This code isn't so bad: it's not hard to understand, it's an existing
pattern, it uses general, powerful constructs.

I think the main opposition is "People aren't currently doing this!"
The problem is, introducing the "except...if" syntax doesn't actually
make anyone do it either. I understand that adding syntax can be seen
as a nudge, but I'm skeptical that it's a strong enough one to justify
the cost.

> - Quite independently, I think it is time that we started documenting
> which exceptions are raised from standard APIs. The trick is to avoid
> overdocumenting: there is no point in documenting that everything can
> raise MemoryError or that passing in wildly invalid arguments can
> raise TypeError or AttributeError (and pretty much anything else). But
> the variety of exceptions that may be raised by something like
> urlopen, depending on which layer of the network and I/O stacks
> detects a problem are worth documenting. Maybe this example could even
> be a starting point for a rationalization of some of the exceptions
> involved. (And if we find there are legitimate situations where a
> networking or I/O problem raises a "generic" error such as TypeError
> or AttributeError, those are probably bugs that should be fixed in the
> code.)

I couldn't agree more with this.


Mike

Guido van Rossum

unread,
Apr 8, 2011, 1:56:50 PM4/8/11
to mikeg...@gmail.com, Python-Ideas
On Fri, Apr 8, 2011 at 10:52 AM, Mike Graham <mikeg...@gmail.com> wrote:
> On Fri, Apr 8, 2011 at 1:11 PM, Guido van Rossum <gu...@python.org> wrote:
>> With apologies for not reading the PEP or this thread in full, some comments:
>>
>> - I really like the syntax "except <exc> [as <var>] [if <test>]:".
>> This addresses a pretty common use case in my experience. I don't care
>> for the alternate proposed syntax that started this thread. I'm not
>> sure that the 'if' subclause makes sense without the 'as' subclause,
>> since most likely you'd want to refer to the caught exception. I note
>> that it is more powerful than putting "if not <test>: raise" in the
>> body of the except-clause, because it will allow subsequent except
>> clauses to match still. I also note that it is a much "cleaner" change
>> than (again) reorganizing the exception hierarchy, since there is no
>> backward compatibility to consider.
>
> Of course, multiple cases can currently be handled by an if statement
>
> except Foo as e:
>    if e.bar == baz:
>        ...
>    elif e.bar == qux:
>        ...
>    else:
>        raise
>
> This code isn't so bad: it's not hard to understand, it's an existing
> pattern, it uses general, powerful constructs.

The problem (as I tried to say, but apparently not clearly enough) is
that if there's a later more general except clause on the same block
it won't work. E.g.

except Foo as e:
<code as above>
except Exception:
<log traceback>

The 'raise' in your code above does not transfer control to the
"except Exception:" clause. The current solution is either to have two
nested try/except blocks, or duplicate the logging code; neither is
great.

> I think the main opposition is "People aren't currently doing this!"

People aren't currently doing what? I have seen many examples like
that, and have myself encountered several times the problem I
explained just now.

> The problem is, introducing the "except...if" syntax doesn't actually
> make anyone do it either. I understand that adding syntax can be seen
> as a nudge, but I'm skeptical that it's a strong enough one to justify
> the cost.

What cost? Adding new optional syntax that doesn't introduce new
keywords is actually pretty cheap -- certainly cheaper than
restructuring the exception hierarchy.

--
--Guido van Rossum (python.org/~guido)

Antoine Pitrou

unread,
Apr 8, 2011, 2:18:07 PM4/8/11
to python...@python.org
On Fri, 8 Apr 2011 10:11:34 -0700

Guido van Rossum <gu...@python.org> wrote:
> With apologies for not reading the PEP or this thread in full, some comments:
>
> - I really like the syntax "except <exc> [as <var>] [if <test>]:".
> This addresses a pretty common use case in my experience. I don't care
> for the alternate proposed syntax that started this thread. I'm not
> sure that the 'if' subclause makes sense without the 'as' subclause,
> since most likely you'd want to refer to the caught exception. I note
> that it is more powerful than putting "if not <test>: raise" in the
> body of the except-clause, because it will allow subsequent except
> clauses to match still. I also note that it is a much "cleaner" change
> than (again) reorganizing the exception hierarchy, since there is no
> backward compatibility to consider.

My main issue with said new syntax is that it doesn't make things much
easier to write. You still have to type an explicit condition, and
remember the appropriate errno mnemonic for the situation. The very
notion of "errno" and its plethora of abbreviated error codes is ok for
people used to C programming, but certainly alien to other programmers
(how long does it take to remember that "file not found" is spelled
"ENOENT"?).

The fact that exception messages typically indicate the errno *number*,
not mnemonic, is an additional annoyance (errno numbers are not
standardized and can vary from system to system).

> - Regarding restructuring the exception tree, I don't think this needs
> to wait for Python 4. (We've done it several times during Python 2.)
> But it is very tricky to do it without breaking existing code, and I'm
> not sure that deprecations will help that much. E.g. if two exceptions
> A and B are currently not related via inheritance, and we were to make
> A a subclass of B, then in code of the form try: ... except B: ...
> except A: ... the except A clause would become unreachable. Maybe it's
> easier to propose small changes instead of trying to refactor a huge
> section of the exception tree?

Well, I fear that small changes will not lead us very far. We have to
level the existing tree before adding new types, because otherwise it's
not obvious where these new types should be grafted, since in practice
OSError, IOError or socket.error can be used for very similar error
conditions.

> - Quite independently, I think it is time that we started documenting
> which exceptions are raised from standard APIs.

Agreed, and unit-test them too.

> (But it may well
> define a new kind of "bad data" exception, and I don't think we need a
> common base class for all "bad data" or "bad state" exceptions.)

Isn't it ValueError actually? For example, closed files raise
ValueError when you try to do an operation on them.

Regards

Antoine.

geremy condra

unread,
Apr 8, 2011, 2:24:05 PM4/8/11
to Guido van Rossum, python...@python.org
On Fri, Apr 8, 2011 at 10:11 AM, Guido van Rossum <gu...@python.org> wrote:

<snip>

> - Quite independently, I think it is time that we started documenting
> which exceptions are raised from standard APIs. The trick is to avoid
> overdocumenting: there is no point in documenting that everything can
> raise MemoryError or that passing in wildly invalid arguments can
> raise TypeError or AttributeError (and pretty much anything else). But
> the variety of exceptions that may be raised by something like
> urlopen, depending on which layer of the network and I/O stacks
> detects a problem are worth documenting. Maybe this example could even
> be a starting point for a rationalization of some of the exceptions
> involved. (And if we find there are legitimate situations where a
> networking or I/O problem raises a "generic" error such as TypeError
> or AttributeError, those are probably bugs that should be fixed in the
> code.)

This would be very nice.

> - Lastly, there is a bunch of standard exception types that are
> usually caused by bad code (as opposed to bad data or an environmental
> problem). This would include NameError, TypeError, AttributeError, but
> not KeyError, IndexError, ValueError. Perhaps it makes sense to
> introduce a common base class for these "bad code" exceptions? Unlike
> the other exception types, I think that the set of "bad code"
> exceptions is pretty much static; a new library module is not going to
> define raise a new kind of "bad code" exception. (But it may well
> define a new kind of "bad data" exception, and I don't think we need a
> common base class for all "bad data" or "bad state" exceptions.)

I actually like both the BadCode exception and the BadData exception.

Geremy Condra

Devin Jeanpierre

unread,
Apr 8, 2011, 3:00:50 PM4/8/11
to Antoine Pitrou, python...@python.org
On Fri, Apr 8, 2011 at 2:18 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
> On Fri, 8 Apr 2011 10:11:34 -0700
> Guido van Rossum <gu...@python.org> wrote:
>> With apologies for not reading the PEP or this thread in full, some comments:
>>
>> - I really like the syntax "except <exc> [as <var>] [if <test>]:".
>> This addresses a pretty common use case in my experience. I don't care
>> for the alternate proposed syntax that started this thread. I'm not
>> sure that the 'if' subclause makes sense without the 'as' subclause,
>> since most likely you'd want to refer to the caught exception. I note
>> that it is more powerful than putting "if not <test>: raise" in the
>> body of the except-clause, because it will allow subsequent except
>> clauses to match still. I also note that it is a much "cleaner" change
>> than (again) reorganizing the exception hierarchy, since there is no
>> backward compatibility to consider.
>
> My main issue with said new syntax is that it doesn't make things much
> easier to write. You still have to type an explicit condition, and
> remember the appropriate errno mnemonic for the situation. The very
> notion of "errno" and its plethora of abbreviated error codes is ok for
> people used to C programming, but certainly alien to other programmers
> (how long does it take to remember that "file not found" is spelled
> "ENOENT"?).
>
> The fact that exception messages typically indicate the errno *number*,
> not mnemonic, is an additional annoyance (errno numbers are not
> standardized and can vary from system to system).

It wouldn't be that difficult to customize the IOError and OSError
reprs to display the errno mnemonic instead of a number, if that's an
issue.

I don't think it's fair to characterize errnos as only for C
programmers. Some other language use errno mnemonics too, not just C
and Python. It's a cross-language way of understanding certain kinds
of error conditions, and as such it makes it a bit easier to do
things. What this PEP does is introduce a language-specific exception
hierarchy instead. I think it's a good idea, but it's not as clearly
superior to errnos as you suggest. It's easier to remember the
language-specific exception hierarchy if you don't have to deal with
other languages that use errnos, but if you do, the situation is
different.

Antoine Pitrou

unread,
Apr 8, 2011, 3:06:50 PM4/8/11
to python...@python.org
On Fri, 8 Apr 2011 15:00:50 -0400
Devin Jeanpierre <jeanpi...@gmail.com>
wrote:

> What this PEP does is introduce a language-specific exception
> hierarchy instead. I think it's a good idea, but it's not as clearly
> superior to errnos as you suggest. It's easier to remember the
> language-specific exception hierarchy if you don't have to deal with
> other languages that use errnos, but if you do, the situation is
> different.

True, but that's like saying we should use sprintf() for string
formatting and fgets() for file access since other languages such as C
and PHP use them. In trying to find elegant solutions to problems, we
often diverge from solutions common in other languages.

(actually, if we followed the errno paradigm closely, we shouldn't even
raise exceptions, but return the error code or store it in a
thread-local variable instead ;-))

Devin Jeanpierre

unread,
Apr 8, 2011, 5:14:23 PM4/8/11
to python...@python.org
On Fri, Apr 8, 2011 at 3:06 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
> True, but that's like saying we should use sprintf() for string
> formatting and fgets() for file access since other languages such as C
> and PHP use them. In trying to find elegant solutions to problems, we
> often diverge from solutions common in other languages.
>
> (actually, if we followed the errno paradigm closely, we shouldn't even
> raise exceptions, but return the error code or store it in a
> thread-local variable instead ;-))
>
> Regards
>
> Antoine.

Well, not quite. The only case that I can see where errnos are
obviously bad is with multiple errnos that mean the same thing in
practice (so much so that they might even have the same value), making
proper IO handling inconvenient and non-obvious. In particular I'm
thinking of errors like EAGAIN and EWOULDBLOCK. The PEP's suggested
exception hierarchy, and its collapse of some errnos into the same
exception, is nice on that front (for example, BlockingIOError).
However, I interpret what you say as a complaint that errnos are
obscure and thus we should avoid making programmers memorize them, and
avoid encouraging their use. I don't agree with that. They aren't
obscure, they're a legitimate part of POSIX and fairly well-known.
Beginners to IO will have to memorize something anyway, and it may as
well be errnos. A more experienced programmer might come from from C
or C# or Ruby, or whatever, and would get less educational overhead
when doing Python IO, because all these languages share a little bit
of POSIX in their IO APIs.

It's not even about following it closely, but about building on what
people know. Python doesn't support sprintf, but it does (or perhaps
"did") support the sprintf syntax. It didn't follow it *closely* --
"%s" % 2 doesn't result in a segfault or anything -- but closely
enough so that programmers familiar with sprintf could get started
quickly, and programmers familiar with Python could understand
sprintf. The replacement, str.format, doesn't come from nowhere
either, but builds on a different string formatting tradition also
shared by C#. Having these ties to the outside world is useful and
desirable, it makes everyone's life easier. I think that breaking
these ties needs an overriding reason, generally relating to the
outside world actually hurting people. The PEP solves the issue where
people don't necessarily handle all the right errnos, without breaking
ties to the IO community and eliminating them altogether. That's
pretty cool. It also offers a way to make errno handling easier.
That's also pretty cool.

Devin

Guido van Rossum

unread,
Apr 8, 2011, 5:50:02 PM4/8/11
to Antoine Pitrou, python...@python.org
On Fri, Apr 8, 2011 at 11:18 AM, Antoine Pitrou <soli...@pitrou.net> wrote:
> On Fri, 8 Apr 2011 10:11:34 -0700
> Guido van Rossum <gu...@python.org> wrote:
>> With apologies for not reading the PEP or this thread in full, some comments:
>>
>> - I really like the syntax "except <exc> [as <var>] [if <test>]:".
>> This addresses a pretty common use case in my experience. I don't care
>> for the alternate proposed syntax that started this thread. I'm not
>> sure that the 'if' subclause makes sense without the 'as' subclause,
>> since most likely you'd want to refer to the caught exception. I note
>> that it is more powerful than putting "if not <test>: raise" in the
>> body of the except-clause, because it will allow subsequent except
>> clauses to match still. I also note that it is a much "cleaner" change
>> than (again) reorganizing the exception hierarchy, since there is no
>> backward compatibility to consider.
>
> My main issue with said new syntax is that it doesn't make things much
> easier to write.

As I explained in other messages, it also adds semantics that are not
so easily emulated with the existing syntax (you'd have to repeat code
or use nested try/except blocks).

> You still have to type an explicit condition, and
> remember the appropriate errno mnemonic for the situation. The very
> notion of "errno" and its plethora of abbreviated error codes is ok for
> people used to C programming, but certainly alien to other programmers
> (how long does it take to remember that "file not found" is spelled
> "ENOENT"?).

Well, this is a fact of life. There are hundreds of errno codes and
they vary across systems, both in names and in numbers (even if there
is a handful that exist nearly everywhere and almost always have the
same values). This is the nature of system calls, and some operating
systems just have a richer set of error conditions that they present
to the (advanced) programmer.

If you are careful you can probably come up with a few handfuls of
common error conditions for which it makes sense to introduce new
exceptions that are defined (and presumably raised) on all platforms.
"File not found" is one of those. But there are always error
conditions that are not in this list, and we would still need the
errno attribute and the errno module to map between numbers, names and
messages in the general case. And occasionally some programmer needs
to select behaviors depending on the precise errno value.

> The fact that exception messages typically indicate the errno *number*,
> not mnemonic, is an additional annoyance (errno numbers are not
> standardized and can vary from system to system).

That would seem to be a problem with the str() and/or repr() of
OSError, and perhaps we can fix it there.

>> - Regarding restructuring the exception tree, I don't think this needs
>> to wait for Python 4. (We've done it several times during Python 2.)
>> But it is very tricky to do it without breaking existing code, and I'm
>> not sure that deprecations will help that much. E.g. if two exceptions
>> A and B are currently not related via inheritance, and we were to make
>> A a subclass of B, then in code of the form try: ... except B: ...
>> except A: ... the except A clause would become unreachable. Maybe it's
>> easier to propose small changes instead of trying to refactor a huge
>> section of the exception tree?
>
> Well, I fear that small changes will not lead us very far. We have to
> level the existing tree before adding new types, because otherwise it's
> not obvious where these new types should be grafted, since in practice
> OSError, IOError or socket.error can be used for very similar error
> conditions.

Alas, you are right, they are all essentially the same thing but
raised by different operations (sometimes it's even the same
underlying system call, e.g. os.write vs. stream.write).

>> - Quite independently, I think it is time that we started documenting
>> which exceptions are raised from standard APIs.
>
> Agreed, and unit-test them too.
>
>> (But it may well
>> define a new kind of "bad data" exception, and I don't think we need a
>> common base class for all "bad data" or "bad state" exceptions.)
>
> Isn't it ValueError actually? For example, closed files raise
> ValueError when you try to do an operation on them.

No, there are lots of other exceptions indicating "bad arguments" or
"bad data" or "bad state" that don't derive from ValueError, from
KeyError to binhex.Error (a random example I picked from the stdlib).

--
--Guido van Rossum (python.org/~guido)

Antoine Pitrou

unread,
Apr 8, 2011, 6:39:50 PM4/8/11
to python...@python.org
On Fri, 8 Apr 2011 14:50:02 -0700
Guido van Rossum <gu...@python.org> wrote:
>
> > You still have to type an explicit condition, and
> > remember the appropriate errno mnemonic for the situation. The very
> > notion of "errno" and its plethora of abbreviated error codes is ok for
> > people used to C programming, but certainly alien to other programmers
> > (how long does it take to remember that "file not found" is spelled
> > "ENOENT"?).
>
> Well, this is a fact of life. There are hundreds of errno codes and
> they vary across systems, both in names and in numbers (even if there
> is a handful that exist nearly everywhere and almost always have the
> same values). This is the nature of system calls, and some operating
> systems just have a richer set of error conditions that they present
> to the (advanced) programmer.

Well, errnos are only the POSIX API side of system calls.
For example, a seasoned Windows C programmer might not have faced a
single EBADF or ENOENT in their entire career, since the Win32 API
doesn't use those.
Similarly, java.io.IOException doesn't seem to have any errno-alike
member, but it does have subclasses named e.g. FileNotFoundException.
(http://download.oracle.com/javase/1.5.0/docs/api/java/io/IOException.html)
Looking at .Net, System.IO.IOException does have a numerical "HResult"
member, but it doesn't seem related to POSIX errnos (“HRESULT is a
32-bit value, divided into three different fields: a severity code, a
facility code, and an error code”). It also has a couple of subclasses
such as FileNotFoundException.
(http://msdn.microsoft.com/en-us/library/system.io.ioexception.aspx)

> > The fact that exception messages typically indicate the errno *number*,
> > not mnemonic, is an additional annoyance (errno numbers are not
> > standardized and can vary from system to system).
>
> That would seem to be a problem with the str() and/or repr() of
> OSError, and perhaps we can fix it there.

True.

Regards

Antoine.

Eric Snow

unread,
Apr 8, 2011, 6:56:54 PM4/8/11
to Devin Jeanpierre, python...@python.org
On Fri, Apr 8, 2011 at 3:14 PM, Devin Jeanpierre <jeanpi...@gmail.com> wrote:
On Fri, Apr 8, 2011 at 3:06 PM, Antoine Pitrou <soli...@pitrou.net> wrote:
> True, but that's like saying we should use sprintf() for string
> formatting and fgets() for file access since other languages such as C
> and PHP use them. In trying to find elegant solutions to problems, we
> often diverge from solutions common in other languages.
>
> (actually, if we followed the errno paradigm closely, we shouldn't even
> raise exceptions, but return the error code or store it in a
> thread-local variable instead ;-))
>
> Regards
>
> Antoine.

Well, not quite. The only case that I can see where errnos are
obviously bad is with multiple errnos that mean the same thing in
practice (so much so that they might even have the same value), making
proper IO handling inconvenient and non-obvious. In particular I'm
thinking of errors like EAGAIN and EWOULDBLOCK. The PEP's suggested
exception hierarchy, and its collapse of some errnos into the same
exception, is nice on that front (for example, BlockingIOError).
However, I interpret what you say as a complaint that errnos are
obscure and thus we should avoid making programmers memorize them, and
avoid encouraging their use. I don't agree with that. They aren't
obscure, they're a legitimate part of POSIX and fairly well-known.
Beginners to IO will have to memorize something anyway, and it may as
well be errnos.

I see your point about that value of knowing stuff, but why should I need to learn errnos until I need them?  I deal with IO errors on occasion and end up having to look it up each time.  That's a pain.  

Consider how much IO is used in Python and how often people have to deal with IO exceptions.  Why should we have to deal with errnos if there is a way that fits into the exception hierarchy?  It's not that they are obscure.  It's that we don't need the actual numbers or mnemonics  in our Python code.  We can hide them behind more meaningful exceptions.  At least, that's my take.

FYI, the the exception objects will still have the errno attribute that you would find in the plain IOError:


-eric

Guido van Rossum

unread,
Apr 8, 2011, 6:59:16 PM4/8/11
to Eric Snow, python...@python.org
On Fri, Apr 8, 2011 at 3:56 PM, Eric Snow <ericsnow...@gmail.com> wrote:
> Consider how much IO is used in Python and how often people have to deal
> with IO exceptions.  Why should we have to deal with errnos if there is a
> way that fits into the exception hierarchy?  It's not that they are obscure.
>  It's that we don't need the actual numbers or mnemonics  in our Python
> code.  We can hide them behind more meaningful exceptions.  At least, that's
> my take.

You can use more useful exceptions for *some* errnos. But not for all.

--
--Guido van Rossum (python.org/~guido)

MRAB

unread,
Apr 8, 2011, 7:29:45 PM4/8/11
to python-ideas
[snip]
Unless you have something like "continue except", which means "continue
checking the following 'except' clauses".

Nick Coghlan

unread,
Apr 8, 2011, 7:50:04 PM4/8/11
to mikeg...@gmail.com, Python-Ideas
On Sat, Apr 9, 2011 at 12:31 AM, Mike Graham <mikeg...@gmail.com> wrote:
> On Fri, Apr 8, 2011 at 10:13 AM, Nick Coghlan <ncog...@gmail.com> wrote:
>> On Sat, Apr 9, 2011 at 12:04 AM, Mike Graham <mikeg...@gmail.com> wrote:
>>> Neither of these seem justified for a rare case (this sort of patter
>>> is fairly rare, most notably this one example) when there's nothing
>>> that awful about the current solution.
>>
>> Actually, there is an awful lot of code in the wild that gives
>> incorrect and wildly misleading error messages *because* correctly
>> checking the errno attribute is so rare. PEP 3151 would improve the
>> quality of information provided by a lot of tracebacks and error
>> messages without many of those developers needing to even set finger
>> to keyboard.
>
> But Nick, that's different from what I'm saying is rare.
>
> I'm saying that the situation where we need an if->raise on a constant
> parameter is rare (this being almost the only common case). The issue
> of whether people handle that case being rare is separate. Neither the
> "except Foo as e if f(e):" syntax nor the "except foo(some_errno):"
> pattern propose something that stops people from doing stupid stuff
> like "except IOError" with no check.

Sorry, I misinterpreted "neither of these" as including PEP 3151
itself (which aims to eliminate the need for most errno pattern
matching). I had missed that someone had brought up a second
pattern-matching idea, and it was the two different approaches that
you were referring to.

Cheers,
Nick.

--
Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

Eric Smith

unread,
Apr 24, 2011, 6:15:33 AM4/24/11
to python...@python.org
> Guido van Rossum <guido at python.org> wrote:
>>On Fri, Apr 8, 2011 at 11:18 AM, Antoine Pitrou <solipsis at

pitrou.net> wrote:
>> On Fri, 8 Apr 2011 10:11:34 -0700
>> Guido van Rossum <guido at python.org> wrote:
>>> With apologies for not reading the PEP or this thread in full, some
comments:
>>>
>>> - I really like the syntax "except <exc> [as <var>] [if <test>]:".
>>> This addresses a pretty common use case in my experience. I don't care
>>> for the alternate proposed syntax that started this thread. I'm not
>>> sure that the 'if' subclause makes sense without the 'as' subclause,
>>> since most likely you'd want to refer to the caught exception. I note
>>> that it is more powerful than putting "if not <test>: raise" in the
>>> body of the except-clause, because it will allow subsequent except
>>> clauses to match still. I also note that it is a much "cleaner" change
>>> than (again) reorganizing the exception hierarchy, since there is no
>>> backward compatibility to consider.
>>
>> My main issue with said new syntax is that it doesn't make things much
>> easier to write.

>As I explained in other messages, it also adds semantics that are not
>so easily emulated with the existing syntax (you'd have to repeat code
>or use nested try/except blocks).

Interestingly, this is one of the few (only?) .NET features that is
exposed in Visual Basic but not in C#. Maybe there's something to learn
from that?

http://blogs.msdn.com/b/jaredpar/archive/2008/10/09/vb-catch-when-why-so-special.aspx

Eric.

Guido van Rossum

unread,
Apr 24, 2011, 6:13:09 PM4/24/11
to Eric Smith, python...@python.org

FWIW I still am at least +0.5 on the idea. I do note that currently
this is lecal syntax:

try: ...
except stuff if cond else otherstuff: ...

so allowing the syntax

except <exception> [as <variable>] [if <condition>]:

could break some code; however

except <exception> [as <variable> [if <condition>]]:

could not.

Anyone want to write a PEP (could be pretty short & sweet, the VB
reference would be useful) and a reference implementation?

--
--Guido van Rossum (python.org/~guido)

Antoine Pitrou

unread,
May 12, 2011, 1:26:45 PM5/12/11
to python...@python.org
On Fri, 08 Apr 2011 10:59:16 +0200
"M.-A. Lemburg" <m...@egenix.com> wrote:
> >
> > I think EnvironmentError, WindowsError, VMSError, OSError, mmap.error
> > and select.error should definitely all be merged with IOError, as they
> > aren't used consistently enough to make handling them differently
> > reliable even in current code.
>
> Their use may be inconsistent in a few places, but those cases
> are still well-defined by the implementation, so code relying
> on that well-defined behavior will break in subtle ways.

Another quirk occurred to me today: select.error doesn't derive from
EnvironmentError, and so it doesn't have the errno attribute (even
though the select module "correctly" instantiates it with a (errno,
message) tuple). Also, its str() is borked:

>>> e = select.error(4, "interrupted")
>>> str(e)
"(4, 'interrupted')"
>>> raise e
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
select.error: (4, 'interrupted')

Regards

Antoine.

M.-A. Lemburg

unread,
May 12, 2011, 3:06:18 PM5/12/11
to Antoine Pitrou, python...@python.org
Antoine Pitrou wrote:
> On Fri, 08 Apr 2011 10:59:16 +0200
> "M.-A. Lemburg" <m...@egenix.com> wrote:
>>>
>>> I think EnvironmentError, WindowsError, VMSError, OSError, mmap.error
>>> and select.error should definitely all be merged with IOError, as they
>>> aren't used consistently enough to make handling them differently
>>> reliable even in current code.
>>
>> Their use may be inconsistent in a few places, but those cases
>> are still well-defined by the implementation, so code relying
>> on that well-defined behavior will break in subtle ways.
>
> Another quirk occurred to me today: select.error doesn't derive from
> EnvironmentError, and so it doesn't have the errno attribute (even
> though the select module "correctly" instantiates it with a (errno,
> message) tuple). Also, its str() is borked:
>
>>>> e = select.error(4, "interrupted")
>>>> str(e)
> "(4, 'interrupted')"
>>>> raise e
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> select.error: (4, 'interrupted')

Works fine in Python 2.7:

>>> import select
>>> e = select.error(4, "intr")
>>> e
error(4, 'intr')
>>> try:
... raise e
... except select.error, x:
... code, text = x
... print code,text
...
4 intr
>>>

Note that existing code will not look for an attribute that
doesn't exist :-) It'll unwrap the tuple and work from there
or use the .args attribute to get at the constructor args.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, May 12 2011)


>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________

2011-06-20: EuroPython 2011, Florence, Italy 39 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

Antoine Pitrou

unread,
May 12, 2011, 3:22:11 PM5/12/11
to python...@python.org
Le jeudi 12 mai 2011 à 21:06 +0200, M.-A. Lemburg a écrit :
> Note that existing code will not look for an attribute that
> doesn't exist :-)

True. My point is that not having "errno" makes it even more obscure how
to check for different kinds of select errors. Also, given that other
"environmental" errors will have an "errno" giving the POSIX error code,
it's easy to get surprised.

Regards

Antoine.

Georg Brandl

unread,
May 12, 2011, 4:15:07 PM5/12/11
to python...@python.org

Note that this is the repr(), while Antoine showed the str().

But the str() looks correct to me as well (for an exception that doesn't
derive from EnvironmentError).

Georg

Antoine Pitrou

unread,
May 12, 2011, 5:17:29 PM5/12/11
to python...@python.org
On Thu, 12 May 2011 22:15:07 +0200
Georg Brandl <g.br...@gmx.net> wrote:
>
> But the str() looks correct to me as well (for an exception that doesn't
> derive from EnvironmentError).

It's technically correct, sure. The point is that "technically correct"
translates to "humanly bogus" here, because of the broken I/O
exception hierarchy.

Regards

Antoine.

Georg Brandl

unread,
May 13, 2011, 1:07:56 AM5/13/11
to python...@python.org
On 12.05.2011 23:17, Antoine Pitrou wrote:
> On Thu, 12 May 2011 22:15:07 +0200
> Georg Brandl <g.br...@gmx.net> wrote:
>>
>> But the str() looks correct to me as well (for an exception that doesn't
>> derive from EnvironmentError).
>
> It's technically correct, sure. The point is that "technically correct"
> translates to "humanly bogus" here, because of the broken I/O
> exception hierarchy.

Yep, and I'm all for fixing it with PEP 3151 :)

Georg

dag.od...@gmail.com

unread,
May 17, 2011, 10:14:34 AM5/17/11
to Georg Brandl, python...@python.org
Excuse me if this has already been discussed, but couldn't
__instancecheck__ be used to add exception types that match with more
precision?

Chris Rebert

unread,
May 17, 2011, 6:41:40 PM5/17/11
to dag.od...@gmail.com, Georg Brandl, python...@python.org
On Tue, May 17, 2011 at 7:14 AM, dag.od...@gmail.com
<dag.od...@gmail.com> wrote:
> Excuse me if this has already been discussed, but couldn't
> __instancecheck__ be used to add exception types that match with more
> precision?

Somewhat related bug:
http://bugs.python.org/issue12029

Cheers,
Chris

dag.od...@gmail.com

unread,
May 18, 2011, 6:07:28 AM5/18/11
to Chris Rebert, Georg Brandl, python...@python.org
On 18 May 2011 00:41, Chris Rebert <pyi...@rebertia.com> wrote:
> On Tue, May 17, 2011 at 7:14 AM, dag.od...@gmail.com
> <dag.od...@gmail.com> wrote:
>> Excuse me if this has already been discussed, but couldn't
>> __instancecheck__ be used to add exception types that match with more
>> precision?
>
> Somewhat related bug:
> http://bugs.python.org/issue12029

Interesting. If that is intentional I'd advocate against it unless
there's a strong argument for it.

Another idea (also likely already proposed) would be to match against
instances as well, by the 'args' attribute:

try:
...
except IOError(32): # isinstance IOError and .args == (32,)
...

If this seems crazy consider that it's (to some extent) similar to the
behavior of 'raise'.

Devin Jeanpierre

unread,
May 18, 2011, 8:24:04 AM5/18/11
to dag.od...@gmail.com, Georg Brandl, python...@python.org
On Wed, May 18, 2011 at 6:07 AM, dag.od...@gmail.com
<dag.od...@gmail.com> wrote:
> Interesting. If that is intentional I'd advocate against it unless
> there's a strong argument for it.
>
> Another idea (also likely already proposed) would be to match against
> instances as well, by the 'args' attribute:
>
> try:
>    ...
> except IOError(32):  # isinstance IOError and .args == (32,)
>    ...
>
> If this seems crazy consider that it's (to some extent) similar to the
> behavior of 'raise'.

Unfortunately, as described it wouldn't match IOError(32, 'Blah blah
blah'). Although maybe it makes sense to create an Anything builtin,
which is equal to everything, such that IOError(X, Y) == IOError(X,
Anything) == IOError(X, Z) for all X, Y, and Z (except stupid X like X
= nan).

I do like it.

Devin Jeanpierre

Greg Ewing

unread,
May 18, 2011, 6:13:09 PM5/18/11
to python...@python.org
Devin Jeanpierre wrote:
> On Wed, May 18, 2011 at 6:07 AM, dag.od...@gmail.com
> <dag.od...@gmail.com> wrote:
>
>>except IOError(32): # isinstance IOError and .args == (32,)
>> ...
>
> Unfortunately, as described it wouldn't match IOError(32, 'Blah blah
> blah').

Also it's a bit magical -- normally one doesn't expect
Haskell-like pattern matching in Python.

Maybe something more explicit would be better:

try:
...
except IOError as e with e.errno == 32:
...

--
Greg

dag.od...@gmail.com

unread,
May 19, 2011, 4:33:11 AM5/19/11
to Greg Ewing, python...@python.org
On 19 May 2011 00:13, Greg Ewing <greg....@canterbury.ac.nz> wrote:
> Devin Jeanpierre wrote:
>>
>> On Wed, May 18, 2011 at 6:07 AM, dag.od...@gmail.com
>> <dag.od...@gmail.com> wrote:
>>
>>> except IOError(32):  # isinstance IOError and .args == (32,)
>>>  ...
>>
>> Unfortunately, as described it wouldn't match IOError(32, 'Blah blah
>> blah').
>
> Also it's a bit magical -- normally one doesn't expect
> Haskell-like pattern matching in Python.
>
> Maybe something more explicit would be better:
>
>  try:
>    ...
>  except IOError as e with e.errno == 32:
>    ...

Then we're back where we started in which case I prefer 'if' as the keyword.

Reply all
Reply to author
Forward
0 new messages