PEP-3151 < http://www.python.org/dev/peps/pep-3151/ > mentions a
really weird syntax for pattern-matching. I was hoping I could suggest
an alternative that's both more concise, and possible to implement
without doing something drastic like changing existing syntax or
semantics.
The PEP offers the pattern-matching syntax:
> except IOError as e if e.errno == errno.ENOENT: ...
I'd instead suggest something along the lines of
> except io_error(errno.ENOENT): ...
Implementing this is fairly straightforward, I've included it in the
bottom of this email. Raising an exception becomes `raise
io_error(errno.ENOENT)(msg)`. Some notational sugar can be implemented
(for example, `raise io_error(errno.ENOENT, msg)` could also be made
to work). I'm not fussed about the details.
I personally prefer keeping the errnos as a big part of handling
exceptions, they're common across OSes and involve less research /
memorization for those that are already aware of errno. I guess what
I'd like to see is the same exception hierarchy proposed by the PEP,
but with the addition of allowing errnos to be specified by
pattern-matching, so that errors not covered by the hierarchy, or more
specific than the hierarchy, can be concisely caught. However, I'm not
really well-versed in the pros and cons for all of this.
Above all, I'd like for the pattern matching alternative to be a bit
more reasonable. It doesn't have to be verbose and it doesn't have to
involve new syntax. Apologies for any mistakes in the code, they are
my own.
Here's the code:
# evil global state or something
error_classes = {}
def io_error(errno_, msg=None): # or something, you get the idea
try:
cls = error_classes[errno_]
except LookupError:
class IOErrorWithErrno(IOError):
errno = errno_
cls = error_classes[errno_] = IOErrorWithErrno
return error_classes[errno_]
# example of usage
import errno
try:
raise io_error(errno.ENOENT)("<Error message>")
except io_error(errno.ENOENT):
print("success")
Thanks for your time!
Devin Jeanpierre
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas
The major difference between your proposal and the one in the
PEP is that in your case, an error object has to be created
or looked up via a function call regardless of whether the
case matches or not.
You typically don't want that to happen in tight loops and
except-clauses are designed to be cheap if they don't match.
> I personally prefer keeping the errnos as a big part of handling
> exceptions, they're common across OSes and involve less research /
> memorization for those that are already aware of errno. I guess what
> I'd like to see is the same exception hierarchy proposed by the PEP,
> but with the addition of allowing errnos to be specified by
> pattern-matching, so that errors not covered by the hierarchy, or more
> specific than the hierarchy, can be concisely caught. However, I'm not
> really well-versed in the pros and cons for all of this.
The main problem with the PEP is the proposal to flatten existing
exception class hierarchies, e.g. making socket.error the same
as IOError.
This introduces very subtle compatibility problems with code that
uses the flattened exception classes in separate except branches, e.g.
try:
...
except socket.error:
...
except IOError:
...
With the PEP implemented, the above code would never get
to execute the IOError branch and instead divert all error
handling to the socket.error branch which may well not be
aware of all the extra cases it now has to handle.
I think that this part of the PEP should not be accepted.
The addition of new IOError subclasses for common errno cases
is useful.
It would be even more useful, if there were a way to catch
standard IOErrors with those errnos using those same classes,
so that the following becomes possible:
try:
...
raise IOError(errno.EPERM, "permission denied")
except PermissionError:
...
and works as one would expect, that is, catch the EPERM error.
Without such support, you'd still have to write:
try:
...
raise IOError(errno.EPERM, "permission denied")
except PermissionError:
...EPERM handling code...
except IOError as error:
if error.errno == errno.EPERM:
...EPERM handling code...
duplicating the error handling code.
Perhaps the IOError constructor could be made to switch
the class of the generated object based on the errno
attribute passed to the constructor.
That way no new syntax would be necessary at all.
> Above all, I'd like for the pattern matching alternative to be a bit
> more reasonable. It doesn't have to be verbose and it doesn't have to
> involve new syntax. Apologies for any mistakes in the code, they are
> my own.
>
> Here's the code:
>
>
> # evil global state or something
> error_classes = {}
>
> def io_error(errno_, msg=None): # or something, you get the idea
> try:
> cls = error_classes[errno_]
> except LookupError:
> class IOErrorWithErrno(IOError):
> errno = errno_
>
> cls = error_classes[errno_] = IOErrorWithErrno
>
> return error_classes[errno_]
>
>
> # example of usage
> import errno
> try:
> raise io_error(errno.ENOENT)("<Error message>")
> except io_error(errno.ENOENT):
> print("success")
>
>
> Thanks for your time!
> Devin Jeanpierre
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Apr 07 2011)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
> Perhaps the IOError constructor could be made to switch
> the class of the generated object based on the errno
> attribute passed to the constructor.
Nice suggestion. I'll try to see if that is possible.
Regards
Antoine.
It should be possible with appropriate fancy footwork in __new__. You
do need to be careful to avoid calling __init__ on the created object
twice.
I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
On Thu, Apr 7, 2011 at 7:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:It should be possible with appropriate fancy footwork in __new__. You
>> Perhaps the IOError constructor could be made to switch
>> the class of the generated object based on the errno
>> attribute passed to the constructor.
>
> Nice suggestion. I'll try to see if that is possible.
do need to be careful to avoid calling __init__ on the created object
twice.
I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas
http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
On 7 April 2011 12:48, Nick Coghlan <ncog...@gmail.com> wrote:On Thu, Apr 7, 2011 at 7:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:It should be possible with appropriate fancy footwork in __new__. You
>> Perhaps the IOError constructor could be made to switch
>> the class of the generated object based on the errno
>> attribute passed to the constructor.
>
> Nice suggestion. I'll try to see if that is possible.
do need to be careful to avoid calling __init__ on the created object
twice.
I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)__init__ is called for you on construction so long as the object returned by __new__ is an instance of the type being constructed or a subclass.So if what you want __new__ to return *is* a subclass, then you create it with subclass.__new__(...) and not subclass(...) (the latter would call __init__ which would then be called again after __new__ returns).If what you're returning *isn't* a subclass (which is best avoided anyway) then you can construct it with otherclass(...) as __init__ won't be called for you.
All the best,Michael Foord
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas
--http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
On 7 April 2011 12:48, Nick Coghlan <ncog...@gmail.com> wrote:On Thu, Apr 7, 2011 at 7:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:It should be possible with appropriate fancy footwork in __new__. You
>> Perhaps the IOError constructor could be made to switch
>> the class of the generated object based on the errno
>> attribute passed to the constructor.
>
> Nice suggestion. I'll try to see if that is possible.
do need to be careful to avoid calling __init__ on the created object
twice.
I know I've read an example that demonstrates the principle, but I
unfortunately don't remember where (I initially thought it was in
Guido's new-style class essay, but I checked and that wasn't it)__init__ is called for you on construction so long as the object returned by __new__ is an instance of the type being constructed or a subclass.So if what you want __new__ to return *is* a subclass, then you create it with subclass.__new__(...)
That's the dance I'm trying to remember. You make it work by playing
identity checking games with the cls argument, but it's been
absolutely ages since I read about it and experimented with it.
I think it's something like:
def __new__(cls, *args, **kwds):
if cls is ThisClass:
# Do fancy footwork to implicitly create an appropriate subclass instead
# via subclass.__new__
obj = cls._create_instance(*args, **kwds)
else:
# Don't do anything tricky for subclasses, that's their problem
obj = object.__new__(*args, **kwds)
return obj
Subclasses then have the option of passing the parent class as "cls"
if they want to invoke the fancy footwork, or themselves if they
don't.
On Thu, Apr 7, 2011 at 11:48 PM, Michael Foord <fuzz...@gmail.com> wrote:That's the dance I'm trying to remember. You make it work by playing
> Hmmm... that would rely on subclass.__new__ both existing *and* not calling
> up to its parent __new__ or you will have infinite recursion.
> Probably what you have to do is call object.__new__(subclass, ...) and
> knowing / praying that subclass.__new__ doesn't do anything important...
identity checking games with the cls argument, but it's been
absolutely ages since I read about it and experimented with it.
I think it's something like:
def __new__(cls, *args, **kwds):
if cls is ThisClass:
# Do fancy footwork to implicitly create an appropriate subclass instead
# via subclass.__new__
obj = cls._create_instance(*args, **kwds)
else:
# Don't do anything tricky for subclasses, that's their problem
obj = object.__new__(*args, **kwds)
return obj
Subclasses then have the option of passing the parent class as "cls"
if they want to invoke the fancy footwork, or themselves if they
don't.
Ah, well, you can always except IOError directly if performance is important.
Also, sorry, I left an error in my code. `msg=None` should be omitted.
Nonetheless it appears my idea has been discarded by now. Doesn't
matter too much, just wanted to get it off my chest. Thanks for the
time, everyone!
Devin Jeanpierre
I think EnvironmentError, WindowsError, VMSError, OSError, mmap.error
and select.error should definitely all be merged with IOError, as they
aren't used consistently enough to make handling them differently
reliable even in current code.
socket.error is the one case where that argument isn't quite as
strong, since the socket module is reasonably consistent about raising
it over a base IOError.
I think it would be useful to look at which errno values may currently
be associated with socket.error, and merge it with the new exception
at that level in the type heirarchy.
For example, taking the draft heirarchy at
http://www.python.org/dev/peps/pep-3151/#new-exception-classes, then
the merger that makes the most sense might be "socket.error =
ConnectionError".
Alternatively, and if necessary, socket.error could be grandfathered
in to cover an appropriate subset of errno values via multiple
inheritance: class error(ConnectionError, TimeoutError): pass
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
Their use may be inconsistent in a few places, but those cases
are still well-defined by the implementation, so code relying
on that well-defined behavior will break in subtle ways.
Example:
If code catches select.error (which is only raised for select.select()
and poll_object.poll() calls), because the try...except is expecting
a possible error from the select call used in the try...except
block, it will most likely do the wrong thing for some IOError
raised by a logging file I/O call in that same block.
Moreover, catching the exception at the wrong level, will
prevent the IOError from bubbling up the call chain and can
effectively mask potential errors at lower levels.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Apr 08 2011)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
The phrases "defined by the implementation" and "well-defined" do not
belong in the same sentence.
> Example:
>
> If code catches select.error (which is only raised for select.select()
> and poll_object.poll() calls), because the try...except is expecting
> a possible error from the select call used in the try...except
> block, it will most likely do the wrong thing for some IOError
> raised by a logging file I/O call in that same block.
A deprecation period for the merged exceptions would be advisable.
That's an argument in favouring of figuring out how to implement
correct deprecation warnings for the merges, not an argument in favour
of not doing them.
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
I agree that exceptions being raised by certain APIs
should probably be documented, but in the absence of such documentation,
the implementation is still the ultimate source of wisdom and
given that it's written down in C and not some legal code, I
think "well-defined" is an appropriate term ;-)
>> Example:
>>
>> If code catches select.error (which is only raised for select.select()
>> and poll_object.poll() calls), because the try...except is expecting
>> a possible error from the select call used in the try...except
>> block, it will most likely do the wrong thing for some IOError
>> raised by a logging file I/O call in that same block.
>
> A deprecation period for the merged exceptions would be advisable.
> That's an argument in favouring of figuring out how to implement
> correct deprecation warnings for the merges, not an argument in favour
> of not doing them.
Indeed, getting deprecations right is yet another aspect to consider,
but not the one the example was supposed to explain.
I don't think that such major changes in class hierarchy can be
implemented in a minor Python release.
Note that select.error currently does not inherit from IOError,
so "except IOError" won't catch select errors.
Renaming the exceptions would be something we could do for a
minor release and then have deprecations hint the user to
the naming change.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Apr 08 2011)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
I don't see how either solution is better than continuing to do what
we have right now. The PEP's idea introduces new syntax for a problem
that is currently solved in two lines. Your suggestion makes a new
pattern within the Python exception world, and further helps make the
exception hierarchy a little harder to understand again.
Neither of these seem justified for a rare case (this sort of patter
is fairly rare, most notably this one example) when there's nothing
that awful about the current solution.
Mike
Actually, there is an awful lot of code in the wild that gives
incorrect and wildly misleading error messages *because* correctly
checking the errno attribute is so rare. PEP 3151 would improve the
quality of information provided by a lot of tracebacks and error
messages without many of those developers needing to even set finger
to keyboard.
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
But Nick, that's different from what I'm saying is rare.
I'm saying that the situation where we need an if->raise on a constant
parameter is rare (this being almost the only common case). The issue
of whether people handle that case being rare is separate. Neither the
"except Foo as e if f(e):" syntax nor the "except foo(some_errno):"
pattern propose something that stops people from doing stupid stuff
like "except IOError" with no check.
Mike
- I really like the syntax "except <exc> [as <var>] [if <test>]:".
This addresses a pretty common use case in my experience. I don't care
for the alternate proposed syntax that started this thread. I'm not
sure that the 'if' subclause makes sense without the 'as' subclause,
since most likely you'd want to refer to the caught exception. I note
that it is more powerful than putting "if not <test>: raise" in the
body of the except-clause, because it will allow subsequent except
clauses to match still. I also note that it is a much "cleaner" change
than (again) reorganizing the exception hierarchy, since there is no
backward compatibility to consider.
- Regarding restructuring the exception tree, I don't think this needs
to wait for Python 4. (We've done it several times during Python 2.)
But it is very tricky to do it without breaking existing code, and I'm
not sure that deprecations will help that much. E.g. if two exceptions
A and B are currently not related via inheritance, and we were to make
A a subclass of B, then in code of the form try: ... except B: ...
except A: ... the except A clause would become unreachable. Maybe it's
easier to propose small changes instead of trying to refactor a huge
section of the exception tree?
- Quite independently, I think it is time that we started documenting
which exceptions are raised from standard APIs. The trick is to avoid
overdocumenting: there is no point in documenting that everything can
raise MemoryError or that passing in wildly invalid arguments can
raise TypeError or AttributeError (and pretty much anything else). But
the variety of exceptions that may be raised by something like
urlopen, depending on which layer of the network and I/O stacks
detects a problem are worth documenting. Maybe this example could even
be a starting point for a rationalization of some of the exceptions
involved. (And if we find there are legitimate situations where a
networking or I/O problem raises a "generic" error such as TypeError
or AttributeError, those are probably bugs that should be fixed in the
code.)
- Lastly, there is a bunch of standard exception types that are
usually caused by bad code (as opposed to bad data or an environmental
problem). This would include NameError, TypeError, AttributeError, but
not KeyError, IndexError, ValueError. Perhaps it makes sense to
introduce a common base class for these "bad code" exceptions? Unlike
the other exception types, I think that the set of "bad code"
exceptions is pretty much static; a new library module is not going to
define raise a new kind of "bad code" exception. (But it may well
define a new kind of "bad data" exception, and I don't think we need a
common base class for all "bad data" or "bad state" exceptions.)
--
--Guido van Rossum (python.org/~guido)
Of course, multiple cases can currently be handled by an if statement
except Foo as e:
if e.bar == baz:
...
elif e.bar == qux:
...
else:
raise
This code isn't so bad: it's not hard to understand, it's an existing
pattern, it uses general, powerful constructs.
I think the main opposition is "People aren't currently doing this!"
The problem is, introducing the "except...if" syntax doesn't actually
make anyone do it either. I understand that adding syntax can be seen
as a nudge, but I'm skeptical that it's a strong enough one to justify
the cost.
> - Quite independently, I think it is time that we started documenting
> which exceptions are raised from standard APIs. The trick is to avoid
> overdocumenting: there is no point in documenting that everything can
> raise MemoryError or that passing in wildly invalid arguments can
> raise TypeError or AttributeError (and pretty much anything else). But
> the variety of exceptions that may be raised by something like
> urlopen, depending on which layer of the network and I/O stacks
> detects a problem are worth documenting. Maybe this example could even
> be a starting point for a rationalization of some of the exceptions
> involved. (And if we find there are legitimate situations where a
> networking or I/O problem raises a "generic" error such as TypeError
> or AttributeError, those are probably bugs that should be fixed in the
> code.)
I couldn't agree more with this.
Mike
The problem (as I tried to say, but apparently not clearly enough) is
that if there's a later more general except clause on the same block
it won't work. E.g.
except Foo as e:
<code as above>
except Exception:
<log traceback>
The 'raise' in your code above does not transfer control to the
"except Exception:" clause. The current solution is either to have two
nested try/except blocks, or duplicate the logging code; neither is
great.
> I think the main opposition is "People aren't currently doing this!"
People aren't currently doing what? I have seen many examples like
that, and have myself encountered several times the problem I
explained just now.
> The problem is, introducing the "except...if" syntax doesn't actually
> make anyone do it either. I understand that adding syntax can be seen
> as a nudge, but I'm skeptical that it's a strong enough one to justify
> the cost.
What cost? Adding new optional syntax that doesn't introduce new
keywords is actually pretty cheap -- certainly cheaper than
restructuring the exception hierarchy.
--
--Guido van Rossum (python.org/~guido)
My main issue with said new syntax is that it doesn't make things much
easier to write. You still have to type an explicit condition, and
remember the appropriate errno mnemonic for the situation. The very
notion of "errno" and its plethora of abbreviated error codes is ok for
people used to C programming, but certainly alien to other programmers
(how long does it take to remember that "file not found" is spelled
"ENOENT"?).
The fact that exception messages typically indicate the errno *number*,
not mnemonic, is an additional annoyance (errno numbers are not
standardized and can vary from system to system).
> - Regarding restructuring the exception tree, I don't think this needs
> to wait for Python 4. (We've done it several times during Python 2.)
> But it is very tricky to do it without breaking existing code, and I'm
> not sure that deprecations will help that much. E.g. if two exceptions
> A and B are currently not related via inheritance, and we were to make
> A a subclass of B, then in code of the form try: ... except B: ...
> except A: ... the except A clause would become unreachable. Maybe it's
> easier to propose small changes instead of trying to refactor a huge
> section of the exception tree?
Well, I fear that small changes will not lead us very far. We have to
level the existing tree before adding new types, because otherwise it's
not obvious where these new types should be grafted, since in practice
OSError, IOError or socket.error can be used for very similar error
conditions.
> - Quite independently, I think it is time that we started documenting
> which exceptions are raised from standard APIs.
Agreed, and unit-test them too.
> (But it may well
> define a new kind of "bad data" exception, and I don't think we need a
> common base class for all "bad data" or "bad state" exceptions.)
Isn't it ValueError actually? For example, closed files raise
ValueError when you try to do an operation on them.
Regards
Antoine.
<snip>
> - Quite independently, I think it is time that we started documenting
> which exceptions are raised from standard APIs. The trick is to avoid
> overdocumenting: there is no point in documenting that everything can
> raise MemoryError or that passing in wildly invalid arguments can
> raise TypeError or AttributeError (and pretty much anything else). But
> the variety of exceptions that may be raised by something like
> urlopen, depending on which layer of the network and I/O stacks
> detects a problem are worth documenting. Maybe this example could even
> be a starting point for a rationalization of some of the exceptions
> involved. (And if we find there are legitimate situations where a
> networking or I/O problem raises a "generic" error such as TypeError
> or AttributeError, those are probably bugs that should be fixed in the
> code.)
This would be very nice.
> - Lastly, there is a bunch of standard exception types that are
> usually caused by bad code (as opposed to bad data or an environmental
> problem). This would include NameError, TypeError, AttributeError, but
> not KeyError, IndexError, ValueError. Perhaps it makes sense to
> introduce a common base class for these "bad code" exceptions? Unlike
> the other exception types, I think that the set of "bad code"
> exceptions is pretty much static; a new library module is not going to
> define raise a new kind of "bad code" exception. (But it may well
> define a new kind of "bad data" exception, and I don't think we need a
> common base class for all "bad data" or "bad state" exceptions.)
I actually like both the BadCode exception and the BadData exception.
Geremy Condra
It wouldn't be that difficult to customize the IOError and OSError
reprs to display the errno mnemonic instead of a number, if that's an
issue.
I don't think it's fair to characterize errnos as only for C
programmers. Some other language use errno mnemonics too, not just C
and Python. It's a cross-language way of understanding certain kinds
of error conditions, and as such it makes it a bit easier to do
things. What this PEP does is introduce a language-specific exception
hierarchy instead. I think it's a good idea, but it's not as clearly
superior to errnos as you suggest. It's easier to remember the
language-specific exception hierarchy if you don't have to deal with
other languages that use errnos, but if you do, the situation is
different.
True, but that's like saying we should use sprintf() for string
formatting and fgets() for file access since other languages such as C
and PHP use them. In trying to find elegant solutions to problems, we
often diverge from solutions common in other languages.
(actually, if we followed the errno paradigm closely, we shouldn't even
raise exceptions, but return the error code or store it in a
thread-local variable instead ;-))
Well, not quite. The only case that I can see where errnos are
obviously bad is with multiple errnos that mean the same thing in
practice (so much so that they might even have the same value), making
proper IO handling inconvenient and non-obvious. In particular I'm
thinking of errors like EAGAIN and EWOULDBLOCK. The PEP's suggested
exception hierarchy, and its collapse of some errnos into the same
exception, is nice on that front (for example, BlockingIOError).
However, I interpret what you say as a complaint that errnos are
obscure and thus we should avoid making programmers memorize them, and
avoid encouraging their use. I don't agree with that. They aren't
obscure, they're a legitimate part of POSIX and fairly well-known.
Beginners to IO will have to memorize something anyway, and it may as
well be errnos. A more experienced programmer might come from from C
or C# or Ruby, or whatever, and would get less educational overhead
when doing Python IO, because all these languages share a little bit
of POSIX in their IO APIs.
It's not even about following it closely, but about building on what
people know. Python doesn't support sprintf, but it does (or perhaps
"did") support the sprintf syntax. It didn't follow it *closely* --
"%s" % 2 doesn't result in a segfault or anything -- but closely
enough so that programmers familiar with sprintf could get started
quickly, and programmers familiar with Python could understand
sprintf. The replacement, str.format, doesn't come from nowhere
either, but builds on a different string formatting tradition also
shared by C#. Having these ties to the outside world is useful and
desirable, it makes everyone's life easier. I think that breaking
these ties needs an overriding reason, generally relating to the
outside world actually hurting people. The PEP solves the issue where
people don't necessarily handle all the right errnos, without breaking
ties to the IO community and eliminating them altogether. That's
pretty cool. It also offers a way to make errno handling easier.
That's also pretty cool.
Devin
As I explained in other messages, it also adds semantics that are not
so easily emulated with the existing syntax (you'd have to repeat code
or use nested try/except blocks).
> You still have to type an explicit condition, and
> remember the appropriate errno mnemonic for the situation. The very
> notion of "errno" and its plethora of abbreviated error codes is ok for
> people used to C programming, but certainly alien to other programmers
> (how long does it take to remember that "file not found" is spelled
> "ENOENT"?).
Well, this is a fact of life. There are hundreds of errno codes and
they vary across systems, both in names and in numbers (even if there
is a handful that exist nearly everywhere and almost always have the
same values). This is the nature of system calls, and some operating
systems just have a richer set of error conditions that they present
to the (advanced) programmer.
If you are careful you can probably come up with a few handfuls of
common error conditions for which it makes sense to introduce new
exceptions that are defined (and presumably raised) on all platforms.
"File not found" is one of those. But there are always error
conditions that are not in this list, and we would still need the
errno attribute and the errno module to map between numbers, names and
messages in the general case. And occasionally some programmer needs
to select behaviors depending on the precise errno value.
> The fact that exception messages typically indicate the errno *number*,
> not mnemonic, is an additional annoyance (errno numbers are not
> standardized and can vary from system to system).
That would seem to be a problem with the str() and/or repr() of
OSError, and perhaps we can fix it there.
>> - Regarding restructuring the exception tree, I don't think this needs
>> to wait for Python 4. (We've done it several times during Python 2.)
>> But it is very tricky to do it without breaking existing code, and I'm
>> not sure that deprecations will help that much. E.g. if two exceptions
>> A and B are currently not related via inheritance, and we were to make
>> A a subclass of B, then in code of the form try: ... except B: ...
>> except A: ... the except A clause would become unreachable. Maybe it's
>> easier to propose small changes instead of trying to refactor a huge
>> section of the exception tree?
>
> Well, I fear that small changes will not lead us very far. We have to
> level the existing tree before adding new types, because otherwise it's
> not obvious where these new types should be grafted, since in practice
> OSError, IOError or socket.error can be used for very similar error
> conditions.
Alas, you are right, they are all essentially the same thing but
raised by different operations (sometimes it's even the same
underlying system call, e.g. os.write vs. stream.write).
>> - Quite independently, I think it is time that we started documenting
>> which exceptions are raised from standard APIs.
>
> Agreed, and unit-test them too.
>
>> (But it may well
>> define a new kind of "bad data" exception, and I don't think we need a
>> common base class for all "bad data" or "bad state" exceptions.)
>
> Isn't it ValueError actually? For example, closed files raise
> ValueError when you try to do an operation on them.
No, there are lots of other exceptions indicating "bad arguments" or
"bad data" or "bad state" that don't derive from ValueError, from
KeyError to binhex.Error (a random example I picked from the stdlib).
--
--Guido van Rossum (python.org/~guido)
Well, errnos are only the POSIX API side of system calls.
For example, a seasoned Windows C programmer might not have faced a
single EBADF or ENOENT in their entire career, since the Win32 API
doesn't use those.
Similarly, java.io.IOException doesn't seem to have any errno-alike
member, but it does have subclasses named e.g. FileNotFoundException.
(http://download.oracle.com/javase/1.5.0/docs/api/java/io/IOException.html)
Looking at .Net, System.IO.IOException does have a numerical "HResult"
member, but it doesn't seem related to POSIX errnos (“HRESULT is a
32-bit value, divided into three different fields: a severity code, a
facility code, and an error code”). It also has a couple of subclasses
such as FileNotFoundException.
(http://msdn.microsoft.com/en-us/library/system.io.ioexception.aspx)
> > The fact that exception messages typically indicate the errno *number*,
> > not mnemonic, is an additional annoyance (errno numbers are not
> > standardized and can vary from system to system).
>
> That would seem to be a problem with the str() and/or repr() of
> OSError, and perhaps we can fix it there.
True.
Regards
Antoine.
On Fri, Apr 8, 2011 at 3:06 PM, Antoine Pitrou <soli...@pitrou.net> wrote:Well, not quite. The only case that I can see where errnos are
> True, but that's like saying we should use sprintf() for string
> formatting and fgets() for file access since other languages such as C
> and PHP use them. In trying to find elegant solutions to problems, we
> often diverge from solutions common in other languages.
>
> (actually, if we followed the errno paradigm closely, we shouldn't even
> raise exceptions, but return the error code or store it in a
> thread-local variable instead ;-))
>
> Regards
>
> Antoine.
obviously bad is with multiple errnos that mean the same thing in
practice (so much so that they might even have the same value), making
proper IO handling inconvenient and non-obvious. In particular I'm
thinking of errors like EAGAIN and EWOULDBLOCK. The PEP's suggested
exception hierarchy, and its collapse of some errnos into the same
exception, is nice on that front (for example, BlockingIOError).
However, I interpret what you say as a complaint that errnos are
obscure and thus we should avoid making programmers memorize them, and
avoid encouraging their use. I don't agree with that. They aren't
obscure, they're a legitimate part of POSIX and fairly well-known.
Beginners to IO will have to memorize something anyway, and it may as
well be errnos.
You can use more useful exceptions for *some* errnos. But not for all.
--
--Guido van Rossum (python.org/~guido)
Sorry, I misinterpreted "neither of these" as including PEP 3151
itself (which aims to eliminate the need for most errno pattern
matching). I had missed that someone had brought up a second
pattern-matching idea, and it was the two different approaches that
you were referring to.
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
>As I explained in other messages, it also adds semantics that are not
>so easily emulated with the existing syntax (you'd have to repeat code
>or use nested try/except blocks).
Interestingly, this is one of the few (only?) .NET features that is
exposed in Visual Basic but not in C#. Maybe there's something to learn
from that?
http://blogs.msdn.com/b/jaredpar/archive/2008/10/09/vb-catch-when-why-so-special.aspx
Eric.
FWIW I still am at least +0.5 on the idea. I do note that currently
this is lecal syntax:
try: ...
except stuff if cond else otherstuff: ...
so allowing the syntax
except <exception> [as <variable>] [if <condition>]:
could break some code; however
except <exception> [as <variable> [if <condition>]]:
could not.
Anyone want to write a PEP (could be pretty short & sweet, the VB
reference would be useful) and a reference implementation?
--
--Guido van Rossum (python.org/~guido)
Another quirk occurred to me today: select.error doesn't derive from
EnvironmentError, and so it doesn't have the errno attribute (even
though the select module "correctly" instantiates it with a (errno,
message) tuple). Also, its str() is borked:
>>> e = select.error(4, "interrupted")
>>> str(e)
"(4, 'interrupted')"
>>> raise e
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
select.error: (4, 'interrupted')
Regards
Antoine.
Works fine in Python 2.7:
>>> import select
>>> e = select.error(4, "intr")
>>> e
error(4, 'intr')
>>> try:
... raise e
... except select.error, x:
... code, text = x
... print code,text
...
4 intr
>>>
Note that existing code will not look for an attribute that
doesn't exist :-) It'll unwrap the tuple and work from there
or use the .args attribute to get at the constructor args.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, May 12 2011)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
2011-06-20: EuroPython 2011, Florence, Italy 39 days to go
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
True. My point is that not having "errno" makes it even more obscure how
to check for different kinds of select errors. Also, given that other
"environmental" errors will have an "errno" giving the POSIX error code,
it's easy to get surprised.
Regards
Antoine.
Note that this is the repr(), while Antoine showed the str().
But the str() looks correct to me as well (for an exception that doesn't
derive from EnvironmentError).
Georg
It's technically correct, sure. The point is that "technically correct"
translates to "humanly bogus" here, because of the broken I/O
exception hierarchy.
Regards
Antoine.
Yep, and I'm all for fixing it with PEP 3151 :)
Georg
Somewhat related bug:
http://bugs.python.org/issue12029
Cheers,
Chris
Interesting. If that is intentional I'd advocate against it unless
there's a strong argument for it.
Another idea (also likely already proposed) would be to match against
instances as well, by the 'args' attribute:
try:
...
except IOError(32): # isinstance IOError and .args == (32,)
...
If this seems crazy consider that it's (to some extent) similar to the
behavior of 'raise'.
Unfortunately, as described it wouldn't match IOError(32, 'Blah blah
blah'). Although maybe it makes sense to create an Anything builtin,
which is equal to everything, such that IOError(X, Y) == IOError(X,
Anything) == IOError(X, Z) for all X, Y, and Z (except stupid X like X
= nan).
I do like it.
Devin Jeanpierre
Also it's a bit magical -- normally one doesn't expect
Haskell-like pattern matching in Python.
Maybe something more explicit would be better:
try:
...
except IOError as e with e.errno == 32:
...
--
Greg
Then we're back where we started in which case I prefer 'if' as the keyword.