Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

try... except with unknown error types

93 views
Skip to first unread message

Yingjie Lin

unread,
Aug 19, 2011, 3:09:12 PM8/19/11
to pytho...@python.org
Hi Python users,

I have been using try...except statements in the situations where I can expect a certain type of errors might occur.
But sometimes I don't exactly know the possible error types, or sometimes I just can't "spell" the error types correctly.
For example,

try:
response = urlopen(urljoin(uri1, uri2))
except urllib2.HTTPError:
print "URL does not exist!"

Though "urllib2.HTTPError" is the error type reported by Python, Python doesn't recognize it as an error type name.
I tried using "HTTPError" alone too, but that's not recognized either.

Does anyone know what error type I should put after the except statement? or even better: is there a way not to specify
the error types? Thank you.


- Yingjie

John Gordon

unread,
Aug 19, 2011, 3:14:54 PM8/19/11
to

> try:
> response = urlopen(urljoin(uri1, uri2))
> except urllib2.HTTPError:
> print "URL does not exist!"

> Though "urllib2.HTTPError" is the error type reported by Python, Python
> doesn't recognize it as an error type name. I tried using "HTTPError"
> alone too, but that's not recognized either.

Have you imported urllib2 in your code?

> Does anyone know what error type I should put after the except
> statement? or even better: is there a way not to specify the error
> types? Thank you.

You can catch all exceptions by catching the base class Exception:

try:
some_method()
except Exception, e:
print "some error happened, here is the explanation:"
print str(e)

--
John Gordon A is for Amy, who fell down the stairs
gor...@panix.com B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

xDog Walker

unread,
Aug 19, 2011, 3:22:35 PM8/19/11
to pytho...@python.org
On Friday 2011 August 19 12:09, Yingjie Lin wrote:
> Hi Python users,
>
> I have been using try...except statements in the situations where I can
> expect a certain type of errors might occur. But sometimes I don't exactly
> know the possible error types, or sometimes I just can't "spell" the error
> types correctly. For example,
>
> try:
> response = urlopen(urljoin(uri1, uri2))
> except urllib2.HTTPError:
> print "URL does not exist!"
>
> Though "urllib2.HTTPError" is the error type reported by Python, Python
> doesn't recognize it as an error type name. I tried using "HTTPError" alone
> too, but that's not recognized either.
>
> Does anyone know what error type I should put after the except statement?
> or even better: is there a way not to specify the error types? Thank you.

You probably need to import urllib2 before you can use urllib2.HTTPError.

Otherwise, you can try using the base class:

except Exception, e:

--
I have seen the future and I am not in it.

Zero Piraeus

unread,
Aug 19, 2011, 3:30:55 PM8/19/11
to pytho...@python.org
:

On 19 August 2011 15:09, Yingjie Lin <Yingj...@mssm.edu> wrote:
>
> I have been using try...except statements in the situations where I can expect a certain type of errors might occur.
> But sometimes I don't exactly know the possible error types, or sometimes I just can't "spell" the error types correctly.
> For example,
>
> try:
> response = urlopen(urljoin(uri1, uri2))
> except urllib2.HTTPError:
> print "URL does not exist!"
>
> Though "urllib2.HTTPError" is the error type reported by Python, Python doesn't recognize it as an error type name.
> I tried using "HTTPError" alone too, but that's not recognized either.
>
> Does anyone know what error type I should put after the except statement? or even better: is there a way not to specify
> the error types? Thank you.

You should always specify the error type, so that your error-handling
code won't attempt to handle an error it didn't anticipate and cause
even more problems.

In this case, I think it's just that you haven't imported HTTPError
into your namespace - if you do, it works:

>>> from urllib2 import urlopen, HTTPError
>>> try:
... response = urlopen("http://google.com/invalid")
... except HTTPError:
... print "URL does not exist!"
...
URL does not exist!
>>>

Alternatively:

>>> import urllib2
>>> try:
... response = urllib2.urlopen("http://google.com/invalid")
... except urllib2.HTTPError:
... print "URL does not exist!"
...
URL does not exist!
>>>

A careful look at the difference between these two ought to make it
clear what's going on.

-[]z.

Yingjie Lin

unread,
Aug 19, 2011, 3:35:56 PM8/19/11
to Zero Piraeus, pytho...@python.org
Hi Zero,

I see! This is very helpful. Thank you.

- Yingjie

Steven D'Aprano

unread,
Aug 19, 2011, 4:13:57 PM8/19/11
to
John Gordon wrote:

> In <mailman.230.13137809...@python.org> Yingjie Lin
> <Yingj...@mssm.edu> writes:
>
>> try:
>> response = urlopen(urljoin(uri1, uri2))
>> except urllib2.HTTPError:
>> print "URL does not exist!"
>
>> Though "urllib2.HTTPError" is the error type reported by Python, Python
>> doesn't recognize it as an error type name. I tried using "HTTPError"
>> alone too, but that's not recognized either.
>
> Have you imported urllib2 in your code?

Good question.

If Python "doesn't recognize it as an error type name", there is a reason
for that. Exceptions are exactly the same as every other name:

>>> foo.spam
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'foo' is not defined
>>> urllib2.HTTPError
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'urllib2' is not defined

>> Does anyone know what error type I should put after the except
>> statement? or even better: is there a way not to specify the error
>> types? Thank you.
>
> You can catch all exceptions by catching the base class Exception:

Except that is nearly always poor advice, because it catches too much: it
hides bugs in code, as well as things which should be caught.

You should always catch the absolute minimum you need to catch.

--
Steven

John Gordon

unread,
Aug 19, 2011, 4:24:38 PM8/19/11
to
In <4e4ec405$0$29994$c3e8da3$5496...@news.astraweb.com> Steven D'Aprano <steve+comp....@pearwood.info> writes:

> > You can catch all exceptions by catching the base class Exception:

> Except that is nearly always poor advice, because it catches too much: it
> hides bugs in code, as well as things which should be caught.

> You should always catch the absolute minimum you need to catch.

I agree, but it did seem to be exactly what he was asking for.

Steven D'Aprano

unread,
Aug 19, 2011, 4:45:36 PM8/19/11
to
John Gordon wrote:

> In <4e4ec405$0$29994$c3e8da3$5496...@news.astraweb.com> Steven D'Aprano
> <steve+comp....@pearwood.info> writes:
>
>> > You can catch all exceptions by catching the base class Exception:
>
>> Except that is nearly always poor advice, because it catches too much: it
>> hides bugs in code, as well as things which should be caught.
>
>> You should always catch the absolute minimum you need to catch.
>
> I agree, but it did seem to be exactly what he was asking for.

Sure, but if we're giving advice to somebody who is clearly a beginner
(doesn't even know how to deal with a simple NameError from failing to
import a module), it is our responsibility to teach *good* habits, not to
teach him to be a crap programmer.

Even if you don't think it's the ethical thing to do, consider that someday
you might be maintaining code written by the OP :)


--
Steven

Mel

unread,
Aug 19, 2011, 5:22:42 PM8/19/11
to
xDog Walker wrote:
> On Friday 2011 August 19 12:09, Yingjie Lin wrote:
[ ... ]

>> Does anyone know what error type I should put after the except statement?
>> or even better: is there a way not to specify the error types? Thank you.
>
> You probably need to import urllib2 before you can use urllib2.HTTPError.
>
> Otherwise, you can try using the base class:
>
> except Exception, e:

There are maybe better base classes to use. Running the interpreter:
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> help (urllib2.HTTPError)

will show you

class HTTPError(URLError, urllib.addinfourl)
| Raised when HTTP error occurs, but also acts like non-error return
|
| Method resolution order:
| HTTPError
| URLError
| exceptions.IOError
| exceptions.EnvironmentError
| exceptions.StandardError
| exceptions.Exception
| exceptions.BaseException
>

So catching any of urllib2.HTTPError, urllib2.URLError, IOError,
EnvironmentError, or StandardError will detect the exception -- with
increasing levels of generality.

Mel.

Stephen Hansen

unread,
Aug 19, 2011, 5:47:04 PM8/19/11
to pytho...@python.org, Yingjie Lin
On 8/19/11 12:09 PM, Yingjie Lin wrote:
> try:
> response = urlopen(urljoin(uri1, uri2))
> except urllib2.HTTPError:
> print "URL does not exist!"
>
> Though "urllib2.HTTPError" is the error type reported by Python, Python doesn't recognize it as an error type name.
> I tried using "HTTPError" alone too, but that's not recognized either.

Exceptions are objects like any other, and they are defined in specific
places. Only the standard ones are available everywhere; things like
IndexError and AttributeError. See the 'exceptions' module for the
standard ones. Everything else, you have to 'grab' the object from where
it lives -- in this case, in urllib2.


> Does anyone know what error type I should put after the except statement? or even better: is there a way not to specify
> the error types? Thank you.

You can use a "bare except", like so:

try:
...
except:
...

But avoid it, if you can. Or at least, don't let it become
habit-forming: for networking code and interaction with external things,
I usually have a bare-except as a final fallback, after trying other
more specific things-- but only as a last resort.

FWIW, the error hierarchy of url fetching is more then a little bit
annoying. My own _request object I end up using for one of our services
catches, in order:

try:
...
except urllib2.HTTPError:
except urllib2.URLError:
except httplib.BadStatusLine:
except httplib.HTTPException:
except socket.timeout:
except:


With each case logging and handling the error in a bit of a different
way. (Though most, eventually, resulting in the request being retried --
all in the name of a mandate that this particular bit of code must not,
under any circumstances, not ultimately work. Even going so far as to
start having hour long waits between retries until the other side is
finally up :P)


--

Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/

signature.asc

Seebs

unread,
Aug 20, 2011, 1:46:39 AM8/20/11
to
On 2011-08-19, Steven D'Aprano <steve+comp....@pearwood.info> wrote:
> Even if you don't think it's the ethical thing to do, consider that someday
> you might be maintaining code written by the OP :)

A common further conclusion people reach is "but then I will be able to get
a job fixing it!"

Trust me, this is NOT where you want to go. :)

-s
--
Copyright 2011, all wrongs reversed. Peter Seebach / usenet...@seebs.net
http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
I am not speaking for my employer, although they do rent some of my opinions.

Paul Rubin

unread,
Aug 20, 2011, 3:18:25 PM8/20/11
to
Steven D'Aprano <steve+comp....@pearwood.info> writes:
>> You can catch all exceptions by catching the base class Exception:
>
> Except that is nearly always poor advice, because it catches too much: it
> hides bugs in code, as well as things which should be caught.
> You should always catch the absolute minimum you need to catch.

But there's no way to know what that minimum is. Python libraries throw
all sorts of exceptions that their documentation doesn't mention.
Java's checked exceptions are obnoxious but they do have their
attractions.

Steven D'Aprano

unread,
Aug 20, 2011, 4:14:37 PM8/20/11
to
Paul Rubin wrote:

> Steven D'Aprano <steve+comp....@pearwood.info> writes:
>>> You can catch all exceptions by catching the base class Exception:
>>
>> Except that is nearly always poor advice, because it catches too much: it
>> hides bugs in code, as well as things which should be caught.
>> You should always catch the absolute minimum you need to catch.
>
> But there's no way to know what that minimum is. Python libraries throw
> all sorts of exceptions that their documentation doesn't mention.

Yes, you're absolutely correct. But it's also irrelevant. Most of those
exceptions should not be caught, even if you know what they are, because
they represent either bugs that should be fixed, or bad data which should
raise an exception. A bare except, or except Exception, is hardly ever the
right approach.

As for exceptions which should be caught, they should be dealt with on a
case-by-case basis. There's no need to identify all those obscure
exception-raising cases ahead of time. After all, unless you're writing
software for a nuclear reactor, or an aeroplane's autopilot, chances are
that *bugs don't really matter*. That is to say, if you release software
with a hidden bug, the consequences generally aren't very important.
(Depends on the nature of the software, and the bug, of course. Sometimes
bugs are important. How's your test suite?) At some point, you will get a
bug report, and then you will fix the bug. The fix may involve catching an
extra exception, or avoiding generating the exception in the first place.

Trying to predict ahead of time every possible exception that could be
raised, and deal with them correctly (as opposed to just sweeping them
under the carpet), is not only impossible but also usually unnecessary.

It took me a long time to realise that the world won't end if I write a
piece of software with a bug. Now I realise that software is never
finished, there's always going to be a next version, so trying to make it
perfect is a fool's errand. It's very liberating :)


> Java's checked exceptions are obnoxious but they do have their
> attractions.

No doubt about it, the concept is attractive, but a few Java heavyweights
now consider checked exceptions to be a mistake.

http://www.mindview.net/Etc/Discussions/CheckedExceptions
http://radio-weblogs.com/0122027/stories/2003/04/01/JavasCheckedExceptionsWereAMistake.html

More here:
http://www.ibm.com/developerworks/java/library/j-jtp05254/index.html

--
Steven

John Nagle

unread,
Aug 20, 2011, 4:15:41 PM8/20/11
to
On 8/19/2011 1:24 PM, John Gordon wrote:
> In<4e4ec405$0$29994$c3e8da3$5496...@news.astraweb.com> Steven D'Aprano<steve+comp....@pearwood.info> writes:
>
>>> You can catch all exceptions by catching the base class Exception:
>
>> Except that is nearly always poor advice, because it catches too much: it
>> hides bugs in code, as well as things which should be caught.
>
>> You should always catch the absolute minimum you need to catch.

Right. When in doubt, catch EnvironmentError. That means something
external to the program, at the OS or network level, has a problem.
"Exception" covers errors which are program bugs, like references to
undefined class members.

John Nagle

Paul Rubin

unread,
Aug 21, 2011, 2:26:51 PM8/21/11
to
Steven D'Aprano <steve+comp....@pearwood.info> writes:
>> But there's no way to know what that minimum is. Python libraries throw
>> all sorts of exceptions that their documentation doesn't mention.
>
> Yes, you're absolutely correct. But it's also irrelevant. Most of those
> exceptions should not be caught, even if you know what they are, because
> they represent either bugs that should be fixed, or bad data which should
> raise an exception. A bare except, or except Exception, is hardly ever the
> right approach.

I'm not sure what to do instead. The exceptions I'm currently dealing
with happen when certain network operations go wrong (e.g. network or
remote host is down, connection fails, etc.) The remedy in each case is
to catch the exception, log the error, and try the operation again
later. But there's no guaranteed-to-be-complete list in the Python docs
of all the exceptions that can be thrown. A new and surprising mode of
network failure can lead to an unhandled exception, unless you catch
everything.

The Erlang approach is tempting. Don't catch the exception at all--just
let the process crash, and restart it. But that's a more heavyweight
operation in Python.

> After all, unless you're writing
> software for a nuclear reactor, or an aeroplane's autopilot, chances are
> that *bugs don't really matter*. That is to say, if you release software
> with a hidden bug, the consequences generally aren't very important.

It's a retail application that would cause some business disruption and
a pissed off customer if the program went down. Also it's in an
embedded box on a customer site. It's not in Antarctica or anything
like that, but it's a few towns over, and someone would have to drive
there (probably through heavy traffic) if something went wrong that
power cycling the box couldn't fix.

> It took me a long time to realise that the world won't end if I write a
> piece of software with a bug.

It's not the end of the world if I get run over a truck, but such an
event would be of enough consequence to me personally that I find it
worth going to some trouble to avoid it.

Chris Angelico

unread,
Aug 21, 2011, 3:17:59 PM8/21/11
to pytho...@python.org
On Sun, Aug 21, 2011 at 7:26 PM, Paul Rubin <no.e...@nospam.invalid> wrote:
> I'm not sure what to do instead.  The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.)  The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later.  But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown.  A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.
>

A new and surprising mode of network failure would be indicated by a
new subclass of IOError or EnvironmentError. If you catch one of
those, you should catch it. That's the benefit of hierarchical
exceptions.

ChrisA

Ethan Furman

unread,
Aug 21, 2011, 3:22:47 PM8/21/11
to pytho...@python.org
Paul Rubin wrote:
> Steven D'Aprano <steve+comp....@pearwood.info> writes:
>>> But there's no way to know what that minimum is. Python libraries throw
>>> all sorts of exceptions that their documentation doesn't mention.
>> Yes, you're absolutely correct. But it's also irrelevant. Most of those
>> exceptions should not be caught, even if you know what they are, because
>> they represent either bugs that should be fixed, or bad data which should
>> raise an exception. A bare except, or except Exception, is hardly ever the
>> right approach.
>
> I'm not sure what to do instead. The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.) The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later. But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown. A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.

In a case like this I can see catching everything so long as (which you
say you are doing) you log the error somehow -- what's really
frustrating is when the error is simply tossed with no record
whatsoever... what a pain to debug!

~Ethan~

Terry Reedy

unread,
Aug 21, 2011, 3:52:28 PM8/21/11
to pytho...@python.org
On 8/21/2011 2:26 PM, Paul Rubin wrote:
> Steven D'Aprano<steve+comp....@pearwood.info> writes:
>>> But there's no way to know what that minimum is. Python libraries throw
>>> all sorts of exceptions that their documentation doesn't mention.
>>
>> Yes, you're absolutely correct. But it's also irrelevant. Most of those
>> exceptions should not be caught, even if you know what they are, because
>> they represent either bugs that should be fixed, or bad data which should
>> raise an exception. A bare except, or except Exception, is hardly ever the
>> right approach.
>
> I'm not sure what to do instead. The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.) The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later. But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown. A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.

I would expect that catching socket.error (or even IOError) should catch
all of those.

"exception socket.error
A subclass of IOError, this exception is raised for socket-related
errors. It is recommended that you inspect its errno attribute to
discriminate between different kinds of errors."

> It's a retail application that would cause some business disruption and
> a pissed off customer if the program went down. Also it's in an
> embedded box on a customer site. It's not in Antarctica or anything
> like that, but it's a few towns over, and someone would have to drive
> there (probably through heavy traffic) if something went wrong that
> power cycling the box couldn't fix.


--
Terry Jan Reedy

Roy Smith

unread,
Aug 21, 2011, 5:14:30 PM8/21/11
to
In article <7xty9ah...@ruckus.brouhaha.com>,
Paul Rubin <no.e...@nospam.invalid> wrote:

> It's a retail application that would cause some business disruption and
> a pissed off customer if the program went down. Also it's in an
> embedded box on a customer site. It's not in Antarctica or anything
> like that, but it's a few towns over, and someone would have to drive
> there (probably through heavy traffic) if something went wrong that
> power cycling the box couldn't fix.

I would do something like this:

try:
do_some_network_stuff()
except IOError:
do_normal_recovery()
except Exception:
call_home()
do_some_other_recovery()

You are right, in an embedded/unattended application, you don't want to
ever crash. If you ever get an error that you don't understand, you
have no choice but to do the best you can to recover. I would also
generate some kind of alert back to home base to tell somebody to take a
look at it and make sure things are fine.

I would expect that the whole application is wrapped in some kind of
watchdog timer which will do a hard reset of the entire system.

Steven D'Aprano

unread,
Aug 21, 2011, 8:25:40 PM8/21/11
to
Paul Rubin wrote:

> Steven D'Aprano <steve+comp....@pearwood.info> writes:
>>> But there's no way to know what that minimum is. Python libraries throw
>>> all sorts of exceptions that their documentation doesn't mention.
>>
>> Yes, you're absolutely correct. But it's also irrelevant. Most of those
>> exceptions should not be caught, even if you know what they are, because
>> they represent either bugs that should be fixed, or bad data which should
>> raise an exception. A bare except, or except Exception, is hardly ever
>> the right approach.
>
> I'm not sure what to do instead. The exceptions I'm currently dealing
> with happen when certain network operations go wrong (e.g. network or
> remote host is down, connection fails, etc.) The remedy in each case is
> to catch the exception, log the error, and try the operation again
> later. But there's no guaranteed-to-be-complete list in the Python docs
> of all the exceptions that can be thrown. A new and surprising mode of
> network failure can lead to an unhandled exception, unless you catch
> everything.

I was waiting for you to raise network errors :)

Network errors are a particularly annoying case, because although rare and
undocumented, they are legitimate errors that should be caught. I feel your
pain.


> The Erlang approach is tempting. Don't catch the exception at all--just
> let the process crash, and restart it. But that's a more heavyweight
> operation in Python.

The Erland approach sounds good, but as I've never used it, I don't know how
well it works in practice.


>> After all, unless you're writing
>> software for a nuclear reactor, or an aeroplane's autopilot, chances are
>> that *bugs don't really matter*. That is to say, if you release software
>> with a hidden bug, the consequences generally aren't very important.
>
> It's a retail application that would cause some business disruption and
> a pissed off customer if the program went down. Also it's in an
> embedded box on a customer site. It's not in Antarctica or anything
> like that, but it's a few towns over, and someone would have to drive
> there (probably through heavy traffic) if something went wrong that
> power cycling the box couldn't fix.

Customers are always pissed off when something goes wrong, but ask them to
pay an extra $300 for a battery backup unit to the hardware RAID controller
*they* insisted on against your advice, and they say no. But I'm not
bitter...

Customer gets pissed off. What's the consequences? Do they sue you? Leave?
Are they locked into a five year contract and have to pay you even if they
do leave? Do you have to fix the incident at no charge, or give them two
hours free support? Are you competing with armour-plated mature software
that never goes down, or is the field yours alone? How price sensitive are
your customers? Will they pay an extra $100,000 for an extra 9 in the
expected uptime?

Without knowing the consequences to *you* of failure, I can't tell you where
you should be spending your time: trying to predict errors ahead of time,
or fixing them once they've been seen. This is a business problem, not a
software problem.

Don't misunderstand me, I'm not suggesting that you shouldn't try to avoid
things going wrong. But it's a hard problem. Solving it is why they pay you
the big bucks *cough*:

http://www.joelonsoftware.com/items/2007/12/06.html

and is the difference between version 0.4 of an application, and version
3.4. If you're trying to go straight to version 3.4 without putting the
software through real-world testing, then your testing better be mean and
tough. *Really* tough.

http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html


--
Steven

Steven D'Aprano

unread,
Aug 21, 2011, 8:30:37 PM8/21/11
to
Chris Angelico wrote:


> A new and surprising mode of network failure would be indicated by a
> new subclass of IOError or EnvironmentError.

/s/would/should/

I don't see why you expect this, when *existing* network-related failures
aren't:

>>> import socket
>>> issubclass(socket.error, EnvironmentError)
False

(Fortunately that specific example is fixed in Python 3.)

Besides, there's a world of difference between "should be" and "are".


--
Steven

Chris Angelico

unread,
Aug 21, 2011, 8:41:47 PM8/21/11
to pytho...@python.org
On Mon, Aug 22, 2011 at 1:30 AM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> /s/would/should/
>
> I don't see why you expect this, when *existing* network-related failures
> aren't

Ehh, granted. Definitely a case of "should". But certainly, there
won't be an infinite number of new exceptions invented; most of the
runtime issues you'll have will fall into a fairly small number of
exception types (either by subclassing or by parameters eg errno).

ChrisA

Roy Smith

unread,
Aug 21, 2011, 10:09:19 PM8/21/11
to
In article <4e51a205$0$29974$c3e8da3$5496...@news.astraweb.com>,

Steven D'Aprano <steve+comp....@pearwood.info> wrote:

> http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html

I *love* being the Chaos Monkey!

A few jobs ago, I had already turned in my resignation and was a
short-timer, counting down the days to when I was out of there. A new
server had just come in and I was helping the guy who was going to
replace me set it up.

This was the first server we had gotten with RAID and redundant power
supplies and I wanted to test them out. My accomplice was horrified,
but technically I was still his boss, so I got to do what I wanted. I
opened up the drive shelve and yanked out a drive. The console printed
a nice neat warning about loss of media and kept chugging along, just
like it should.

Then I pulled out a second drive. Sure enough, the whole array failed,
just like expected.

Then, I decided to have a go at the redundant power supplies. We paid a
lot of money for that, and I damn well was going to test them now, when
there was no real risk. I grabbed the big handle on the front of one of
the (hot swappable) power supplies and pulled. The whole thing went
dark. Turns out there had been an configuration error and the second
power supply had never been installed (just a faceplate that looked like
a power supply). My buddy was pissed, but I figure I'd just done him a
big favor. Better to find out about it now than when the supply failed
at some critical juncture.

Steven D'Aprano

unread,
Aug 22, 2011, 12:01:55 AM8/22/11
to

Er, you can't know that either.

Except that all exceptions must be rooted at BaseException, there *can* be
an infinite number of new exception types invented. Or functions could
raise unexpected, but known, exceptions, whether built-in or not.

Of course, the set of hypothetical exceptions that could theoretically be
raised is much, much bigger than the set of exceptions which you can
realistically expect to see, especially if you limit yourself to those that
don't leave you wondering about the sanity of the developer ("WTF? Why is
x+1 raising HTTPBadGatewayError?"). But still, as a general rule you should
not be surprised to see any of:

AttributeError
TypeError
ValueError
(and likely others)

The more important question is, if you get an exception, any exception, is
that a bug in the function you are calling, a bug in your calling code (you
supplied a bad argument), or an expected result that you should catch and
work around?


--
Steven

Paul Rubin

unread,
Aug 23, 2011, 3:21:45 AM8/23/11
to
Chris Angelico <ros...@gmail.com> writes:
> Ehh, granted. Definitely a case of "should". But certainly, there
> won't be an infinite number of new exceptions invented;

Right, the number is finite, but the issue is that it's unknown. It's
like never knowing whether you've fixed the last bug in a program.

Chris Angelico

unread,
Aug 23, 2011, 4:26:11 AM8/23/11
to pytho...@python.org

Yeah. Oh, I know when I've fixed the last bug in a program. It's the
day the program gets deleted. Until then? Nope.

ChrisA

Steven D'Aprano

unread,
Aug 23, 2011, 4:33:06 AM8/23/11
to
On Mon, 22 Aug 2011 04:26 am Paul Rubin wrote:

> The Erlang approach is tempting. Don't catch the exception at all--just
> let the process crash, and restart it. But that's a more heavyweight
> operation in Python.

You might be interested in this paper:

http://usenix.org/events/hotos03/tech/full_papers/candea/candea.pdf


--
Steven

gene heskett

unread,
Aug 23, 2011, 4:43:18 AM8/23/11
to pytho...@python.org
On Tuesday, August 23, 2011 04:42:04 AM Chris Angelico did opine:

> On Tue, Aug 23, 2011 at 8:21 AM, Paul Rubin <no.e...@nospam.invalid>
wrote:

> Yeah. Oh, I know when I've fixed the last bug in a program. It's the
> day the program gets deleted. Until then? Nope.
>
> ChrisA

OTOH, ChrisA, I have it on good authority that no program is ever finished,
until someone shoots the programmer. :)

Cheers, gene
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
<http://204.111.25.156:85/gene/>

Practice is the best of all instructors.
-- Publilius

Chris Angelico

unread,
Aug 23, 2011, 5:07:28 AM8/23/11
to pytho...@python.org
On Tue, Aug 23, 2011 at 9:43 AM, gene heskett <ghes...@wdtv.com> wrote:
> OTOH, ChrisA, I have it on good authority that no program is ever finished,
> until someone shoots the programmer.  :)
>

Correct, although I've had projects that were killed by changes to
requirements - such as my fantastic system for writing device drivers
that leveraged DEBUG.EXE to translate assembly code into machine code,
and a REXX script to handle jump labels and such. That project was
quite thoroughly finished on the day that I met nasm :)

But that's quite off-topic.

ChrisA

Paul Rubin

unread,
Aug 23, 2011, 1:43:35 PM8/23/11
to
gene heskett <ghes...@wdtv.com> writes:
> OTOH, ChrisA, I have it on good authority that no program is ever finished,
> until someone shoots the programmer. :)

The way I heard it was "software is never finished until the last user
is dead".

Chris Torek

unread,
Aug 31, 2011, 5:01:34 PM8/31/11
to
In article <mailman.286.13139563...@python.org>,

Terry Reedy <tjr...@udel.edu> wrote:
>I would expect that catching socket.error (or even IOError) should catch
>all of those.
>
>"exception socket.error
>A subclass of IOError ...

Except that, as Steven D'Aprano almost noted elsethread, it isn't
(a subclass of IOError -- the note was that it is not a subclass
of EnvironmentError). In 2.x anyway:

>>> import socket
>>> isinstance(socket.error, IOError)
False
>>> isinstance(socket.error, EnvironmentError)
False
>>>

(I just catch socket.error directly for this case.)

(I have also never been sure whether something is going to raise
an IOError or an OSError for various OS-related read or write
operation failures -- such as exceeding a resource limit, for
instance -- so most places that do I/O operations on OS files, I
catch both. Still, it sure would be nice to have a static analysis
tool that could answer questions about potential exceptions. :-) )
--
In-Real-Life: Chris Torek, Wind River Systems
Intel require I note that my opinions are not those of WRS or Intel
Salt Lake City, UT, USA (40�39.22'N, 111�50.29'W) +1 801 277 2603
email: gmail (figure it out) http://web.torek.net/torek/index.html

John Nagle

unread,
Sep 1, 2011, 12:22:23 AM9/1/11
to
On 8/21/2011 5:30 PM, Steven D'Aprano wrote:
> Chris Angelico wrote:
>
>
>> A new and surprising mode of network failure would be indicated by a
>> new subclass of IOError or EnvironmentError.
>
> /s/would/should/
>
> I don't see why you expect this, when *existing* network-related failures
> aren't:
>
>>>> import socket
>>>> issubclass(socket.error, EnvironmentError)
> False
>
> (Fortunately that specific example is fixed in Python 3.)

I think I reported that some years ago.

There were some other errors in the URL and SSL area that
weren't subclasses of EnvironmentError. It's also possible
to get UnicodeError from URL operations.

John Nagle

Stefan Krah

unread,
Sep 9, 2011, 5:41:18 PM9/9/11
to pytho...@python.org
Chris Torek <nos...@torek.net> wrote:
> (I have also never been sure whether something is going to raise
> an IOError or an OSError for various OS-related read or write
> operation failures -- such as exceeding a resource limit, for
> instance -- so most places that do I/O operations on OS files, I
> catch both. Still, it sure would be nice to have a static analysis
> tool that could answer questions about potential exceptions. :-) )

There is an effort to fix this:

http://www.python.org/dev/peps/pep-3151/


And an implementation ...

http://hg.python.org/features/pep-3151/


... together with a feature request:

http://bugs.python.org/issue12555


I think the whole concept makes a lot of sense and is really worth
taking a look at.


Stefan Krah


Nobody

unread,
Sep 10, 2011, 5:33:03 AM9/10/11
to
On Wed, 31 Aug 2011 21:01:34 +0000, Chris Torek wrote:

> Still, it sure would be nice to have a static analysis
> tool that could answer questions about potential exceptions. :-) )

That's an impossibility in a dynamic language.

If you call f.read() where f was passed in as a parameter, the exceptions
which f.read() may throw depend upon exactly what f is; there's no
guarantee that it will actually be a built-in file object, only that it
will have a .read() method (otherwise you will get an AttributeError).

Even if f was returned from open(), there's no guarantee that
__builtins__.open() won't have been replaced by the time of the call.

Peter Otten

unread,
Sep 10, 2011, 6:17:23 AM9/10/11
to pytho...@python.org
Chris Torek wrote:

> >>> import socket
> >>> isinstance(socket.error, IOError)
> False

Here you test if the socket.error *class* is an instance of IOError; this
would print True if IOError were socket.error's metaclass. However:

>>> isinstance(socket.error(), IOError)
True

or more directly:

>>> issubclass(socket.error, IOError)
True
>>> issubclass(socket.error, EnvironmentError)
True

This is a relatively recent change:

$ python2.5 -c'from socket import error; print issubclass(error, IOError),
issubclass(error, EnvironmentError)'
False False
$ python2.6 -c'from socket import error; print issubclass(error, IOError),
issubclass(error, EnvironmentError)'
True True

0 new messages