exceptions and unicode

[Stuart McGraw]

I am having a problem with exceptions and unicode.

try: open ('file.txt')
except IOError, e: pass
str (e)
=> "[Errno 2] No such file or directory: 'file.txt'"

which is fine but...

try: open (u'フィイル.txt')
except IOError, e: pass
str (e)
=> "[Errno 2] No such file or directory: u'\\u30d5\\u30a3\\u30a4\\u30eb.txt'"

ok, I did ask for a str string so no reason I should expect a unicode
string, but then (in Python 2.6)...

unicode (e)
=> u"(2, 'No such file or directory')"

i.e. no formatting at all, or in Python 2.6.5

unicode (e)
=> u"[Errno 2] No such file or directory: u'\\u30d5\\u30a3\\u30a4\\u30eb.txt'"

i.e. the result is a unicode string in name only. :-(

What I was expecting (or at least hoping for) of course was

=> u"[Errno 2] No such file or directory: '\u30d5\u30a3\u30a4\u30eb.txt'"

which when printed would produce something useful

=> [Errno 2] No such file or directory: 'フィイル.txt'

So how do I get what I want? (Python 3.x is not an option.)
Note that the exceptions may be anything (I just used IOError
as an example) and are generated in bowels of an API that I
can't/won't mess with.

[Thomas Jollans]

On 06/16/2010 10:10 PM, Stuart McGraw wrote:
> I am having a problem with exceptions and unicode.
>
> try: open ('file.txt')
> except IOError, e: pass
> str (e)
> => "[Errno 2] No such file or directory: 'file.txt'"
>
> which is fine but...
>
> try: open (u'フィイル.txt')
> except IOError, e: pass
> str (e)
> => "[Errno 2] No such file or directory: u'\\u30d5\\u30a3\\u30a4\\u30eb.txt'"
>
> ok, I did ask for a str string so no reason I should expect a unicode
> string, but then (in Python 2.6)...
>
> unicode (e)
> => u"(2, 'No such file or directory')"
>
> i.e. no formatting at all, or in Python 2.6.5
>
> unicode (e)
> => u"[Errno 2] No such file or directory: u'\\u30d5\\u30a3\\u30a4\\u30eb.txt'"

it looks like IOError simply calls repr(self.filename) to create an
error message.

e.filename is, however, still a unicode object, so you could access that
directly, and format it yourself. A simple approach to get the same
message would be
u"[Errno {0.errno}] {0.strerror}: '{0.filename}'".format(e)

but it looks like the standard exceptions use repr (barely surprising),
and in Python 2, repr is as much of a arse about unicode as the rest of
the language.

> (Python 3.x is not an option.)

If it were, it'd be a good one though ;-)

> Note that the exceptions may be anything (I just used IOError
> as an example) and are generated in bowels of an API that I
> can't/won't mess with.

Yeah, well, you'd have to special-case every single exception type that
gets unicode arguments, as they probably all treat them in the same
ungrateful way.

Thomas

[Martin v. Loewis]

> So how do I get what I want?

Submit a patch. You would have to explain why this is a bug fix and not
a new feature, as new features are not allowed anymore for 2.x.

Regards,
Martin

[Stuart McGraw]

Thanks. Actually I have no idea if this is a bug or a feature
(despite reading bug tracker issues 2517 and 6108, most of which
I did not understand) so I'm not in a position to argue either.
What I think I'll do is note in my documentation that the unreadable
error messages are from Python and are only temporary for a couple
years until the app can move to Python 3.

[Stuart McGraw]

Unfortunate. In general it does not seem possible to know what
exceptions could be generated (they could be custom exceptions
defined in the api) without examining all the code.

Seems like I'll have to live with it or try some grotesque hack
like looking for a repr-of-a-unicode-string-like text and converting
back to unicode.

Thanks.