Ideal behavior for characters outside an encoding?
Tom Miller
I'm wondering what should be the proper behavior when characters try to get encoded by an encoding that has no idea what the character is. For example, many Windows XP users do not have a UTF-8 encoding set as their default, so when they list their contact that has cyrillic characters, GoogleCL must do one (or two) of the following things:
1) Cough and die (less than ideal)
2) Drop the contact from the list
3) Print the contact, omitting the offensive character(s)
4) Print the contact, replacing offensive character(s) with something (usually 1-4 '?'s, for each offensive character)
5) Print the contact, replacing offensive character(s) with their unicode "code points"
and/or
6) Print a warning message
I just pushed a change that does 5 and 6 (r425). What do YOU think should happen?
- Tom
Ferran Busquets
As a native Catalan (ISO-8859-1 charset) speaker I often hit this problem and currently I've hit an issue running GoogleCL:
$ google picasa post --title "ç" --src "x.jpg" Just implemented options.src and options.out, which CANNOT take globbed expressions or multiple strings! Bear with me.
(That is what you get for being so bleeding edge.)
Traceback (most recent call last):
File "/usr/bin/google", line 818, in <module>
main()
File "/usr/bin/google", line 804, in main
run_once(options, args)
File "/usr/bin/google", line 586, in run_once
task.run(client, options, args)
File "/usr/bin/googlecl/picasa/service.py", line 384, in _run_post
user=options.owner or options.user)
File "/usr/bin/googlecl/picasa/service.py", line 233, in insert_media_list
LOG.info('Loading file ' + path + ' to album ' + album.title.text)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)
About your options, I think that the "offensive" characters have to be shown, and a warning is not needed because is not an error/problem. I suppose you are a native English speaker. As an example to understand what means for an extended character user, image an application displaying "?" instead of a "k", for instance.
I think all the strings in a Python application have to be unicode because it covers the wider character representation. Also for compatibility, in Python 3.x all the text will be
unicode (http://docs.python.org/release/3.0.1/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit).
If some users cannot show correctly some characters may be is because they have some charset misconfiguration.
I hope this helps,
Ferran