Issue 360 in googlecl: Encoding issues when running without terminal
goog...@googlecode.com
Owner: ----
Labels: Type-Defect Priority-Medium
New issue 360 by eckho...@gmail.com: Encoding issues when running without
terminal
http://code.google.com/p/googlecl/issues/detail?id=360
What steps will reproduce the problem?
1. Create a contact containing a non-ascii character (e.g. "Jörg")
2. Add cronjob "googlecl contacts list --fields='name' --title=''"
What is the expected output? What do you see instead?
googlecl should use the character encoding from the environment (utf-8),
instead it uses sys.getdefaultencoding() which defaults to ascii and
results in \x encoded strings.
What version of the product are you using? On what operating system? What
version of gdata-python-client (aka python-gdata)?
googlecl 0.9.12-1 from .deb on Debian squeeze, python-gdata 2.0.8-1.1.
Please provide any additional information below.
Replacing the line return_enc = sys.getdefaultencoding() in __init__.py by
something like return_enc = 'utf-8' yields exspected results. I'm no python
guru so I don't know which function call would result in the encoding that
e.g. locale knows about.
However, if everything else fails I would welcome a config variable to be
used ("fallback_terminal_encoding"?) in case the encoding can not be
automatically determined.
goog...@googlecode.com
Status: Feedback
Comment #1 on issue 360 by tom.h.mi...@gmail.com: Encoding issues when
running without terminal
http://code.google.com/p/googlecl/issues/detail?id=360
I just went right for the config file :)
Try applying this patch and adding an option called "default_encoding =
utf-8" to the GENERAL section of your config file
(~/.config/googlecl/config). Please let me know if it does or does not work.
Attachments:
encoding_from_config.patch 1.0 KB
goog...@googlecode.com
Comment #2 on issue 360 by eckho...@gmail.com: Encoding issues when running
without terminal
http://code.google.com/p/googlecl/issues/detail?id=360
Unfortunately, there appears to be something wrong with that patch:
Traceback (most recent call last):
File "/usr/bin/google", line 54, in <module>
import googlecl
File "/usr/lib/pymodules/python2.6/googlecl/__init__.py", line 70, in
<module>
TERMINAL_ENCODING = determine_terminal_encoding()
File "/usr/lib/pymodules/python2.6/googlecl/__init__.py", line 52, in
determine_terminal_encoding
return_enc = googlecl.config.parser.safe_get('GENERAL',
AttributeError: 'module' object has no attribute 'safe_get'
goog...@googlecode.com
Comment #3 on issue 360 by tom.h.mi...@gmail.com: Encoding issues when
http://code.google.com/p/googlecl/issues/detail?id=360
Whoops. This patch (applied to 0.9.12) should actually fix the problem. If
you applied the patch from comment 1, you'll have to remove it.
There may be some garbled characters as GoogleCL gets started, but the
output you want from the cron job should come through.
Again, let me know if something's wrong. Thanks for the quick report
turnaround!
Attachments:
encoding_from_config2.patch 2.1 KB
goog...@googlecode.com
Comment #4 on issue 360 by eckho...@gmail.com: Encoding issues when running
without terminal
http://code.google.com/p/googlecl/issues/detail?id=360
Hey. Thanks again for your effort. However, it seems when
determine_terminal_encoding is called on module initialization (without
config) something goes wrong now.
Traceback (most recent call last):
File "/usr/bin/google", line 54, in <module>
import googlecl
File "/usr/lib/pymodules/python2.6/googlecl/__init__.py", line 75, in
<module>
TERMINAL_ENCODING = determine_terminal_encoding()
File "/usr/lib/pymodules/python2.6/googlecl/__init__.py", line 55, in
determine_terminal_encoding
if not return_enc:
UnboundLocalError: local variable 'return_enc' referenced before assignment
When I add "return_enc = 'utf-8'" before "if config is not None" (to see
what happens when the function is called the second time) I get this:
Traceback (most recent call last):
File "/usr/bin/google", line 839, in <module>
main()
File "/usr/bin/google", line 825, in main
run_once(options, args)
File "/usr/bin/google", line 530, in run_once
options.config)
File "/usr/bin/google", line 354, in import_service
config = googlecl.config.load_configuration(config_file_path)
File "/usr/lib/pymodules/python2.6/googlecl/config/__init__.py", line 89,
in load_configuration
googlecl.TERMINAL_ENCODING =
googlecl.determine_terminal_encoding(config)
File "/usr/lib/pymodules/python2.6/googlecl/__init__.py", line 54, in
determine_terminal_encoding
return_enc = config.parser.safe_get('GENERAL',
AttributeError: ConfigParser instance has no attribute 'safe_get'
goog...@googlecode.com
Comment #5 on issue 360 by tom.h.mi...@gmail.com: Encoding issues when
http://code.google.com/p/googlecl/issues/detail?id=360
Thanks for your patience with this. I've heard the third time is the charm,
so here's the third version of that patch (again applied to 0.9.12
directly).
(Your instinct with return_enc was right, but unfortunately I had a typo
with the config code, also fixed in this patch)
Attachments:
encoding_from_config3.patch 2.0 KB
goog...@googlecode.com
Comment #6 on issue 360 by eckho...@gmail.com: Encoding issues when running
without terminal
http://code.google.com/p/googlecl/issues/detail?id=360
Your latest patch indeed fixes the issue. Thanks.
goog...@googlecode.com
Status: Fixed
Comment #7 on issue 360 by tom.h.mi...@gmail.com: Encoding issues when
running without terminal
http://code.google.com/p/googlecl/issues/detail?id=360
Finally! Sorry for all the botched attempts.
Closed by r542.
goog...@googlecode.com
Comment #8 on issue 360 by bloghe...@gmail.com: Encoding issues when
running without terminal
https://code.google.com/p/googlecl/issues/detail?id=360
I have similar UTF8 youtube upload Encoding issues when running without
terminal (running from cron)
sudo patch -p0 -i encoding_from_config3.patch complains as follows (current
__init__.py attached)
patching file googlecl/config/__init__.py
Reversed (or previously applied) patch detected! Assume -R? [n]
I am running a slightly later version of python on ubuntu 12.04 with
/usr/lib/pymodules/python2.7
Attachments:
__init__.py 8.8 KB
--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings
goog...@googlecode.com
Comment #9 on issue 360 by bloghe...@gmail.com: Encoding issues when
Please help as I have the same googlecl.SafeDecodeError trouble when
running from cron (works perfectly on the Command Line). My goal is to add
UTF-8 characters to youtube uploads (no trouble with plain ascii). I am
using what I believe is the latest google version 0.9.13 and python2.7 on
ubuntu 14.04 (an Azure VM if it matters).
Traceback (most recent call last):
main()
File "/usr/bin/google", line 835, in main
run_once(options, args)
File "/usr/bin/google", line 588, in run_once
setattr(options, attr_name, safe_decode(attr,
googlecl.TERMINAL_ENCODING))
File "/usr/lib/pymodules/python2.7/googlecl/__init__.py", line 264, in
safe_decode
repr(string))
googlecl.SafeDecodeError: ascii could not
decode '\xe2\x80\xa2\xd0\xbc\xcf\x85\xd1\x95\xce\xb9c\xe2\x80\xa2....
Here is what I have tried:
1) Compared the encoding_from_config3.patch with __init__.py and it looks
to me that your patch change apparently needed for 0.9.13 is already in
version 0.9.13
A sudo patch -p0 -i encoding_from_config3.patch complains gives:
Reversed (or previously applied) patch detected! Assume -R? [n]
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
3) Added to the crontab
LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8
goog...@googlecode.com
Comment #10 on issue 360 by bloghe...@gmail.com: Encoding issues when
https://code.google.com/p/googlecl/issues/detail?id=360
Please help as I have the same googlecl.SafeDecodeError trouble when
running from cron (works perfectly on the Command Line). My goal is to add
UTF-8 characters to youtube uploads (no trouble with plain ascii). I am
using what I believe is the latest google version 0.9.13 and python2.7 on
ubuntu 14.04 (an Azure VM if it matters).
Traceback (most recent call last):
File "/usr/bin/google", line 849, in <module>
main()
File "/usr/bin/google", line 835, in main
run_once(options, args)
File "/usr/bin/google", line 588, in run_once
setattr(options, attr_name, safe_decode(attr,
googlecl.TERMINAL_ENCODING))
File "/usr/lib/pymodules/python2.7/googlecl/__init__.py", line 264, in
safe_decode
repr(string))
googlecl.SafeDecodeError: ascii could not
decode '\xe2\x80\xa2\xd0\xbc\xcf\x85\xd1\x95\xce\xb9c\xe2\x80\xa2....
Here is what I have tried:
1) Compared the encoding_from_config3.patch with __init__.py and it looks