Issue 209 in couchdb-python: couchdb-python doesn't support locales
couchdb...@googlecode.com
Owner: ----
Labels: Type-Defect Priority-Medium
New issue 209 by emmanuel...@gmail.com: couchdb-python doesn't support
locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
The couchdb-python cache system is based on the RFC 2822 Date system for
object expiration. When the locale changes, strptime() is no more able to
parse this kind of date.
What steps will reproduce the problem?
1. Run this (or see attachment):
import locale
locale.setlocale(locale.LC_ALL, '')
import couchdb
import couchdb.http
# disable cache. 0 means force cache validation
# on every request.
couchdb.http.CACHE_SIZE = 10, 0
couch = couchdb.Server()
try:
db = couch.create('test')
db['foo'] = {'bar': 'test'}
print db['foo']
finally:
couch.delete('test')
What is the expected output? What do you see instead?
Expected:
<Document u'foo'@u'1-1d1b1ba13d97badec02a57adb942cd42' {u'bar': u'test'}>
Instead:
Traceback (most recent call last):
File "plop.py", line 15, in <module>
print db['foo']
File "c:\Pyramid\lib\site-packages\couchdb\client.py", line 322, in
__getitem__
_, _, data = self.resource.get_json(id)
File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 393, in get_json
status, headers, data = self.get(*a, **k)
File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 374, in get
return self._request('GET', path, headers=headers, **params)
File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 419, in _request
credentials=self.credentials)
File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 316, in request
self._clean_cache()
File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 324, in
_clean_cache
ls = sorted(self.cache.iteritems(), key=cache_sort)
File "c:\Pyramid\lib\site-packages\couchdb\http.py", line 84, in
cache_sort
t =
time.mktime(time.strptime(i[1][1]['Date'][5:-4], '%d %b %Y %H:%M:%S'))
File "c:\Python27\Lib\_strptime.py", line 454, in _strptime_time
return _strptime(data_string, format)[0]
File "c:\Python27\Lib\_strptime.py", line 325, in _strptime
(data_string, format))
ValueError: time data '21 Feb 2012 15:18:38' does not match
format '%d %b %Y %H:%M:%S'
What version of the product are you using? On what operating system?
- CouchDB-0.8-py2.7
- WinXP32 SP3
Please provide any additional information below.
IMHO the best solution is to remove the month part of the date (so as to
only have numbers which should be locale independant) before passing it to
strptime() (and to adjust the parse pattern accordingly)
Attachments:
plop.py 351 bytes
couchdb...@googlecode.com
Comment #1 on issue 209 by kxepal: couchdb-python doesn't support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
Looks like windows-only bug or I couldn't reproduce it on Linux system, but
could on Windows one.
There are two options: wait for @Matt http module and cache system
refactoring or quick fix current behavior. I'm for first one.
couchdb...@googlecode.com
Comment #2 on issue 209 by emmanuel...@gmail.com: couchdb-python doesn't
support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
Something is missing: the default locale, or the choosen locale shouldn't
be in english. Since the locale name is OS dependant (fr_FR on Linux,
French_France on Windows), I didn't include it in the example.
couchdb...@googlecode.com
Comment #3 on issue 209 by kxepal: couchdb-python doesn't support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
This is important note, thanks!
# Quick fix for your example:
import time
from email.Utils import parsedate
from datetime import datetime
def cache_sort(i):
t = time.mktime(parsedate(i[1][1]['Date']))
return datetime.fromtimestamp(t)
# And monkey-patch cache_sort function:
couchdb.http.cache_sort = cache_sort
This should work! At least for me with ('ru_RU', 'cp1251') locale info.
I'll prepare patch and tests later.
couchdb...@googlecode.com
Status: Accepted
Owner: kxepal
Labels: Milestone-0.9
Comment #4 on issue 209 by kxepal: couchdb-python doesn't support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
(No comment was entered for this change.)
couchdb...@googlecode.com
Comment #5 on issue 209 by kxepal: couchdb-python doesn't support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
Anyone has ideas how to write test case that depended on some non en_US
locale? If I'll set some ru_RU this test will pass for me, but not for
those who hasn't this locale enabled. Same thing is about fr_FR one that is
missed for me.
couchdb...@googlecode.com
Comment #6 on issue 209 by wickedg...@gmail.com: couchdb-python doesn't
support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
Maybe do some trickery with unittest.skip? Check for the presence of the
locale at module import time, and conditionally skip the test?
couchdb...@googlecode.com
Comment #7 on issue 209 by kxepal: couchdb-python doesn't support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
unittest.skip will bring unittest2 dependency which couldn't help with
prevent problem in future. Also it's impossible to check locale because it
could not be available - as for me I have ('ru_RU', 'cp1251') while 'fr_FR'
is missed for me.
There are two solutions:
1. Make tests for Windows only against ANSI encoding since it brings a
problem, while Linux long ago has *.UTF-8 system encoding by default.
2. Since this bug dependent on OS environment AND processed Date value
follows well known rules[1][2] it will be right decision to use right
ways[3][4][4] to parse such value. Tests are redundant in this case since
there is actually nothing to test - Python functions are well tested by his
community and bug occurs not because of couchdb-python logic code.
As for me, second variant is better because:
- it follows RFC like parsed value does;
- it is not locale dependent;
- I don't see any portable and stable ways to test locale specific things
without providing some dependency overhead (babel and mock may help there);
Patch attached. Thoughts?
[1]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.18
[2]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.3.1
[3]: http://docs.python.org/library/email.util.html#email.utils.parsedate
[4]:
http://stackoverflow.com/questions/1471987/how-do-i-parse-an-http-date-string-in-python
[5]:
http://stackoverflow.com/questions/225086/rfc-1123-date-representation-in-python
Attachments:
couchdb-python-209-r1.patch 882 bytes
couchdb...@googlecode.com
Status: Fixed
Comment #8 on issue 209 by kxepal: couchdb-python doesn't support locales
http://code.google.com/p/couchdb-python/issues/detail?id=209
This issue was closed by revision b7ada64cd590.