PyMongo timezone transparency
Stuart Rackham
I'm using PyMongo+Django and finding the the recommended practice of
explicitly managing user timezone to UTC conversion at the app level
quite tedious. It would be nice if the driver could manage this
transparently.
My idea is that if a global timezone object (bson.tzinfo) is set then
it would be applied transparently to all naive datetimes read from and
written to the database i.e. naive datetimes in the app would behave
like local times in the user's timezone. If bson.tzinfo is not set or
is None then PyMongo reverts to default UTC behavior.
I implemented an ugly hack (proof of concept only) to illustrate what
I'm getting at. Here's some interactive output:
8<---------------------------------------------------
$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)�
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo, bson, datetime, pytz
>>> bson.tzinfo = None # Disable timezone processing.
>>> coll = pymongo.Connection().datetest.dates
>>>�
>>> # No timezone set (write and read UTC).
... coll.drop()
>>> coll.insert({'date': datetime.datetime.utcnow()})
ObjectId('4d759db84c08730b4e000000')
>>> coll.find_one()['date']
datetime.datetime(2011, 3, 8, 3, 8, 40, 573000)
>>>�
>>> # Enable timezone (write and read bson.tzinfo datetimes).
... bson.tzinfo = pytz.timezone('Pacific/Auckland')
>>> coll.find_one()['date']
datetime.datetime(2011, 3, 8, 16, 8, 40, 573000, tzinfo=<DstTzInfo
'Pacific/Auckland' NZDT+13:00:00 DST>)
>>>�
>>> coll.drop()
>>> coll.insert({'date': datetime.datetime.now()}) # Use local times.
ObjectId('4d759db84c08730b4e000001')
>>> coll.find_one()['date']
datetime.datetime(2011, 3, 8, 16, 8, 40, 581000, tzinfo=<DstTzInfo
'Pacific/Auckland' NZDT+13:00:00 DST>)
>>> bson.tzinfo = None # Disable timezone processing.
>>> coll.find_one()['date']
datetime.datetime(2011, 3, 8, 3, 8, 40, 581000)
>>>�
8<---------------------------------------------------
Is this a reasonable approach?
is it worth pursuing?
or have I missed something?
or is there an easier way?
Cheers, Stuart
--
Stuart Rackham
Here's the patch I made to bson/__init__.py (PyMongo version 1.9).
NOTE: This is not a patch submission, just a proof of concept hack.
It's buggy e.g. when used wih Django 1.3rc1 I encounted
datetime.datetime.now() generating the the wrong time -- problems in
the Python (non-C) bson implementation???
8<---------------------------------------------------
--- __init__.py.ORIG 2011-02-28 09:53:32.751812102 +1300
+++ __init__.py 2011-03-08 16:07:24.395865445 +1300
@@ -27,6 +27,7 @@
from bson.errors import (InvalidBSON,
InvalidDocument,
InvalidStringData)
+import bson
from bson.max_key import MaxKey
from bson.min_key import MinKey
from bson.objectid import ObjectId
@@ -37,7 +38,7 @@
�
try:
import _cbson
- _use_c = True
+ _use_c = False
except ImportError:
_use_c = False
�
@@ -140,7 +141,10 @@
�
def _get_date(data, as_class, tz_aware):
seconds = float(struct.unpack("<q", data[:8])[0]) / 1000.0
- if tz_aware:
+ tz = bson.__dict__.get('tzinfo')
+ if tz:
+ return (datetime.datetime.fromtimestamp(seconds, tz), data[8:])
+ elif tz_aware:
return (datetime.datetime.fromtimestamp(seconds, utc), data[8:])
return (datetime.datetime.utcfromtimestamp(seconds), data[8:])
�
@@ -304,6 +308,10 @@
return "\x12" + name + struct.pack("<q", value)
return "\x10" + name + struct.pack("<i", value)
if isinstance(value, datetime.datetime):
+ tz = bson.__dict__.get('tzinfo')
+ if not value.tzinfo and tz:
+ value = tz.localize(value)
+ value = value.astimezone(utc)
if value.utcoffset() is not None:
value = value - value.utcoffset()
millis = int(calendar.timegm(value.timetuple()) * 1000 +
8<---------------------------------------------------
Bernie Hackett
functionality into the driver. You can open an enhancement request in
jira (jira.mongodb.org) and we'll consider it for a future release.
> I encounted datetime.datetime.now() generating the the wrong time -- problems in
> the Python (non-C) bson implementation???
Maxim Penzin
> Hi
>
> I'm using PyMongo+Django and finding the the recommended practice of
> explicitly managing user timezone to UTC conversion at the app level
> quite tedious. It would be nice if the driver could manage this
> transparently.
It is very good idea!
Many Python developers like just to user datetime() and don't care
about timezones
how they did with MySQL or PostgreSQL.
When I use timestamp without time zone in PostgreSQL I silently assume
'my local time'
for timestap values.
But for MongoDB that assumption is wrong because UTC mentioned in spec.
Would be great to have a driver option that automagically converts
local time values to mongo's utc.
--
-- mpe...@gmail.com -- www.penzin.ru --
srackham
On Mar 9, 6:38 am, Bernie Hackett <ber...@10gen.com> wrote:
> This is a valid idea but I'm not sure we want to push this
> functionality into the driver. You can open an enhancement request in
> jira (jira.mongodb.org) and we'll consider it for a future release.
>
> > I encounted datetime.datetime.now() generating the the wrong time -- problems in
> > the Python (non-C) bson implementation???
>
> Can you give an example of the issue you ran into?
server's timezone.
Django uses this setting to ensure naive datetimes returned by
datetime.datetime.now() are in that timezone.
My patch seems to be working fine with Django now.
srackham
> functionality into the driver. You can open an enhancement request in
> jira (jira.mongodb.org) and we'll consider it for a future release.
having it in the driver.
Andrew Armstrong
'Kind' property that indicates whether the time it holds is specified
in local, universal or 'unknown' timezone.
This also lets you call myTime.ToLocalTime() / ToUniversalTime() to
retrun a copied date/time in the desired timezone (its a no-op if the
timezone is already the requested timezone).
This sort of thing would let you always call ToUniversalTime() before
storage, and ToLocalTime() while loading the object from the database?
Regards,
Andrew
srackham
> functionality into the driver. You can open an enhancement request in
> jira (jira.mongodb.org) and we'll consider it for a future release.
browse/PYTHON-222).