Confusing datetime.datetime

[Damjan]

I've been struggling with an app that uses
Postgresql/Psycopg2/SQLAlchemy and I've come to this confusing
behaviour of datetime.datetime.


First of all, the "Seconds since Epoch" timestamps are always in UTC, so
shouldn't change with timezones. So I'd expect that a round trip of a
timestamp through datetime.datetime, shouldn't change it.


Now, all is good when I use a naive datetime.datetime


-- TZ=UTC python
>>> from datetime import datetime
>>> dt = datetime.fromtimestamp(1341446400)
>>> dt
datetime.datetime(2012, 7, 5, 0, 0)
>>> dt.strftime('%s')
'1341446400'


-- TZ=Asia/Tokyo python
>>> from datetime import datetime
>>> dt = datetime.fromtimestamp(1341446400)
>>> dt
datetime.datetime(2012, 7, 5, 9, 0)
>>> dt.strftime('%s')
'1341446400'



But when I use an timezone aware datetime.datetime objects, the
timestamp roundtrip is destroyed. I get 2 different timestamps.
Am I missing something here, I've been reading the datetime
documentation several times, but I can't understand what is the intended
behaviour.


-- TZ=UTC python
>>> from datetime import datetime
>>> import pytz
>>> tz = pytz.timezone('Europe/Skopje')
>>> dt = datetime.fromtimestamp(1341446400, tz)
>>> dt
datetime.datetime(2012, 7, 5, 2, 0, tzinfo=<DstTzInfo 'Europe/Skopje'
CEST+2:00:00 DST>)
>>> dt.strftime('%s')
'1341453600'


-- TZ=Asia/Tokyo python
>>> from datetime import datetime
>>> import pytz
>>> tz = pytz.timezone('Europe/Skopje')
>>> dt = datetime.fromtimestamp(1341446400, tz)
>>> dt
datetime.datetime(2012, 7, 5, 2, 0, tzinfo=<DstTzInfo 'Europe/Skopje'
CEST+2:00:00 DST>)
>>> dt.strftime('%s')
'1341421200'



Python 2.7.3, pytz 2012c

--
damjan

[Damjan]

On 05.07.2012 16:10, Damjan wrote:
> I've been struggling with an app that uses
> Postgresql/Psycopg2/SQLAlchemy and I've come to this confusing
> behaviour of datetime.datetime.

Also this:

#! /usr/bin/python2
# retardations in python's datetime

import pytz
TZ = pytz.timezone('Europe/Skopje')

from datetime import datetime

x1 = datetime.now(tz=TZ)
x2 = datetime(x1.year, x1.month, x1.day, tzinfo=TZ)

assert x1.tzinfo == x2.tzinfo


WHY does the assert throw an error???

[Steven D'Aprano]

On Fri, 06 Jul 2012 00:55:48 +0200, Damjan wrote:

> Also this:
>
> #! /usr/bin/python2
> # retardations in python's datetime
>
> import pytz
> TZ = pytz.timezone('Europe/Skopje')
>
> from datetime import datetime
>
> x1 = datetime.now(tz=TZ)
> x2 = datetime(x1.year, x1.month, x1.day, tzinfo=TZ)
>
> assert x1.tzinfo == x2.tzinfo
>
>
> WHY does the assert throw an error???

I don't have pytz, so I can't test your exact code. But using my own time
zone class, it seems to work fine in Python 2.5, 2.6 and 2.7:

from datetime import datetime, timedelta, tzinfo
ZERO = timedelta(0)
HOUR = timedelta(hours=1)

class UTC(tzinfo):
def utcoffset(self, dt):
return ZERO
def tzname(self, dt):
return "UTC"
def dst(self, dt):
return ZERO

utc = UTC()
t1 = datetime.now(tz=utc)
t2 = datetime(t1.year, t1.month, t1.day, tzinfo=utc)
assert t1.tzinfo == t2.tzinfo


No assertion error at all.

This makes me think that the "retardation" as you put it is not in
Python's datetime module at all, but in pytz.

What does TZ == TZ give? If it returns False, I recommend you report it
as a bug against the pytz module.


--
Steven

[Damjan]

> from datetime import datetime, timedelta, tzinfo
> ZERO = timedelta(0)
> HOUR = timedelta(hours=1)
>
> class UTC(tzinfo):
> def utcoffset(self, dt):
> return ZERO
> def tzname(self, dt):
> return "UTC"
> def dst(self, dt):
> return ZERO
>
> utc = UTC()
> t1 = datetime.now(tz=utc)
> t2 = datetime(t1.year, t1.month, t1.day, tzinfo=utc)
> assert t1.tzinfo == t2.tzinfo
>
>
> No assertion error at all.
>
> This makes me think that the "retardation" as you put it is not in
> Python's datetime module at all, but in pytz.
>
> What does TZ == TZ give? If it returns False, I recommend you report it
> as a bug against the pytz module.

It returns True, so it seems to be changed in the datetime object??

I tried both 2.7 and 3.2

--
damjan

[Hans Mulder]

Because x1 and x2 have different time zones.

The tzinfo field in x2 is equal to TZ and has a UTC offset of 1 hour.
The tzinfo field in x1 contains the DST version of that timezone,
with a UTC offset of 2 hours, because Skopje is currently on DST.

I think you want:

x2 = TZ.localize(datetime(x1.year, x1.month, x1.day))

That produces a datetime with the year, month and day set as indicated
and tzinfo set to the correct UTC offset for that date, at 00:00 hours.

Or maybe you need:

x2 = TZ.localize(datetime(x1.year, x1.month, x1.day, 12))
x2 = x2.replace(hour=0)

That determines whether DST should be on at noon, and then resets the
hour field to zero. This produces the same outcome as the one liner,
except on days when DST is switched on or off.


Hope this helps,

-- HansM

[Damjan]

> Because x1 and x2 have different time zones.
>
> The tzinfo field in x2 is equal to TZ and has a UTC offset of 1 hour.
> The tzinfo field in x1 contains the DST version of that timezone,
> with a UTC offset of 2 hours, because Skopje is currently on DST.
>
> I think you want:
>
> x2 = TZ.localize(datetime(x1.year, x1.month, x1.day))
>
> That produces a datetime with the year, month and day set as indicated
> and tzinfo set to the correct UTC offset for that date, at 00:00 hours.
>
> Or maybe you need:
>
> x2 = TZ.localize(datetime(x1.year, x1.month, x1.day, 12))
> x2 = x2.replace(hour=0)
>
> That determines whether DST should be on at noon, and then resets the
> hour field to zero. This produces the same outcome as the one liner,
> except on days when DST is switched on or off.

Thanks, I think this will help me.

Although these issues seem very much underdocumented in the datetime
documentation. Are there any good references of using good times in Python?



--
damjan