pytz and Python timezones

[Johannes Bauer]

Hi there,

first off, let me admit that I have a hard time comprehensively wrapping
my head around timezones. Everything around them is much more
complicated than it seems, IMO. That said, I'm trying to do things the
"right" way and stumbled upon some weird issue which I can't explain.
I'm unsure what is happening here. I try to create a localized timestamp
in the easiest possible way. So, intuitively, I did this:

datetime.datetime(2016,1,1,0,0,0,tzinfo=pytz.timezone("Europe/Berlin"))

Which gives me:

datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin'
LMT+0:53:00 STD>)

Uuuuuh... what?

This here:

pytz.timezone("Europe/Berlin").localize(datetime.datetime(2016,1,1))

Gives me the expected result of:

datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin'
CET+1:00:00 STD>)

Can someone explain what's going on here and why I end up with the weird
"00:53" timezone? Is this a bug or am I doing things wrong?

Thanks,
Cheers,
Johannes

--
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
- Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1...@speranza.aioe.org>

[Irmen de Jong]

On 11-6-2016 13:37, Johannes Bauer wrote:
> Hi there,
>
> first off, let me admit that I have a hard time comprehensively wrapping
> my head around timezones. Everything around them is much more
> complicated than it seems, IMO.

They might not seem complicated, but actually they are. Mindbogglingly so:
https://www.youtube.com/watch?v=-5wpm-gesOY

> pytz.timezone("Europe/Berlin").localize(datetime.datetime(2016,1,1))
>
> Gives me the expected result of:
>
> datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin'
> CET+1:00:00 STD>)
>
> Can someone explain what's going on here and why I end up with the weird
> "00:53" timezone? Is this a bug or am I doing things wrong?

I ran into the same issue a while ago and have since accepted that "the right way to do
it" is indeed not passing a tzinfo into the datetime constructor, but to always use the
second form with tz.localize(datetime).
I suppose it is a problem (not so much a bug) caused by the way timezones are done
internally in pytz and/or datetime but haven't looked into the details. Mostly because
of what is said the video I linked above :)

Irmen

[Lawrence D’Oliveiro]

On Saturday, June 11, 2016 at 11:37:38 PM UTC+12, Johannes Bauer wrote:
> I try to create a localized timestamp in the easiest possible way.

Localized timestamps are perhaps not as easy as you think.

> So, intuitively, I did this:
>
> datetime.datetime(2016,1,1,0,0,0,tzinfo=pytz.timezone("Europe/Berlin"))
>
> Which gives me:
>
> datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin'
> LMT+0:53:00 STD>)
>
> Uuuuuh... what?

A careful reading of <https://docs.python.org/3/library/datetime.html#datetime.datetime> indicates that those classes expect their attributes to already be in local time. I don’t think they have the smarts to examine the tzinfo you pass them and decide which of possibly several sections of historical information might apply to them. So you ended up with it being interpreted according to the oldest section of the “Europe/Berlin” timezone data (the one up to April 1893).

Anyway, that’s my guess.

> This here:
>
> pytz.timezone("Europe/Berlin").localize(datetime.datetime(2016,1,1))
>
> Gives me the expected result of:
>
> datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin'
> CET+1:00:00 STD>)

Clearly pytz has much more smarts about handling local times.

My general rule about dates/times is: always work in UTC. Convert to local dates/times only for display purposes.

[Carl Meyer]

Hi Johannes,

On 06/11/2016 05:37 AM, Johannes Bauer wrote:
> I try to create a localized timestamp
> in the easiest possible way. So, intuitively, I did this:
>
> datetime.datetime(2016,1,1,0,0,0,tzinfo=pytz.timezone("Europe/Berlin"))

That is indeed intuitive, but unfortunately (due to a misunderstanding
between the original authors of Python's datetime module and the author
of pytz about how timezone-aware datetimes should work in Python) it is
not correct. The correct way to create a localized datetime using pytz
is this:

tz = pytz.timezone('Europe/Berlin')
dt = tz.localize(datetime.datetime(2016, 1, 1, 0, 0, 0)

This is documented prominently in the pytz documentation:
http://pytz.sourceforge.net/

> Which gives me:
>
> datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin'
> LMT+0:53:00 STD>)
>
> Uuuuuh... what?

When you create a pytz timezone object, it encompasses all historical
UTC offsets that have ever been in effect in that location. When you
pass a datetime to the `localize()` method of that timezone object, it
is able to figure out which actual UTC offset was in effect at that
local time in that location, and apply the correct "version" of itself
to that datetime.

However, no such logic is built into the datetime module itself. So when
you just apply a pytz timezone directly to the tzinfo property of a
datetime, pytz by default falls back to the first entry in its
historical table of UTC offsets for that location. For most locations,
that is something called "LMT" or Local Mean Time, which is the
customary time in use at that location prior to the standardization of
timezones. And in most locations, LMT is offset from UTC by a strange
number of minutes. That's why you see "LMT" and the odd 53-minute offset
above.

> This here:
>
> pytz.timezone("Europe/Berlin").localize(datetime.datetime(2016,1,1))
>
> Gives me the expected result of:
>
> datetime.datetime(2016, 1, 1, 0, 0, tzinfo=<DstTzInfo 'Europe/Berlin'
> CET+1:00:00 STD>)
>
> Can someone explain what's going on here and why I end up with the weird
> "00:53" timezone? Is this a bug or am I doing things wrong?

It is not a bug in pytz or in datetime, in that it is intended behavior,
although that behavior is unfortunately obscure, bug-prone, and
little-understood.

If you are masochistic enough to want to understand how this bad
situation came to be, and what might be done about it, you can read
through PEPs 431 and 495.

Carl