Parsing time with timezones seem to fail

[Thomas Casteleyn]

Hi, I originally asked on Gophers slack where they directed me to this group.

It seems I'm not able to parse these 2 timestamps with timezone correctly: https://go.dev/play/p/VZwD29701ps

The responses show confusing time formats and the playground even seems to be more wrong than on my local machine.

How should I correctly parse both timestamps in Go?

[Ian Lance Taylor]

It's not clear to me what you are actually trying to do.
ParseInLocation will look for a timezone name like CET or MST relative
to the location that you give it. This approach is used because
identifiers like CET or MST are ambiguous. So in general it's odd to
use ParseInLocation with an arbitrary timezone identifier. What is
your actual goal?

Ian

[Thomas Casteleyn]

Being able to parse these timestamps correctly and produce correct Unix time from them;

Together with the Gophers slack, we found this ugly, but working hack: https://go.dev/play/p/nG-M0pUrm0Z

stamp, _ := time.Parse(layout, v)

loc, _ := time.LoadLocation(stamp.Location().String())

stamp, _ = time.ParseInLocation(layout, v, loc)

glibc is doing this better IMHO.

[Sven Rebhan]

The issue here is that the timezone information is ignored when computing the timestamp value. Take the following examples

layout :=  "2006-01-02 15:04:05 MST"

mytimeMST, err := time.Parse(layout, "2023-02-09 02:55:00 MST")

mytimeCET, err :=  time.Parse(layout, "2023-02-09 10:55:00 CET")

Both `mytimeMST` and `mytimeCET` refer to the exactly same point-in-time (i.e. 2023-02-09 09:55:00 UTC) with Unix timestamp

1675936500. However, when you run the code (see playground example) the result is

mytimeMST:     2023-02-09 02:55:00 +0000 MST    which is timestamp 1675911300

mytimeCET:     2023-02-09 10:55:00 +0000 CET    which is timestamp 1675940100

This is totally unexpected as, according to the documentation of `time.Parse`

When parsing a time with a zone abbreviation like MST, if the zone abbreviation has a defined offset in the current location, then that offset is used. The zone abbreviation "UTC" is recognized as UTC regardless of location. If the zone abbreviation is unknown, Parse records the time as being in a fabricated location with the given zone abbreviation and a zero offset. [...]

Obviously, the `time.Parse` function treats MST and CET as unknown abbreviations,  however, when using those abbrevs in

other locations (e.g. in `time.LoadLocation`) they are perfectly known with a defined offset in the current location.

So my expectation (and I guess also Thomas') would be that both `mytimeMST` and `mytimeCET` result in timestamp 1675936500!

Either this or clarify the documentation that the timezone abbrev is never taken into account and had to be manually treated.

Best regards,

    Sven

[Ian Lance Taylor]

On Fri, Feb 10, 2023 at 9:04 AM Sven Rebhan <sre...@influxdata.com> wrote:
>
> The issue here is that the timezone information is ignored when computing the timestamp value. Take the following examples
>
> layout := "2006-01-02 15:04:05 MST"
> mytimeMST, err := time.Parse(layout, "2023-02-09 02:55:00 MST")
> mytimeCET, err := time.Parse(layout, "2023-02-09 10:55:00 CET")
>
> Both `mytimeMST` and `mytimeCET` refer to the exactly same point-in-time (i.e. 2023-02-09 09:55:00 UTC) with Unix timestamp
> 1675936500. However, when you run the code (see playground example) the result is
>
> mytimeMST: 2023-02-09 02:55:00 +0000 MST which is timestamp 1675911300
> mytimeCET: 2023-02-09 10:55:00 +0000 CET which is timestamp 1675940100
>
> This is totally unexpected as, according to the documentation of `time.Parse`
>
> When parsing a time with a zone abbreviation like MST, if the zone abbreviation has a defined offset in the current location, then that offset is used. The zone abbreviation "UTC" is recognized as UTC regardless of location. If the zone abbreviation is unknown, Parse records the time as being in a fabricated location with the given zone abbreviation and a zero offset. [...]
>
> Obviously, the `time.Parse` function treats MST and CET as unknown abbreviations, however, when using those abbrevs in
> other locations (e.g. in `time.LoadLocation`) they are perfectly known with a defined offset in the current location.
>
> So my expectation (and I guess also Thomas') would be that both `mytimeMST` and `mytimeCET` result in timestamp 1675936500!
> Either this or clarify the documentation that the timezone abbrev is never taken into account and had to be manually treated.

But it's not the case that the timezone abbreviation is never taken
into account. As the comment says, the timezone abbreviation is taken
into account if it is defined for the current location. For example,
if the current location is America/Boise, then MST is meaningful (and
so is MDT). if the current location is Europe/Brusselss, then CET is
meaningful (and so is CEST among others). This approach is taken
because the timezone abbreviations are ambiguous.

Ian

[Jason E. Aten]

I'll add more strongly proscriptive advice on dealing with timestamps. 

Personally I hate code that will do different things depending

on where in the world (which computer, different notion of "Local") it is run on. It

makes reproducing issues hellish/very hard. So I strongly recommend:

1. Don't use ambiguous timezone indications like MST in your timestamps, use numerical offsets from UTC, like -0700 instead.

2.  Always have a numerical timezone offset on your timestamp, never leave it blank.

3. If you have to deal with a blank timezone, because some idiot has failed follow rules 1 and 2, that is the only time you should be using time.ParseInLocation().

4. Otherwise, you should be using time.Parse(), with your numerical offset in the string being parsed. This means use a format like 

const layoutTZ = "2006-01-02 15:04:05 -0700"

These rules will make your life dealing with timezones in timestamps sane.

Given those guidelines, there are two ways of dealing with the ambiguous (poor, bad idea) timestamp "2023-02-09 02:55:00 MST". These

are both illustrated here

https://go.dev/play/p/0dU7an_bctP 

a. Truncate off the ambiguous timezone indication, and then use ParseInLocation while supplying a location like America/Denver:

    denver, err := time.LoadLocation("America/Denver")
    panicOn(err)
    testParseLocation("2023-02-09 02:55:00", denver)

b. Replace the ambiguity with an unambiguous numerical offset before parsing:

    testParse("2023-02-09 02:55:00 -0700")

[Sven Rebhan]

On Friday, February 10, 2023 at 7:45:37 PM UTC+1 Ian Lance Taylor wrote:

But it's not the case that the timezone abbreviation is never taken
into account. As the comment says, the timezone abbreviation is taken
into account if it is defined for the current location. For example,
if the current location is America/Boise, then MST is meaningful (and
so is MDT). if the current location is Europe/Brusselss, then CET is
meaningful (and so is CEST among others). This approach is taken
because the timezone abbreviations are ambiguous.

I do not yet see where timezone abbreviations are ambiguous... Do you mean there are multiple timezones with the same abbrev? This is certainly not the case for MST...

What is the point in taking the timezone into account only if that timezone is in the local timezone? I really wish there would be a way to handle abbreviations exactly

the same as numeric offsets as the offset to UTC should be known for the abbreviations, shouldn't they? 

[Sven Rebhan]

Hey Jason E. Aten,

while I fully agree that using abbreviations is suboptimal, we have to deal with them in projects like 

influxdata/telegraf: The plugin-driven server agent for collecting & reporting metrics. (github.com) as users

potentially query all sort of services outside our responsibility. Those services do weird things like sending 

timestamps with timezone abbreviations. ;-)

The current behavior of `time.Parse` is very cumbersome as we probably always need to work-around it by 

stamp, _ := time.Parse(layout, v)

loc, _ := time.LoadLocation(stamp.Location().String())

stamp, _ = time.ParseInLocation(layout, v, loc)

while I would expect that parsing `"2023-02-09 02:55:00 MST"` with `time.Parse` would return the correct

time located in UTC.... And I don't see how that time is ambiguous. Sorry.

[Sam Vilain]

On Feb 13, 2023, at 9:55 AM, Sven Rebhan <sre...@influxdata.com> wrote:

I do not yet see where timezone abbreviations are ambiguous... Do you mean there are multiple timezones with the same abbrev? This is certainly not the case for MST...

What is the point in taking the timezone into account only if that timezone is in the local timezone? I really wish there would be a way to handle abbreviations exactly

the same as numeric offsets as the offset to UTC should be known for the abbreviations, shouldn't they? 

Yeah, there’s quite a few.  The first time I looked at Olson there was only some overlap between Australia and the US (both claiming “Eastern Time” as their own territory, which as British Kiwi I found somewhat amusing).

CDT could be Cuba (-4) or Central (-5) Daylight Time

EST could be New York (-5) or Sydney (+10)

AST could be Bermuda (-4) or Bahrain (+3)

See more for yourself on

Sam

[Thomas Casteleyn]

Hi Sam,

While I do acknowledge there might be some timezones that are ambiguous, it certainly is not the case for CET and MST. IMHO those should be able to be correctly parsed by time.Parse and time.ParseInLocation. If provided an ambiguous timezone to time.Parse, that should maybe return an error instead? time.ParseInLocation should always be able to provide something meaningful.

[Ian Lance Taylor]

On Mon, Feb 13, 2023 at 7:55 AM 'Thomas Casteleyn' via golang-nuts <golan...@googlegroups.com> wrote:

Hi Sam,

While I do acknowledge there might be some timezones that are ambiguous, it certainly is not the case for CET and MST. IMHO those should be able to be correctly parsed by time.Parse and time.ParseInLocation. If provided an ambiguous timezone to time.Parse, that should maybe return an error instead? time.ParseInLocation should always be able to provide something meaningful.

The exact definitions and meanings of timezone abbreviations are not standardized.  In practice on Unix systems they depend on the specific contents of the tzdata files on your local system.  And new abbreviations are introduced from time to time and old one are retired.

We aren't going to change the standard library time package as you suggest.  If you absolutely must use timezone abbreviations, as opposed to the standardized and well understand hour offsets recommended earlier, then I suggest that you write a preparser to convert your recognized timezone abbreviations into hour offsets.  Sorry this isn't very helpful, but it seems like the only portable and consistent approach that we can take.

Ian

 

On Monday, February 13, 2023 at 4:13:37 PM UTC+1 Sam Vilain wrote:

On Feb 13, 2023, at 9:55 AM, Sven Rebhan <sre...@influxdata.com> wrote:

I do not yet see where timezone abbreviations are ambiguous... Do you mean there are multiple timezones with the same abbrev? This is certainly not the case for MST...

What is the point in taking the timezone into account only if that timezone is in the local timezone? I really wish there would be a way to handle abbreviations exactly

the same as numeric offsets as the offset to UTC should be known for the abbreviations, shouldn't they? 

Yeah, there’s quite a few.  The first time I looked at Olson there was only some overlap between Australia and the US (both claiming “Eastern Time” as their own territory, which as British Kiwi I found somewhat amusing).

CDT could be Cuba (-4) or Central (-5) Daylight Time

EST could be New York (-5) or Sydney (+10)

AST could be Bermuda (-4) or Bahrain (+3)

See more for yourself on

Sam

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1e2e9e76-ee5d-446c-94bd-53f5896e16f0n%40googlegroups.com.

[Emrah Mutlu]

Using location from stamp 2023-02-09 10:55:00 +0100 CET is unix 1675936500 (true) 2023-02-09 02:55:00 -0700 MST is unix 1675936500 (true) Using time.Parse 2023-02-09 10:55:00 +0000 CET is unix 1675940100 (false) 2023-02-09 02:55:00 +0000 MST is unix 1675911300 (false) As reference 2023-02-09 09:55:00 +0000 GMT is unix 1675936500 (true) Program exited.