Calculating epoch time having local time, timezone and DST flag

[András Szilárd]

Hi all,

we have a use case, where we need to calculate epoch time having local time with the timezone and we know whether this local time is during DST or not.

I figured out to use below logic, which is interesting in the DST transition time period.

My questions:

 - Did I miss any helper function in the library API to make this easier/simpler?

 - If there is no such function, than is below logic correct?

 - Would it make sense to put something like this in the library API?

   Motivation: there would be no need to come up with possibly faulty logic to handle this kind of use cases on the client-side.

#include "cctz/time_zone.h"

std::chrono::seconds convert_to_epoch_time(const int year, const int month, const int day,

                                           const int hour, const int minute, const int second,

                                           const bool dst, const cctz::time_zone& tz) {

    const auto civil_sec = cctz::civil_second(year, month, day, hour, minute, second);

    const cctz::time_zone::civil_lookup cl = tz.lookup(civil_sec);

    const cctz::time_point<cctz::sys_seconds> utc_time_point = [&]() -> cctz::time_point<cctz::sys_seconds> {

        if (cl.kind == cctz::time_zone::civil_lookup::UNIQUE) {

            return cl.trans;

        } else if (cl.kind == cctz::time_zone::civil_lookup::SKIPPED) {

            return dst ? cl.post : cl.pre;

        } else {  // cctz::time_zone::civil_lookup::REPEATED:

            return dst ? cl.pre : cl.post;

        }

    }();

    return utc_time_point.time_since_epoch();

}

Thank you in advance!

/ András

[Bradley White]

On Tuesday, July 17, 2018 at 9:44:51 AM UTC-4, András Szilárd wrote:

Hi all,

Hi András,

we have a use case, where we need to calculate epoch time having local time with the timezone and we know whether this local time is during DST or not.

I figured out to use below logic, which is interesting in the DST transition time period.

I'll answer your questions directly first, and then get into some discussion below.

 

My questions:

 - Did I miss any helper function in the library API to make this easier/simpler?

No, there is no helper like your function. 

 

 - If there is no such function, than is below logic correct?

I'll say no, it is not correct, but see the discussion. 

 

 - Would it make sense to put something like this in the library API?

   Motivation: there would be no need to come up with possibly faulty logic to handle this kind of use cases on the client-side.

Given the previous answer, I don't think we should add something like that.

Discussion: 

The core problem is that "bool dst" is not sufficient to distinguish between repeated civil times.  The marking of a period as "dst" is an administrative thing, so you can't really code to it.  (In cctz::absolute_lookup we even warn about using it.)

Consider, for example, a transition from America/New_York, where dst ? cl.pre : cl.post does as expected, ...

1541311199 = 2018-11-04 01:59:59 -04:00:00 (EDT) [wd=Sun dst=T off=-14400]

1541311200 = 2018-11-04 01:00:00 -05:00:00 (EST) [wd=Sun dst=F off=-18000]

but in a transition from Europe/Dublin it does the exact opposite.

1540688399 = 2018-10-28 01:59:59 +01:00:00 (IST) [wd=Sun dst=F off=3600]

1540688400 = 2018-10-28 01:00:00 +00:00:00 (GMT) [wd=Sun dst=T off=0]

Also consider transitions where there is no change in "dst".  For example, in America/Porto_Acre we repeat the last hour of 2013-11-09, but both sides of the transition are marked as dst=F.  So, "2013-11-09 23:30:00 dst=F" remains ambiguous.

1384055999 = 2013-11-09 23:59:59 -04:00:00 (-04) [wd=Sat dst=F off=-14400]

1384056000 = 2013-11-09 23:00:00 -05:00:00 (-05) [wd=Sat dst=F off=-18000]

For skipped civil times it is difficult to imagine how you got the YMDhms/dst data in the first place.

Some other comments:

std::chrono::seconds convert_to_epoch_time(const int year, const int month, const int day,

                                           const int hour, const int minute, const int second,

                                           const bool dst, const cctz::time_zone& tz) {

Try to refrain from storing a "time" as a "duration".  Then you'll find you never need to say things like "epoch" (at least until you have to convert to some legacy time format).

I'd change the YMDhms parameters to a cctz::civil_second (and push that as far back in the call chain as you can).

 

    const cctz::time_point<cctz::sys_seconds> utc_time_point = [&]() -> cctz::time_point<cctz::sys_seconds> {

        if (cl.kind == cctz::time_zone::civil_lookup::UNIQUE) {

            return cl.trans;

        } else if (cl.kind == cctz::time_zone::civil_lookup::SKIPPED) {

            return dst ? cl.post : cl.pre;

        } else {  // cctz::time_zone::civil_lookup::REPEATED:

            return dst ? cl.pre : cl.post;

        }

    }();

What you're doing there is mapping a cctz::time_zone::civil_lookup to a cctz::time_point<cctz::seconds> using some logic to resolve the SKIPPED and REPEATED cases, so that is the function I'd write.

As mentioned, bool dst isn't up to the task, but perhaps you could have some other indicator that is.  We actually do have a helper function, ...

  cctz::time_point<cctz::seconds> cctz::convert(const cctz::civil_second&, const cctz::time_zone)

that encapsulate the simple (no indicator) disambiguation logic we find most useful.

Please yell if any of that raises other questions/issues.

Bradley

[András Szilárd]

Hi Bradley,

2018. július 17., kedd 23:02:35 UTC+2 időpontban Bradley White a következőt írta:

On Tuesday, July 17, 2018 at 9:44:51 AM UTC-4, András Szilárd wrote:

Hi all,

Hi András,

we have a use case, where we need to calculate epoch time having local time with the timezone and we know whether this local time is during DST or not.

I figured out to use below logic, which is interesting in the DST transition time period.

I'll answer your questions directly first, and then get into some discussion below.

 

My questions:

 - Did I miss any helper function in the library API to make this easier/simpler?

No, there is no helper like your function. 

 

 - If there is no such function, than is below logic correct?

I'll say no, it is not correct, but see the discussion. 

 

 - Would it make sense to put something like this in the library API?

   Motivation: there would be no need to come up with possibly faulty logic to handle this kind of use cases on the client-side.

Given the previous answer, I don't think we should add something like that.

Discussion: 

The core problem is that "bool dst" is not sufficient to distinguish between repeated civil times.  The marking of a period as "dst" is an administrative thing, so you can't really code to it.  (In cctz::absolute_lookup we even warn about using it.)

I understand this.

 

Consider, for example, a transition from America/New_York, where dst ? cl.pre : cl.post does as expected, ...

1541311199 = 2018-11-04 01:59:59 -04:00:00 (EDT) [wd=Sun dst=T off=-14400]

1541311200 = 2018-11-04 01:00:00 -05:00:00 (EST) [wd=Sun dst=F off=-18000]

but in a transition from Europe/Dublin it does the exact opposite.

1540688399 = 2018-10-28 01:59:59 +01:00:00 (IST) [wd=Sun dst=F off=3600]

1540688400 = 2018-10-28 01:00:00 +00:00:00 (GMT) [wd=Sun dst=T off=0]

I do not understand this last example.

IST means Irish Summer Time, right? Then why is DST=F[alse] in that case?

As I tested:

2018-11-04 01:59:59 [DST: True, timezone: America/New_York]

Epoch: 1541311199

2018-11-04 01:00:00 [DST: False, timezone: America/New_York]

Epoch: 1541311200

2018-10-28 01:59:59 [DST: True, timezone: Europe/Dublin]

Epoch: 1540688399

2018-10-28 01:00:00 [DST: False, timezone: Europe/Dublin]

Epoch: 1540688400

Last two with switched DST settings:

2018-10-28 01:59:59 [DST: False, timezone: Europe/Dublin]

Epoch: 1540691999

2018-10-28 01:00:00 [DST: True, timezone: Europe/Dublin]

Epoch: 1540684800

All these seem to be correct to me.

 

Also consider transitions where there is no change in "dst".  For example, in America/Porto_Acre we repeat the last hour of 2013-11-09, but both sides of the transition are marked as dst=F.  So, "2013-11-09 23:30:00 dst=F" remains ambiguous.

1384055999 = 2013-11-09 23:59:59 -04:00:00 (-04) [wd=Sat dst=F off=-14400]

1384056000 = 2013-11-09 23:00:00 -05:00:00 (-05) [wd=Sat dst=F off=-18000]

I see, in our case with the available input data we cannot do anything about it (see below).

Is it possible to do anything with this having the input described below? (I think the answer is "no").

 

For skipped civil times it is difficult to imagine how you got the YMDhms/dst data in the first place.

That branch (in the logic) is a best effort approach on our side.

 

Some other comments:

std::chrono::seconds convert_to_epoch_time(const int year, const int month, const int day,

                                           const int hour, const int minute, const int second,

                                           const bool dst, const cctz::time_zone& tz) {

Try to refrain from storing a "time" as a "duration".  Then you'll find you never need to say things like "epoch" (at least until you have to convert to some legacy time format).

I'd change the YMDhms parameters to a cctz::civil_second (and push that as far back in the call chain as you can).

The YMDhms + dst comes from an external source (our input), we cannot control anything about it. The tz comes from the configuration (manually setting the location/timezone of that external source).

We need the epoch for our output (again we cannot control anything about it).

 

 

    const cctz::time_point<cctz::sys_seconds> utc_time_point = [&]() -> cctz::time_point<cctz::sys_seconds> {

        if (cl.kind == cctz::time_zone::civil_lookup::UNIQUE) {

            return cl.trans;

        } else if (cl.kind == cctz::time_zone::civil_lookup::SKIPPED) {

            return dst ? cl.post : cl.pre;

        } else {  // cctz::time_zone::civil_lookup::REPEATED:

            return dst ? cl.pre : cl.post;

        }

    }();

What you're doing there is mapping a cctz::time_zone::civil_lookup to a cctz::time_point<cctz::seconds> using some logic to resolve the SKIPPED and REPEATED cases, so that is the function I'd write.

As mentioned, bool dst isn't up to the task, but perhaps you could have some other indicator that is.  We actually do have a helper function, ...

  cctz::time_point<cctz::seconds> cctz::convert(const cctz::civil_second&, const cctz::time_zone)

that encapsulate the simple (no indicator) disambiguation logic we find most useful.

As I see, this does not help in our case. The only indicator we have is the DST flag and we need that for the repeated times.

 

Please yell if any of that raises other questions/issues.

Bradley

Thank you for the quick feedback and suggestions!

/ András 

[Greg Miller]

Hi, 

IST now means Irish Standard Time. See the comments at https://github.com/eggert/tz/blob/master/europe#L369

Note that "DST" != "Summer time". Think of the "is dst" bit as meaning "is a non-standard offset". The best approach is to completely ignore the "is dst" bit, because as Bradley mentioned, it is just an administrative thing, not a technical thing.

 

As I tested:

2018-11-04 01:59:59 [DST: True, timezone: America/New_York]

Epoch: 1541311199

2018-11-04 01:00:00 [DST: False, timezone: America/New_York]

Epoch: 1541311200

2018-10-28 01:59:59 [DST: True, timezone: Europe/Dublin]

Epoch: 1540688399

2018-10-28 01:00:00 [DST: False, timezone: Europe/Dublin]

Epoch: 1540688400

Last two with switched DST settings:

2018-10-28 01:59:59 [DST: False, timezone: Europe/Dublin]

Epoch: 1540691999

2018-10-28 01:00:00 [DST: True, timezone: Europe/Dublin]

Epoch: 1540684800

All these seem to be correct to me.

 

Also consider transitions where there is no change in "dst".  For example, in America/Porto_Acre we repeat the last hour of 2013-11-09, but both sides of the transition are marked as dst=F.  So, "2013-11-09 23:30:00 dst=F" remains ambiguous.

1384055999 = 2013-11-09 23:59:59 -04:00:00 (-04) [wd=Sat dst=F off=-14400]

1384056000 = 2013-11-09 23:00:00 -05:00:00 (-05) [wd=Sat dst=F off=-18000]

I see, in our case with the available input data we cannot do anything about it (see below).

Most existing libraries encouraged the use of "is dst" as a technical piece of information. As a result, it's not uncommon for programs to produce and consume this data, under the incorrect assumption that it has a clear and unambiguous meaning. This is a mistake, but sadly we have to deal with it.

One long-term thing that you can start now is to file bugs and push back against the data producers sending you this incomplete information. Point out the bugs. Suggest fixes (such as sending along a precise UTC offset, or a time instant rather than a civil-time, etc). I realize this doesn't help your immediate situation, but future you and your future peers will thank present you for starting to move things in the right direction.

 

Is it possible to do anything with this having the input described below? (I think the answer is "no").

Possibly; I'll suggest something below.

 

 

For skipped civil times it is difficult to imagine how you got the YMDhms/dst data in the first place.

That branch (in the logic) is a best effort approach on our side.

 

Some other comments:

std::chrono::seconds convert_to_epoch_time(const int year, const int month, const int day,

                                           const int hour, const int minute, const int second,

                                           const bool dst, const cctz::time_zone& tz) {

Try to refrain from storing a "time" as a "duration".  Then you'll find you never need to say things like "epoch" (at least until you have to convert to some legacy time format).

+1 to Bradley's advice here.

 

I'd change the YMDhms parameters to a cctz::civil_second (and push that as far back in the call chain as you can).

The YMDhms + dst comes from an external source (our input), we cannot control anything about it. The tz comes from the configuration (manually setting the location/timezone of that external source).

We need the epoch for our output (again we cannot control anything about it).

 

 

    const cctz::time_point<cctz::sys_seconds> utc_time_point = [&]() -> cctz::time_point<cctz::sys_seconds> {

        if (cl.kind == cctz::time_zone::civil_lookup::UNIQUE) {

            return cl.trans;

        } else if (cl.kind == cctz::time_zone::civil_lookup::SKIPPED) {

            return dst ? cl.post : cl.pre;

        } else {  // cctz::time_zone::civil_lookup::REPEATED:

            return dst ? cl.pre : cl.post;

        }

    }();

What you're doing there is mapping a cctz::time_zone::civil_lookup to a cctz::time_point<cctz::seconds> using some logic to resolve the SKIPPED and REPEATED cases, so that is the function I'd write.

As mentioned, bool dst isn't up to the task, but perhaps you could have some other indicator that is.  We actually do have a helper function, ...

  cctz::time_point<cctz::seconds> cctz::convert(const cctz::civil_second&, const cctz::time_zone)

that encapsulate the simple (no indicator) disambiguation logic we find most useful.

As I see, this does not help in our case. The only indicator we have is the DST flag and we need that for the repeated times.

I agree with everything Bradley said. In particular, "is dst" is not really up to the challenge. However, if I were forced into your situation and had to come up with something that somewhat works, I might do the following (**AFTER** I filed a bug with the upstream data provider! :-) )

// Converts cs to an absolute time in the given time zone 

// if and only if the converted absolute time's "is dst" bit matches the given is_dst argument.

// If the dst bits do not match, returns nullopt.

// If a repeated time is specified and both absolute time's match is_dst, returns nullopt.

std::optional<cctz::time_point<cctz::seconds>> choose_time_point(

    const cctz::civil_second& cs, 

    const cctz::time_zone tz, 

    bool is_dst);

 

That is, write a function that allows for errors, and handle those error cases as errors.

HTH,

Greg

 

Please yell if any of that raises other questions/issues.

Bradley

Thank you for the quick feedback and suggestions!

/ András 

--
https://github.com/google/cctz
---
You received this message because you are subscribed to the Google Groups "cctz" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cctz+uns...@googlegroups.com.
To post to this group, send email to cc...@googlegroups.com.
Visit this group at https://groups.google.com/group/cctz.
To view this discussion on the web visit https://groups.google.com/d/msgid/cctz/a4e85016-1c7d-4ea6-8827-034a09bf15a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[András Szilárd]

Thank you guys!

[Bradley White]

On Wednesday, July 18, 2018 at 9:42:10 AM UTC-4, Greg Miller wrote:

Hi, 

On Wed, Jul 18, 2018 at 3:52 AM András Szilárd <andras....@gmail.com> wrote:

but in a transition from Europe/Dublin it does the exact opposite.

1540688399 = 2018-10-28 01:59:59 +01:00:00 (IST) [wd=Sun dst=F off=3600]

1540688400 = 2018-10-28 01:00:00 +00:00:00 (GMT) [wd=Sun dst=T off=0]

I do not understand this last example.

IST means Irish Summer Time, right? Then why is DST=F[alse] in that case?

IST now means Irish Standard Time. See the comments at https://github.com/eggert/tz/blob/master/europe#L369

Note that "DST" != "Summer time". Think of the "is dst" bit as meaning "is a non-standard offset". The best approach is to completely ignore the "is dst" bit, because as Bradley mentioned, it is just an administrative thing, not a technical thing.

Right.  I think the important thing to keep in mind is that there is nothing in the interpretation of an "into Daylight Saving Time" transition that implies it is a positive offset delta that moves daylight to the end of the civil day (summer) rather than a negative offset delta that moves daylight to the start of the civil day (winter).

As I tested:

2018-10-28 01:59:59 [DST: True, timezone: Europe/Dublin]

Epoch: 1540688399

2018-10-28 01:00:00 [DST: False, timezone: Europe/Dublin]

Epoch: 1540688400

That indicates you are using old or "rearguard" data, and I would agree with those results.  That is, ...

2018-10-28 01:00:00 dst=T Europe/Dublin => 1540684800

2018-10-28 01:59:59 dst=T Europe/Dublin => 1540688399

2018-10-28 01:00:00 dst=F Europe/Dublin => 1540688400

2018-10-28 01:59:59 dst=F Europe/Dublin => 1540691999

Last two with switched DST settings:

2018-10-28 01:59:59 [DST: False, timezone: Europe/Dublin]

Epoch: 1540691999

2018-10-28 01:00:00 [DST: True, timezone: Europe/Dublin]

Epoch: 1540684800

All these seem to be correct to me.

I'm not sure what you mean by "with switched DST settings", but with "vanguard" data I would assert that the answers are ...

2018-10-28 01:00:00 dst=F Europe/Dublin => 1540684800

2018-10-28 01:59:59 dst=F Europe/Dublin => 1540688399

2018-10-28 01:00:00 dst=T Europe/Dublin => 1540688400

2018-10-28 01:59:59 dst=T Europe/Dublin => 1540691999

which does not seem to match what you wrote.

In any case, it sounds like you had better ensure that you agree with your external source on the interpretation of "dst" for zones like Europe/Dublin.  (As Greg mentioned though, it would be better if they could just send you unambiguous data.)

 

I agree with everything Bradley said. In particular, "is dst" is not really up to the challenge. However, if I were forced into your situation and had to come up with something that somewhat works, I might do the following (**AFTER** I filed a bug with the upstream data provider! :-) )

// Converts cs to an absolute time in the given time zone 

// if and only if the converted absolute time's "is dst" bit matches the given is_dst argument.

// If the dst bits do not match, returns nullopt.

// If a repeated time is specified and both absolute time's match is_dst, returns nullopt.

std::optional<cctz::time_point<cctz::seconds>> choose_time_point(

    const cctz::civil_second& cs, 

    const cctz::time_zone tz, 

    bool is_dst);

 

That is, write a function that allows for errors, and handle those error cases as errors.

That sound reasonable.  One thing I would suggest is that rather than using bool dst to directly select between cl.pre and cl.post, run both those time points back through the absolute_lookup version of tz.lookup(tp) to see which one has a unique match on al.is_dst.  If there is no unique match, you've hit one of the error cases. 

On Wednesday, July 18, 2018 at 10:26:53 AM UTC-4, András Szilárd wrote:

Thank you guys!

You're welcome.  Good luck.

Bradley 

[Greg Miller]

+1. That's exactly what I was thinking too. I should have made that more explicit. Thanks for clearing that up, Brad.

 

On Wednesday, July 18, 2018 at 10:26:53 AM UTC-4, András Szilárd wrote:

Thank you guys!

You're welcome.  Good luck.

Bradley 

--

https://github.com/google/cctz
---
You received this message because you are subscribed to the Google Groups "cctz" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cctz+uns...@googlegroups.com.
To post to this group, send email to cc...@googlegroups.com.
Visit this group at https://groups.google.com/group/cctz.

To view this discussion on the web visit https://groups.google.com/d/msgid/cctz/5e751ee2-2755-4cc7-8878-f458842b2bf2%40googlegroups.com.

[Bradley White]

Just one one additional observation in parting: I think it is a strength of the API that it is difficult to do questionable things.  It is a feature that this has caused you to question your external source.