New calendar_dates.txt exception_types to more compactly represent Daylight Saving Time exceptions?

[Brian Ferris]

One minor annoying thing about GTFS feeds is accounting for Daylight Saving Time (DST).  Some details are discussed in this post I made a while back:

http://groups.google.co.nz/group/googletransit/browse_thread/thread/5f41b59f600553f3

For a trip that starts at the same time every day, including a DST day, it seems the only way to properly capture the correct start time on the DST day is to duplicate the trip and all its stop times with a new DST-specific trip, adjusting the duplicated stop times by an hour in the proper direction.  While this works, it can significantly increase the size of the resulting GTFS, as you essential end up duplicating any trip that falls on a DST day, where the resulting trip and stop time copies are almost exactly the same except for the DST offset.

A more compact representation might be to add two new exception_types to calendar_dates.txt.  We'll call them DST+1 and DST-1.  Now, to apply a DST offset on the appropriate day, we can apply it to an entire service_id:

service_id,date,exception_type
SomeServiceId,20090308,DST-1

Here we see an example of applying the March 08, 2009 United States DST.  Since we "sprang forward" on that day, an hour actually disappeared from the day.  A trip that would normally start at 10:00am would usually be represented as an offset since midnight of "10:00:00" in GTFS time.  On the spring DST day, 10:00am is now only nine hours from midnight, so the time should be "09:00:00", which we get by applying our DST-1 exception_type and subtracting one hour from the normal time.

Any thoughts on this?  How do others deal with DST?

[Joe Hughes]

Brian,

Thanks for bringing this up--it's an annoyance in the current spec,
and your solution is reasonably elegant. I was wondering whether you
could get away with two new enum values for +1 and -1 hour--according
to this page, the only place that I could find with a non-1h daylight
savings time adjustment these days is Lord Howe Island in Australia:
http://en.wikipedia.org/wiki/List_of_zoneinfo_timezones

And according to this page, Lord Howe Island "does not need public transport":
http://www.lordhoweisland.info/important-info.htm

So maybe that works out.

Technically, we'd probably want to require the DST date to fall inside
the bounds of the service_id that it's attached to, either by a range
specified in calendar.txt, or by another calendar_dates.txt entry with
the same service_id and date and exception_type 1.

More thoughts on this? Are there feed publishers that are interested
in trying this out?

Joe

[Jehiah Czebotar]

It isn't clear to me if this proposal is designed specifically for
adjusting the schedule on the day of DST change, or if it's meant for
adjusting the schedule on all dates with DST.

If it's the former (which i think it is) I feel like we need to
account for the fact that DST actually changes at 2am local time (in
the states anyway.. i don't recall about elsewhere), so a schedule at
1:30am should *not* have the +/-1 applied to it.

This of course means when time falls backwards (ie: -1) that it's not
clear to a user if a 1:30am am schedule is the first time through
1:30am, or the second time (ie: 2:30am-1hr). I'm not sure there is any
decent way to clarify that though. (granted, this is a display to user
type problem, not a feed spec problem)

Not working for any agency, i'm also curious what an agency would do
that runs a schedule every hour (ie: 10:30pm, 11:30pm, 12:30am). in
the +1 case the 11:30pm schedule becomes 12:30am the following day,
however the next day will already have a 12:30am schedule. Do agencies
normally run both? Do we drop anything departing in the last hour of
the day?

I certainly feel like this topic needs more input from the agency/feed
publisher side than the feed consumer side.

--
Jehiah

[Brian Ferris]

So I bring up this issue mostly out of my work with King County Metro.  During DST, the bulk of their routes have their time adjusted appropriately to reflect the time change.  However, they also have a number of night routes that continue running their regular schedules as if DST had not occurred.

You can model this distinction with the proposed feature change by noting that the DST change only applies to a particular service ids on a particular date.  Thus, in the KCM case, I would use two separate service ids to model the two different types of routes (normal and night service), and then only apply the DST calendar_dates.txt exception to the normal service id.

Does that make sense?

Brian

[Chas_Belov_SFMTA]

I don't believe the service quirks of the switch between Daylight time
and Standard time would be reflected in the SFMTA Google Transit feed.
Actually, I don't even believe that Standard vs. Daylight time would
be reflected in the SFMTA feed at all. The SFMTA runs on local time,
whether that is Standard or Daylight time.

Service quirks of the switch are handled through a cutomer alert
message. It is unlikely that trip planning crossing this time change
barrier would be accurate. Fortunately, the SFMTA has 24-hour customer
phone service line, but there is no technical solution available.

Hope this helps,
Charles "Chas" Belov
SFMTA Webmaster

> > On Fri, Jul 10, 2009 at 12:51 PM, Joe Hughes<joe.hughes.c...@gmail.com>

> > wrote:
>
> > > Brian,
>
> > > Thanks for bringing this up--it's an annoyance in the current spec,
> > > and your solution is reasonably elegant.  I was wondering whether you
> > > could get away with two new enum values for +1 and -1 hour--according
> > > to this page, the only place that I could find with a non-1h daylight
> > > savings time adjustment these days is Lord Howe Island in Australia:
> > >http://en.wikipedia.org/wiki/List_of_zoneinfo_timezones
>
> > > And according to this page, Lord Howe Island "does not need public
> > transport":
> > >http://www.lordhoweisland.info/important-info.htm
>
> > > So maybe that works out.
>
> > > Technically, we'd probably want to require the DST date to fall inside
> > > the bounds of the service_id that it's attached to, either by a range
> > > specified in calendar.txt, or by another calendar_dates.txt entry with
> > > the same service_id and date and exception_type 1.
>
> > > More thoughts on this?  Are there feed publishers that are interested
> > > in trying this out?
>
> > > Joe
>

> > > On Wed, Jul 8, 2009 at 4:54 PM, Brian Ferris<bdfer...@gmail.com> wrote:
> > >> One minor annoying thing about GTFS feeds is accounting for Daylight
> > Saving
> > >> Time (DST).  Some details are discussed in this post I made a while
> > back:
>

> >http://groups.google.co.nz/group/googletransit/browse_thread/thread/5...

[Tom Brown]

Apologies for the delay.

You bring up some very good points about the DST transition. Looking at a few feeds in the USA (BART, Caltrain, San Diego, TriMet) I can't find anyone that has special service keys that run only the transition days in 2009 (begins on March 8 and ends on November 1). As Chas says SFMTA doesn't handle it specially. Are any GTFS producers correctly handling DST transitions according to seconds since midnight?

So why hasn't Google's trip planner returned trips off by one hour 2 days a year? Because when surrounded by time handling code someone had the good idea to interpret the HH:MM:SS in stop_times as seconds past 12h before midnight. In other words all stop times on transition days are treated as being in the new offset. This is convenient because most trips are scheduled at local times after the DST transition and don't need to be duplicated. Unlike adding special entries to the calendar files this interpretation doesn't require the data producer to pay attention to when the DST transition happens, except for the few trips scheduled at local times before the transition.

The problem is that we didn't go back and put this in the specification. So what are our options, as the community responsible for gtfs?

1) Leave spec is and require correct feeds to duplicate trips on transition days

2) Add new exceptions to service_id,date pairs which say that on these dates add or subtract one hour from stop_times. require correct feeds to add these exceptions to their data

3a) Continue interpreting stop_times as "seconds since midnight" unless a new exception marks a service as using "seconds since noon - 12h"

3b) Change specification to interpret all stop_times as "seconds since noon - 12h", breaking backwards compatibility.

3c) Change specification to interpret all stop_times as "seconds since noon - 12h", breaking backwards compatibility. Add new flag to force a particular service_id to use the old interpretation "seconds since midnight"

Assuming that most data today doesn't handle DST transitions (1,2,3a) require widespread changes which isn't very attractive. I think we should change the specification to document what is done in practice: stop_times values don't change on DST transition days for daytime service. The gtfs-changes welcome message says "Changes to the spec should be backwards-compatible. When adding features to the specification, we want to avoid making changes that will make existing feeds invalid." While this is a backwards incompatible change I think it actually makes existing feeds valid.

Details

[mostly based on DST leaps in the USA (spring 01:59:59 to 03:00:00; fall 01:59:59 to 01:00:00)]

 A quick skim of http://en.wikipedia.org/wiki/Daylight_saving_time_around_the_world finds other problems. In Egypt and Brazil there is no midnight when clocks are moved forward. I can't find anywhere that the leaps happen during the day so noon should be safe.

Feed producers who assume the HH:MM:SS are equivalent to local times are currently incorrect according to "s since midnight" for service days with a DST transition by one hour after the leap forward or backward, which is most of the day. Changing GTFS to "s since noon - 12h" will mean they are incorrect before the leap, a few hours in the night.

Feed producers who correctly produce HH:MM:SS as "s since midnight" will need to change the times forward or back an hour on transition days or move the service to the previous day and add 24h to the times. In the unlikely event that this is really hard we could offer a new flag to these services (3c).

Here is a little table of examples during transitions in the USA. Note that "s since midnight" is different from local time during most of transition days and times from the previous day are the same under "s since midnight" and "s since noon - 12h" interpretations.

          s since midnight         local time

calendar            s since noon - 12h

2009-03-08    00:00      01:00         0:00

2009-03-08    02:00      03:00         3:00

2009-03-08    07:00      08:00         8:00

2009-10-31    24:00      24:00        24:00

2009-11-01    00:00      *             0:00

2009-10-31    25:30      25:30         1:30 (1st 1:30)

2009-11-01    01:30      00:30         1:30 (1st 1:30)

2009-10-31    26:30      26:30         1:30 (2nd 1:30)

2009-11-01    02:30      01:30         1:30 (2nd 1:30)

2009-10-31    27:00      27:00         2:00

2009-11-01    03:00      02:00         2:00

2009-11-01    09:00      08:00         8:00

* can't represented on 2009-11-01 and needs to be moved to 2009-10-31