Tools and process to flatten GTFS calendars

[Aaron Antrim]

Hi Transit Developers,

Trillium has been asked to provide GTFS data for a consumer that requires no more than one entry in calendars.txt should be active on a given service day. Typically, we would provide layered services — if there is service that runs Mon-Fri and also additional service on Mon-Thurs, we would express each with a distinct calendar_id. Similarly, if there is a year-round service with additional supplemental seasonal service, we’d express each with a distinct service_id.

Are there any tools or libraries that would allow us to “flatten” services so there is only one calendar_id active per day?

Currently, GTFS we are producing might look like this:

        |-----------|          Service A: Oct 1-Nov 30

|------------------------|     Service B: Sep 1-Dec 15

            |------------|     Service C: Oct 15-Dec 15

We need to make this:

|-------|---|-------|----|     Sep 1-Oct 1; Oct 1-15; Oct 16-Nov 30; Dec 1-15

Has anyone else had to solve this problem?

Also — GTFS Best Practices are silent on this (https://gtfs.org/best-practices/#calendartxt). Would more technical guidance be helpful? I tend to think that layered rather than flattened GTFS datasets are preferable because it makes it possible to see how services are being turned on and off.

Thanks!

-- 
Aaron Antrim
Chief Technology Officer & Founder
Trillium - We make transit easier to use.

[Mike Gilligan (TriMet)]

Aaron,

-1, flattening GTFS calendars

The layered approach to service is standard practice in transit scheduling. Flattening the service per calendar date would cause an enormous amount of trip/stop time duplication and added burden for GTFS producers. The layered approach has been supported by GTFS since the beginning and I welcome establishing it as a best practice.

Mike Gilligan

Data Engineer, TriMet

[Laurent Grégoire]

Hi Aaron,

FYI we (at Mecatran) do that kind of calendar "flattening" (we call it internally "calendar partitioning") for a few of our customers, for various reasons, mainly internal and technical.

For example, for Paris Metro lines, we reconstruct frequencies from plain trips, but our internal algorithm requires calendars to be partitioned beforehand.

This partitioning is rather simple, but it creates indeed new calendars and duplicate trips. Personally, I agree with you, I would not recommend doing that as a "best practice".

HTH,

--

Laurent GRÉGOIRE

Mecatran

[harringtonp]

I would be curious to know why they need this done ?

On the consuming side I generally read each schedule file into its own table and build sql queries across the tables. Sql queries were getting quite complex but performance improved (and queries simplified) by merging the calendar and calendar dates tables into a single simpler structure. 

Sent from my Samsung Galaxy smartphone.

[Aaron Antrim]

(From Drew… looks like it didn't hit the public thread.)

---------- Forwarded message ---------
From: Drew Dara-Abrams <dr...@interline.io>
Date: Tue, Oct 27, 2020 at 12:20 PM
Subject: Re: [transit-developers] Tools and process to flatten GTFS calendars
To: <transit-d...@googlegroups.com>
Cc: Aaron Antrim <aa...@trilliumtransit.com>

Hi Aaron and all,

We do this type of calendar/calendar date manipulation for the Bay Area's Historical GTFS Feed. An overview of the method is at 

https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#historical-feeds

The script used to create this particular GTFS output is quite custom and isn't public, but it's all built atop our open-source GTFS processing library which is public at https://github.com/interline-io/transitland-lib

Drew

-- 

Drew Dara-Abrams || Ph.D. || Principal || he/him
Interline Technologies || San Francisco Bay Area
www.interline.io || +1 (415) 610-4304‬

--

Aaron Antrim (pronouns - he/him)

Chief Technology Officer & Founder

Trillium - We make transit easier to use.

+1 (503) 567-8422 ext. 3