August changes to GTFS and API, including trip_route_type

[Developer at MBTA]

Dear developers,

We wanted to tell you about changes to expect to our GTFS feed and API in August.

Changes to alerts in API
We plan to make some changes to API values on the evening of Thursday, August 3, detailed here:
https://groups.google.com/d/msg/massdotdevelopers/GQMRAHDzqY8/xDTc2gQ5AgAJ

Changes to transfers.txt

We plan to roll out some changes to transfers.txt in August, detailed here:
https://groups.google.com/d/msg/massdotdevelopers/xG8YAZh2Rm0/nheG-BY5AgAJ

Addition to trips.txt

We also plan to append a new field to trips.txt called trip_route_type. When used, the particular trip’s trip_route_type will override the route_type trip defined in routes.txt. The values for trip_route_type will be exactly the same as those for route_type, which can be found in the GTFS specification {link to https://developers.google.com/transit/gtfs/reference/routes-file}.

This field will generally remain null except when the type of transportation used for a particular trip is different from that typically used on the trip’s route. For example, if a Commuter Rail trip, on a route with a route_type of 2 for “rail” is operated using a replacement shuttle bus for an extended period, due to a construction project, for example, the entry for this trip in trips.txt could now include a trip_route_type of 3 for “bus”. Developers should assume that a trip will operate using the respective route_type unless superseded by a non-null trip_route_type. This change is designed to allow developers to more accurately show changes in service to clients.

--

Thanks for your attention!

Sincerely,
developer@mbta

[Paul Harrington]

I know these changes have been flagged for a long time but just catching up now on what they mean now in both general GTFS terms and for me as a consumer.

In general GTFS terms does it mean that by adding extra fields to files and files to tables you are moving from a compliant GTFS system to a system that is based on GTFS or "GTFS-Like" ?

Regarding my own implementation extra files is not a problem. However extra fields will currently result in a new schedule failing to load in to a database because that field does not exist in the schema. The code for loading DB tables from corresponding files is very compact as it dynamically prepares a SQL statement for each file based on the header of that file and this is done in a generic fashion. So preparing a statement using:

sql = String.format("insert into %s (%s) values (%s)", feedTable, fieldNameList, preparedStatementPlaceholders);

PreparedStatement statement = db.prepareStatement(sql);

with the fieldNameList containing a field which is not in the schema will result in an exception and aborting the load.

I hope to use this "opportunity" to harden my loader so that if a field is not in the DB schema I ignore it and its value when doing DB insertion. While not particularly complex to do, it is however non trivial.

I also need to review and relax my DB schema so that the likes of location_type in the stops table is no longer:

location_type char(1) check (location_type in ('','0','1')) not null default '0',

I am wondering are there any other GTFS consumers out there who will also need to change their implementation ?

Regards Paul.

[Developer at MBTA]

Great question, Paul! The MBTA's GTFS feed remains a GTFS feed. GTFS does allow additional files, fields, and values* beyond those that are defined in the spec to be included without the feed violating the rules of the spec. This is actually the path by which the spec is changed -- a change is proposed in which a field is added, several agencies try it out, and if the change is deemed beneficial it's put to a vote and the change is officially added to the specification. The MBTA's aim when adding files, fields, and values is to produce a feed in which developers can use this additional data to improve the customer experience, but developers that choose to ignore these additions are still getting the best feed the MBTA can produce.
 
Sincerely,
developer@mbta

* i.e. a stop location_type of 3 to represent auto entrance/exit, something we're thinking about as we map our stations out.

[Paul Harrington]

Thanks for the feedback, I had not spotted anything either way in the spec on this. However it did make sense that providers could add extra files or fields as they wish and an extra 20 lines or so of code enabled me to get around this and not bomb out. I'll look forward to testing further with the next version of your schedule when it is available. 

I will work around your new location_type of 3 by changing

location_type char(1) check (location_type in ('','0','1')) not null default '0',

to

location_type char(1) not null default '0',

However in general with changing fields it get a bit more vague with fields like route_type where the GTFS extension changes it from having a value of 0..9 to a much broader numerical value. My schema would change in this case from

route_type char(1) check (route_type in ('0','1','2','3','4','5','6','7')) not null,

to something like:

route_type varchar(256)

where you would have to look at an implementation to see if there are knock on effects. Adding fields and files is one thing but as most consumers would load into a DB I wonder how much field values can be added to without effecting existing DB schemas out there.

--
You received this message because you are subscribed to the Google Groups "MBTA Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelopers+unsubscribe@googlegroups.com.
To post to this group, send email to massdotdevelopers@googlegroups.com.
Visit this group at https://groups.google.com/group/massdotdevelopers.
For more options, visit https://groups.google.com/d/optout.

[Developer at MBTA]

Hello developers,

We would like to discuss some additional GTFS changes on their way and provide a timeline for the changes we have already announced.

Commuter rail transfers in transfers.txt, including the addition of the min_walk_time, min_wheelchair_time, and suggested_buffer_time, will be added to GTFS on Thursday, August 17. See the earlier post at https://groups.google.com/d/msg/massdotdevelopers/xG8YAZh2Rm0/nheG-BY5AgAJ for more information.

We also plan on adding transfer time information for several major subway-to-subway stations on Thursday, August 24.

Also on Thursday, August 24, we will add the new trip_route_type field to trips.txt (see first post in this thread).

In the near future after August we plan on rolling out several other changes to GTFS:

• the addition of platforms to commuter rail stations, station entrances, and other changes to stops.txt described at https://groups.google.com/d/msg/massdotdevelopers/7tb2pwvj-rM/ruPSn9HoAQAJ.
• A new file and a new field to stop_times.txt to add data that helps us with our own on-time performance analysis. The new information will be called "checkpoints," and although it's aimed primarily at internal use, we will provide more information about it in the coming week.
• Special event service to and from Gillette Stadium in Foxborough, so that these trains can appear in electronic schedules and be tracked in real time. We will announce dates and detail such changes and additions as we are able.

Sincerely,

developer@mbta