Commuter Rail trips missing from APIv2

65 views
Skip to first unread message

StefanW

unread,
Dec 9, 2016, 10:34:37 AM12/9/16
to MBTA Developers
Right now (2016-12-09 10:30 AM) train 405 is in the legacy feed with all data points - lat/long, stop predictions, lateness, etc. - but it is completely missing from the APIv2. I've checked predictionsByRoutes, vehiclesByRoutes and predictionsByStop. The legacy feed has it at 65 minutes late. It was late from the start of the trip, due to the collision earlier today of train 406 with an auto. 

This missing train 405 is very concerning, and another example of delayed / non-moving trains not showing at all in the API while still being fully shown in the legacy feed. How can we address this? Now that we're in another winter season we especially need to be able to give riders all available information. I would hate to be someone who missed train 405 because they thought it wasn't running due to not being in the API.

As I've suggested before, one possible solution is to present all trains in the vehiclesByRoute(s) calls, even when there is no prediction available.

StefanW

unread,
Dec 9, 2016, 11:15:10 AM12/9/16
to MBTA Developers
Another example from this morning is train 406 / 408. 

406 struck an auto at a crossing and was taken out of service. It disappeared from the API long before it disappeared from the legacy feed, but that's OK because it was out of service. (I still would expect it to be in the vehiclesByRoutes calls for as long as it was also showing in the legacy feed however.)

Train 408 however never went out of service. It held at Belmont for roughly 1 hour before moving up beside 406 to take on those passengers, then finished its trip. When it reached Porter the legacy feed was reporting 60 minutes late. In spite of never being taken out of service, 408 disappeared from the API around 9AM while it was waiting at Belmont.

Obviously in this example the only passengers impacted by the lack of train 408 data were waiting at Porter to go to North Station, or waiting at North Station to meet an arriving passenger (both are typically small head-counts) but what if instead it had been an outbound train? or an inbound train early in the trip? It appears that after no movement of X minutes a train is completely dropped from the API never to return - even if it resumes its trip after that X minute period. 
(X seems to be roughly in the range of 20 to 30 minutes.)

I feel strongly that in exceptional cases where a train is delayed for a significant amount of time it is even more important to provide data on it - at least a minimum of all the data points included in the vehiclesByRoutes returns.

Developer at MBTA

unread,
Dec 12, 2016, 10:21:49 AM12/12/16
to MBTA Developers
Thanks Stefan. We've investigated the reasons that this train was missing from API v2. It comes down to a balance between throwing out data that's probably wrong, and keeping data that's probably right. Investigating this incident we've identified two configuration changes that will go into effect tomorrow which will reduce the number of scenarios in which a train that's actually in service can be excluded from the production feed. 

Sincerely,
developer@mbta

Stefan Wuensch

unread,
Dec 12, 2016, 1:05:44 PM12/12/16
to massdotd...@googlegroups.com
Thank you! That sounds great.

Do you have thoughts about the vehiclesByRoute(s) calls?

I have always thought that the entire point of having vehiclesByRoute(s) in addition to predictionsByRoute(s) was so that the _vehicles_ could be tracked every time, all the time - even if predictions can't be made or shouldn't be made. Can you make it so that _nothing_ is thrown out from vehiclesByRoute(s), ever?

If your concern about publishing bad data applies to just the trip info in vehiclesByRoute(s) then what about creating a new API call literally for vehicles only, that would *only* return the direction->trip->vehicle objects?

In other words... if you think data for a trip is wrong, only drop the "trip_headsign", "trip_id", and "trip_name" fields. Providing just the "vehicle" objects with none of them thrown out entirely would work for me.

Thanks!!

Eric

unread,
Dec 13, 2016, 10:36:31 AM12/13/16
to MBTA Developers
Just wanted to voice my support for Stefan's vehicleByRoute proposal. The same issue happened with Fitchbug 400 yesterday morning when it was delayed due to another car on the tracks.

I have two band-aids in place at the moment to address this. First, if a delay is posted for the trip in alertsByRoute I parse the number of minutes from the description and add it to the scheduled trip's end time. For example, Fitchburg 400 is scheduled to arrive at North Station at 6:27 am; if there's a 20-30 minute delay posted in alertsByRoute, I add 30 minutes to 6:27 so if the current time is <= 6:57 I assume the trip is still in progress and force it to appear in my app.

This is obviously fragile and it breaks down quickly when a time estimate isn't attached to a delay, e.g. "Train is stopped at _______, updates to follow".

My second workaround is to login to an administration panel that I build specifically for this issue where I can force trips to appear in my app.

StefanW

unread,
Jun 1, 2017, 1:46:49 PM6/1/17
to MBTA Developers
FYI - a continuing issue with the API Commuter Rail predictions...

Train 109 was missing from the API for about 10 or 15 minutes for no reason that I can see other than it was sitting still on an otherwise normal trip. Train 166 was also missing from the API but for a shorter period of time. Because of events like this - trains being dropped mid-trip - I'm still not 100% confident in the API for Commuter Rail. It's especially troublesome because these trains were missing from the API during a delay, which is exactly when riders need the information the most!

Background: outbound train 161 died at Salem on the single-track section, blocking the route. It was later cancelled. Inbound train 166 bolted on as a rescue and the combination is (as I type this) running in normal revenue service at Chelsea albeit 55 minutes late and at reduced speed. The API did fairly well for those two trains 161 and 166: the dead/cancelled 161 was dropped from the API after a short period of time running in reverse (inbound) and 166 was "only" missing from the API for about 5 minutes. 

Train 109 missing from the API is my biggest concern from today though. It was waiting south of Salem for 161 (and later 161/166 combo) to clear the single-track section. Other than not moving for a period of time, there was nothing unusual about train 109... but it was dropped from the API. Yes it did reappear in the API later but it shouldn't have been dropped at all.

Because my app is all about having the latitude/longitude of a train set there's nothing I can do for a work-around other than to continue to use the legacy Commuter Rail feed as my authoritative data source. I would like to move off the legacy feed and solely onto the API but I still feel I can't yet.

As noted before, if vehiclesByRoute(s) calls could be more "raw" and not filtered like the predictionsByRoute(s) calls then that would work for me.


-- Stefan


 
Begin forwarded message:

Subject: Update: Newburyport/Rockport Line delay
Date: June 1, 2017 at 1:00:00 PM EDT

Rockport Train 109 (12:00 pm from North Station) is operating 25-35 minutes behind schedule between Salem and Rockport due to an earlier mechanical issue.

Affected direction: Outbound

Last updated: Jun 01 2017 12:58 PM
Sent by the MBTA.

Stefan Wuensch

unread,
Jun 2, 2017, 6:54:41 PM6/2/17
to massdotd...@googlegroups.com
Same thing again tonight, but magnified. There's been an apparent trespasser struck by train 194.

Trains 194, 119, 171 are all intermittently missing (more missing than showing) from the API but still showing in the legacy feed. None of those trains were cancelled... they were simply sitting still waiting for EMS to clear the scene.

Stefan Wuensch

unread,
Jun 13, 2017, 8:40:29 AM6/13/17
to massdotd...@googlegroups.com, MBTA Developers
MBTA folks please take a look at the API for train 492 (control coach 1637) this morning. The legacy feed has been showing 492 progressing along its normal route - right now as I type this in Lincoln - but it is completely missing from the API vehiclesByRoutes and predictionsByRoutes.

I am aware of the Alert that went out saying 492 was cancelled, so I'd like to know if the legacy feed is showing a deadhead trip or if the 492 is actually running. The train appears to be coming to a stop at each station (based on the speed), which would appear to rule out a deadhead move.

My guess is that 492 was "un-cancelled" but nobody corrected the API which is still hiding it.

Thanks!








Sent from my iPad Micro

Developer at MBTA

unread,
Jun 13, 2017, 12:06:17 PM6/13/17
to MBTA Developers, deve...@mbta.com
We've checked with operations and confirmed that train 492 was a cancelled train and did not run. 

-developer@mbta

Stefan Wuensch

unread,
Jun 13, 2017, 12:07:49 PM6/13/17
to massdotd...@googlegroups.com, Developer
Thank you very much! I greatly appreciate it.
Reply all
Reply to author
Forward
0 new messages