API Request Problems

[rohit mittal]

Hey everyone,

I've been working on this really cool app and was ready to integrate MTAs feeds into it. I requested an API through the online form but I didn't get any clarity or link to where the API's are located. I even emailed the MTA last week about this, but received nothing back from them about the steps I should take. Is there any link or anything I can do in order to ensure that I can get the data?

Thanks, Rohit

[Aaron Donovan]

Hi Rohit,

By all means, start right here:

http://web.mta.info/developers/

The API links are here:

http://web.mta.info/developers/download.html

In order to see the links, you need to click where it says "Click here to review the full terms and conditions of MTA data usage."

[TF BKNY]

Hi Aaron, 

Unfortunately MTA's API is horrendous when it comes to providing organized and well documented information to developers. It's also rather unfortunate that something as simple as the Service Status is being provided as a text file in a rather outdated XML format with JSON remaining still unavailable... (Yes I got a key but that didn't help when trying to retrieve data for the service status.

There seems to be no unified platform serving transit data (real time or not) as well as no alternative source providing transit feeds. Not to mention docs. The process and experience with MTA's API so far has been truly frustrating.

Even a simple ajax call to the http://web.mta.info/status/serviceStatus.txt url couldn't be performed without cross site issues requiring to use jsonp as a workaround... and even then, it would still not work in the end.

So after my ranting out of the massive frustration caused by all of the above, my question is: Are there (will there be) docs and code examples that devs can consult and use to become able to consume MTA's API data for their applications?

Thanks much.

tf

[Dan Jabbour]

As brought up during the subway map debate last night at Cooper Union (which representatives from the MTA flatly declined an invitation to) the formatting of the service status feed can not be consistently or reliably parsed in a useful way by a computer. This severely limits the usefulness of the information provided and for all intents and purposes only offers a way to display the existing feed status web page to users. A panelist at the debate noted that the MTA has no plans to make updates to the current outdated and anemic XML status feed.

While any information is better than no information at all, this is really unacceptable for one of the largest public transportation systems in the world. To expect developers to try to find patterns in sentence structure in order to scrape useful delay information out of unpredictably formatted HTML without information connecting it to a specific set of lines or station IDs and no unique identifier or status code per delay in XML is a huge waste of time and incredibly frustrating when little nuances in the language used happen to change destroying hours of work and leaving end-users in the dark.

These inconsistencies give small and independent developers an unnecessarily high and often insurmountable barrier to entry when attempting to use MTA data since they are often unable or unwilling to spend hours or more likely days of development time just to work out how to obtain the information to display in an application. It also results in a poor end-user experience for those that do put in the time needed since features reliant on inconsistent information either need to be omitted or run the risk of providing inaccurate information.

Clearly there is an agency or internal team or college intern or someone somewhere typing this delay information into a computer. Is there any reason more specific information can’t be included in a machine-readable format (even if it is outdated XML)? I won’t go farther into what’s wrong with the current feed, but will suggest that the feed should be restructured to offer a list of delays affecting lines rather than a list of lines with delays. I’ve proposed additional feed information below which should not be difficult for a person to include along with the information already provided. There are hundreds of very capable developers in this google group that I’m sure would be willing to put some of their own time into designing and implementing a better system for the MTA to use if given the opportunity.

-Dan

Current pseudo-code:

[

{

lineName : 'ACE';

status : "PLANNED WORK";

delays : "<tr><td><h1>Northbound F trains are running on the A line</h1></td></tr><tr><td>Take a southbound train to Jay St bla bla bla</td></tr>";

date : "10/28/2015";

time : " 1:11PM";

},

{

lineName : 'BDFM';

status : "PLANNED WORK";

delays : "<tr><td><h1>Northbound F trains are running on the A line</h1></td></tr><tr><td>Take a southbound train to Jay St bla bla bla</td></tr>";

date : "10/28/2015";

time : " 1:11PM";

}

]

Proposed pseudo-code, open to suggestions:

[

{

uniqueDisruptionID : "289af89c9a9220bcd"; // This is randomly generated when the disruption information is first created and remains constant as the disruption changes status or is modified

lastModified : "10/28/2015 18:00 GMT+0400"; // This allows a developer to know if changes have been made since they last received this disruption information

disruptionType : 1; // (0:undefined, 1:track change, 2:station closure, 3:line closure, etc.)

disruptionStatus : 1; // (0:pending, 1:active, 2:finished, 3:cancelled, etc.)

plannedDisruption : 1; // (0:unplanned, 1:planned)

expectedStart : "10/27/2015 12:00 GMT+0400"; // (can be omitted if unknown)

expectedEnd : "10/29/2015 14:00 GMT+0400"; // (can be omitted if unknown)

linesAffected : ("F");

stationsAffected : ("F20", "F21", "F22");

titleText : "Northbound F trains are running on the A line";

detailText: "Take a southbound train to Jay St bla bla bla";

formattedText : "<tr><td><h1>Northbound F trains are running on the A line</h1></td></tr><tr><td>Take a southbound train to Jay St bla bla bla</td></tr>";

}

]

--
You received this message because you are subscribed to the Google Groups "mtadeveloperresources" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperreso...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nathan Johnson]

The MTA already uses GTFS-realtime for subway/LIRR/MNR and SIRI for buses -- both GTFS-realtime and SIRI provide a standard way to communicate service alerts in a machine-readable format that specifies the routes, trips, stops, etc. affected. For bus service, the MTA does specify routes and directions affected (but not stops) in the SIRI "SituationExchangeDelivery" element. I'm not familiar with the MTA's implementation of GTFS-realtime, but the documentation for the subway feed mentions that "[t]he only alerts included in the NYCT Subway GTFS-realtime feed are notifications about delayed trains", which suggests that scheduled service changes are not included. Instead of reinventing the wheel, the MTA need only implement the standards it is already using; i.e., GTFS-realtime and SIRI. This would allow journey planners to take service changes into account when recommending routes (instead of suggesting journeys that are not possible!)

References

https://developers.google.com/transit/gtfs-realtime/service-alerts

http://user47094.vs.easily.co.uk/siri/schema/1.3/doc/Handbook/Handbookv15.pdf#page=77

http://bustime.mta.info/wiki/Developers/SIRISituationExchangeDelivery

http://datamine.mta.info/sites/all/files/pdfs/GTFS-Realtime-NYC-Subway%20version%201%20dated%207%20Sep.pdf#page=12

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperresources+unsub...@googlegroups.com.

[Samuel Wong]

Nathan,

That's some good feedback.  We are internally discussing altetnatives and modifications if necessary, but keep the constructive feedback coming along.

Sam

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperreso...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mtadeveloperresources" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperreso...@googlegroups.com.

[Aaron Donovan]

TF,

Thanks for the comments, and sorry you're experiencing frustrations.  Is this an opportunity for crowdsourcing?  What if all the developers on this board contributed information about those pieces of our API that they DO know, as well as questions about those they don't?  I don't mind collecting all the information that is known and formatting it in some sort of uniform way, keeping track of the questions, and hopefully getting answers to the questions, if not from other developers, then from MTA sources.  At the end of the whole project, we would be able to post the results right onto pages where we keep the data feeds.  

Other folks feel free to jump in. Is this a workable approach? If yes, I'll start a new sticky thread.  

Let's make this happen!

-Aaron  

[Nathan Johnson]

Aaron,

Thanks for offering to collect and collate this information -- I can certainly think of a lot of details worth including in such a resource and I'm sure other developers will have useful information to add based on their own experiences. However, what happens after you've finished the project and posted the results? What happens when the MTA changes/adds to its APIs or other developers have further insights to add? I think the establishment of a developer wiki would be a more comprehensive and sustainable approach -- information could be easily corrected and kept up to date by both the MTA and the developer community, which would be less work for the MTA in the long run. Such a wiki would encourage and maintain developer engagement, minimize the frustrations of incorrect or incomplete documentation, and consolidate relevant information spread out across the MTA website. An example of such a resource is the Open Rail Data Wiki (hosted by Rockshore, contractor to Network Rail) for developers working with British railway data. The wiki includes a FAQ page that registered users can add to, code examples for working with the APIs, a catalog of relevant data sources, and a list of projects using the data. The Open Rail Data Wiki is regularly cited on their own Google group. I found it an invaluable resource when making sense of British railway data. If you think this would be a good solution, I'd be happy to help set it up. I would also be happy to discuss this further, e.g. over the phone, if you'd find it useful.

Nathan

[Yakov Okshtein]

I think the developer wiki Nathan proposed would be quite helpful.

Personally, I did have to look around a bit trying to find the info necessary to make a solution, eventually finding it in pieces - some with the GTFS spec, some on MTA's website, some in this group's archives, etc.

--

You received this message because you are subscribed to the Google Groups "mtadeveloperresources" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperreso...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Yakov Okshtein
US: 201 873 6487
IL: 052 865 7288 (+972 52 865 7288)
fly...@gmail.com

[Richard Dunks]

+1 for the wiki. I think it's a great way to share the knowledge of the community in a way that's easily accessible for the entire community. I'd also be happy to help share the load along with the rest of the community and the MTA for maintaining it. 

Richard Dunks, Founder

Datapolitan, LLC

www.datapolitan.com

@datapolitan

[Aaron Donovan]

OK, great idea, guys.   Let me look into this.  

-Aaron

On Monday, November 16, 2015 at 2:25:25 PM UTC-5, Richard Dunks wrote:

+1 for the wiki. I think it's a great way to share the knowledge of the community in a way that's easily accessible for the entire community. I'd also be happy to help share the load along with the rest of the community and the MTA for maintaining it. 

On Thu, Nov 12, 2015 at 8:11 AM, Yakov Okshtein wrote:

I think the developer wiki Nathan proposed would be quite helpful.

Personally, I did have to look around a bit trying to find the info necessary to make a solution, eventually finding it in pieces - some with the GTFS spec, some on MTA's website, some in this group's archives, etc.

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperresources+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Yakov Okshtein
US: 201 873 6487
IL: 052 865 7288 (+972 52 865 7288)
fly...@gmail.com

--
You received this message because you are subscribed to the Google Groups "mtadeveloperresources" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperresources+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nathan Johnson]

Hi Aaron,

Has there been any progress concerning the possible MTA Developer Wiki?

Nathan

[Nathan Johnson]

Hi Sam,

What was the outcome of your internal discussions? Are there any plans to communicate service changes in a machine-readable format (that can be reliably parsed and matched to the static schedules)?

Thanks,

Nathan

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperresources+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "mtadeveloperresources" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperresources+unsub...@googlegroups.com.