Need help parsing protobuff data from MTA API

93 views
Skip to first unread message

Glenn Tatum

unread,
Jun 5, 2022, 9:22:27 AMJun 5
to mtadeveloperresources
Hello mtadevelopers,

I am looking for help on parsing the incoming protobuff data from the MTA API

I have compiled the necessary proto files to utilize with my python scripts

But when trying to read the protobuff data I get the error:

google.protobuff.message.DecodeError: Error Parsing Message

Files are attached below

Here is my directory structure

mta/
- gtfs-realtime.proto
- nyct.proto
- gtfs_realtime_pb2.py
- nyct_pb2.py
- nyc.proto
- data-reader.py
- data-receiver.py
- out.bin

data-receiver.py

import requests

url = 'https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-ace'

headers = {'x-api-key': 'my-api-key'}

r = requests.get(url, headers=headers)

with open('data-encoded.bin', 'wb') as f:
    f.write(r.text.encode())

data-reader.py

import nyct_pb2
import gtfs_realtime_pb2

with open('out.bin', 'rb') as f:
    read_nyct = nyct_pb2.NyctTripDescriptor()
    read_nyct.ParseFromString(f.read())
    print(read_nyct)

Thank you for your help.


out.bin
data-receiver.py
data-reader.py

James Fennell

unread,
Jun 5, 2022, 2:04:09 PMJun 5
to mtadeveloperresources
The root message type in a GTFS realtime file is FeedMessage. So instead of

> read_nyct = nyct_pb2.NyctTripDescriptor()

you need to have

> read_nyct = gtfs_realtime_pb2.FeedMessage()


James
--
You received this message because you are subscribed to the Google Groups "mtadeveloperresources" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperreso...@googlegroups.com.


Attachments:
  • out.bin
  • data-receiver.py
  • data-reader.py

Glenn Tatum

unread,
Jun 6, 2022, 11:00:43 AMJun 6
to mtadeveloperresources
Hello everyone I have solved my issue.

Issues:
  • The first issue was using the requests.text method instead of the requests.content method as requests.content is designed for a binary response.
  • Second issue as James message pointed out was that the root message type in a GTFS realtime file is FeedMessage.
Solution:

There is a github (https://github.com/MobilityData/gtfs-realtime-bindings) repository that provides examples for parsing the GTFS response data in Java, Javascript, Python, and C#

In there python directory they had a complete example of getting the data.

Here is the new code that works if anyone else runs into the issue with the api key redacted.

import requests
import gtfs_realtime_pb2


headers = {'x-api-key': 'my-api-key'}

response = requests.get(url, headers=headers)

gfeed = gtfs_realtime_pb2.FeedMessage()

gfeed.ParseFromString(response.content)

print(gfeed)

Thanks for helping.
Reply all
Reply to author
Forward
0 new messages