Need help parsing protobuff data from MTA API

[Glenn Tatum]

Hello mtadevelopers,

I am looking for help on parsing the incoming protobuff data from the MTA API

I have compiled the necessary proto files to utilize with my python scripts

But when trying to read the protobuff data I get the error:

google.protobuff.message.DecodeError: Error Parsing Message

Files are attached below

Here is my directory structure

mta/

- gtfs-realtime.proto

- nyct.proto

- gtfs_realtime_pb2.py

- nyct_pb2.py

- nyc.proto

- data-reader.py

- data-receiver.py

- out.bin

data-receiver.py

import requests

url = 'https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-ace'

headers = {'x-api-key': 'my-api-key'}

r = requests.get(url, headers=headers)

with open('data-encoded.bin', 'wb') as f:
    f.write(r.text.encode())

data-reader.py

import nyct_pb2
import gtfs_realtime_pb2

with open('out.bin', 'rb') as f:
    read_nyct = nyct_pb2.NyctTripDescriptor()
    read_nyct.ParseFromString(f.read())
    print(read_nyct)

Thank you for your help.

[James Fennell]

The root message type in a GTFS realtime file is FeedMessage. So instead of

> read_nyct = nyct_pb2.NyctTripDescriptor()

you need to have

> read_nyct = gtfs_realtime_pb2.FeedMessage()

James

--

You received this message because you are subscribed to the Google Groups "mtadeveloperresources" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mtadeveloperreso...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mtadeveloperresources/1fc359fb-dcc5-4f9f-9186-0d2cad552eden%40googlegroups.com.

Attachments:

• out.bin
  
• data-receiver.py
  
• data-reader.py
  

[Glenn Tatum]

Hello everyone I have solved my issue.

Issues:

• The first issue was using the requests.text method instead of the requests.content method as requests.content is designed for a binary response.

• Second issue as James message pointed out was that the root message type in a GTFS realtime file is FeedMessage.

Solution:

There is a github (https://github.com/MobilityData/gtfs-realtime-bindings) repository that provides examples for parsing the GTFS response data in Java, Javascript, Python, and C#

In there python directory they had a complete example of getting the data.

Here is the new code that works if anyone else runs into the issue with the api key redacted.

import requests

import gtfs_realtime_pb2

url = 'https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-ace'

headers = {'x-api-key': 'my-api-key'}

response = requests.get(url, headers=headers)

gfeed = gtfs_realtime_pb2.FeedMessage()

gfeed.ParseFromString(response.content)

print(gfeed)

Thanks for helping.