Any advice on decoding GTFS in chunk streams?

59 views
Skip to first unread message

Daniel Z

unread,
Oct 10, 2024, 9:52:38 AMOct 10
to mtadeveloperresources
I am currently trying to read realtime MTA GTFS and I run into a memory error as my pico w has too little ram. I figured if I chunk the response.content into parts I can have less overhead but I run into the issue of trying to figure out how to break up the bytes into different parts without losing info (e.g what if my chunk "cuts" off part of a message")

Thank you

Daniel

dude

unread,
Oct 11, 2024, 7:54:44 AMOct 11
to mtadeveloperresources
Maybe try running a server for the raspberry pi that does all the work for it?

Also I have never done ”chunking” before but maybe you could split up each chunk by finding every “}” so you ensure you don’t mess up the information like if it’s the stopTimeUpdate array in the arrivals response then divide length by x or and round that result and then simultaneously process array[0…(Math.round(array.length / x))] and array[Math.round(array.length / x)), Math.round(array.length / x)) * 2] or something like that???

I’m not sure what data your trying to chunk sorry if this is completely not what you want :(



Reply all
Reply to author
Forward
0 new messages