Green line predictions

[Jeff Lopes]

Is there an ETA on the remaining Green line stations getting predictions? Park Westbound, for example. 

If it's going to be a while, would you consider putting stops away in the API? 

Thanks,

Jeff

[Jeff Lopes]

Any thoughts on this? I'm readying an app update and I'm trying to decide if I rope off certain Green line stations or leave them in knowing the data is coming soon. 

[Izzy Vivian Dupree]

I think stations like Park Westbound aren't getting time based
predictions. I think it's because Green Line dispatching is hard for
MBTA's computers to predict, and also passenger boarding times downtown
are so unpredictable. (The terminus stops like Riverside also don't have
inbound predictions, and the stops after them like Woodland only have
predictions after a train departs the terminus stop.)

I think app code is supposed to look at the train locations in order to
compute Stops Away numbers. It could be nice if the API could do it for
us in a way that matches the exact way that MBTA's signs do it. (Though
I assume GTFS-realtime doesn't have a field for this sort of prediction.)

> --
> You received this message because you are subscribed to the Google
> Groups "MBTA Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to massdotdevelop...@googlegroups.com
> <mailto:massdotdevelop...@googlegroups.com>.
> To post to this group, send email to massdotd...@googlegroups.com
> <mailto:massdotd...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/massdotdevelopers.
> For more options, visit https://groups.google.com/d/optout.

[Developer at MBTA]

It will require both new software and operational changes to make accurate predictions from Green Line terminals. We expect to make them available next year. 

-developer@mbta

[K Adam White]

Thank you for the update on the timeline. In the interim, would there be any way to get visibility into how "stops away" is calculated, so that our apps can have parity with the platform signs? I believe including those values accurately would do a lot to boost confidence both in the reliability and utility of our API-consuming applications, but also the perceived reliability of the green line itself. 

Best,

KAdam

--
You received this message because you are subscribed to the Google Groups "MBTA Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelop...@googlegroups.com.
To post to this group, send email to massdotd...@googlegroups.com.

[Ken Yee]

On Sunday, September 4, 2016 at 3:23:29 PM UTC-4, K. Adam White wrote:

Thank you for the update on the timeline. In the interim, would there be any way to get visibility into how "stops away" is calculated, so that our apps can have parity with the platform signs? I believe including those values accurately would do a lot to boost confidence

I'm curious where the platform signs get data from as well if that data isn't really available? :-)
Although sometimes it seems like a random number generator when I'm waiting ;-)

[StefanW]

It would be a help to know even something as basic as if the "stops away" messages are based 100% on train positions, or if there's any time factor to it.

For example: at Haymarket Green Line westbound does "1 stop away" come from nothing more than a train being at North Station on the westbound track?

I agree with KAdam - if you (MBTA folks) could share with us the algorithms / logic that you use for the signs, then that would be better than nothing until next year.

Thanks!

[Developer at MBTA]

Hello all,

Per these requests, we will be putting together a document describing how to match the "stops away" values on the relevant Green Line station signs.

In the interim, we can answer Stefan's question: They are based on the location of a train and the route indicated by the AVI system on that train. For example, if you are at Haymarket on the Westbound platform waiting for an E branch train, and that train is using the normal "Lechmere to Heath Street" route code, then the meanings of the "stops away" on the sign would be as follows:

"Now boarding" - the train is on the platform at Haymarket

"Approaching" - the train has departed North Station WB but not yet reached Haymarket

"1 stop away" - the train is on the platform at North Station WB

"2 stops away" - the train is either on the platform at Science Park WB or has departed Science Park WB but not yet reached North Station

"3 stops away" - the train is either on the platform at Lechmere or has departed Lechmere but not yet reached Science Park

There are certain oddities such as how we treat the North Station Turnback (we count it as a separate stop because trains often lay over for a significant amount of time there), but the above may be useful as a starting point. We will compile more detailed information into a spreadsheet and provide it for download sometime this week.

Sincerely,

Developer@MBTA

[StefanW]

Superb!!! This is great information - and exactly what I had in mind. This makes my day! I look forward to the additional info / spreadsheet. 

Thank you so much!!

[Developer at MBTA]

Hi all,

The document describing the "Stops Away" configuration can be found here: https://www.dropbox.com/s/a5xtacqw44ijyaw/Stops_Away_for_developers.xlsx?dl=0

Please note first: "Stops Away" has always been intended as a temporary solution while we improve our ability to provide time-based predictions at stops near terminals. This document is being provided to developers for informational purposes, to allow you all to produce outputs that match what shows up on the station signs, if you wish to do so. We've attempted to include all information that would be necessary to reproduce the "Stops Away" format, but it's possible that we've made mistakes or missed something in the transcription, so please test thoroughly and report any issues in this thread.

The spreadsheet is organized as follows:

• Each tab represents one of the four Eastern terminals of the Green Line (Park Street, Government Center, North Station, and Lechmere). This information should be available via the trip_headsign field in the API.
  
• In each tab, the rows represent the segment of track where the train is, and columns represent the location of the relevant sign. The values in the cells represent the number of stops away shown on the sign or "Approaching" or "Now boarding" if that is the text on the sign. "REMOVE" means that the train should be removed from the sign if it is there.
  
• The "Segments" tab allows you to look up a segment name based on the latitude and longitude produced by the API (there are currently a limited set of lat-long pairs that are output when trains are in this area). If the latitude/longitude for a train is not found in this tab, the train must be outside of the area for which we produce "Stops Away" outputs.
  

Feel free to test this data out and let us know how it works. If there's something we can do to make it easier to reproduce "Stops Away" we will try to help out.

Thanks,

Developer@MBTA

[Jeff Lopes]

Awesome! Thanks for taking the time to put this together. 

One quick question that I think you answered, but just making sure. Will all trains in these areas have the exact lat/long in the spreadsheet or should we be calculating which of those segments the train is closest to? 

On Fri, Sep 9, 2016 at 1:32 PM, Developer at MBTA <deve...@mbta.com> wrote:

Hi all,

The document describing the "Stops Away" configuration can be found here: https://www.dropbox.com/s/a5xtacqw44ijyaw/Stops_Away_for_developers.xlsx?dl=0

Please note first: "Stops Away" has always been intended as a temporary solution while we improve our ability to provide time-based predictions at stops near terminals. This document is being provided to developers for informational purposes, to allow you all to produce outputs that match what shows up on the station signs, if you wish to do so. We've attempted to include all information that would be necessary to reproduce the "Stops Away" format, but it's possible that we've made mistakes or missed something in the transcription, so please test thoroughly and report any issues in this thread.

The spreadsheet is organized as follows:

Each tab represents one of the four Eastern terminals of the Green Line (Park Street, Government Center, North Station, and Lechmere). This information should be available via the trip_headsign field in the API.

In each tab, the rows represent the segment of track where the train is, and columns represent the location of the relevant sign.

The "Segments" tab allows you to look up a segment name based on the latitude and longitude produced by the API (there are currently a limited set of lat-long pairs that are output when trains are in this area).

Feel free to test this data out and let us know how it works. If there's something we can do to make it easier to reproduce "Stops Away" we will try to help out.

Thanks,

Developer@MBTA

On Tuesday, September 6, 2016 at 4:22:51 PM UTC-4, StefanW wrote:

Superb!!! This is great information - and exactly what I had in mind. This makes my day! I look forward to the additional info / spreadsheet. 

Thank you so much!!

--

You received this message because you are subscribed to the Google Groups "MBTA Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelopers+unsubscribe@googlegroups.com.
To post to this group, send email to massdotdevelopers@googlegroups.com.

[Jeff Lopes]

Digging through this a bit more I'm confused about headsigns. For Park WB I would expect headsigns like: Boston College, Cleveland Circle, etc. But the tabs in the spreadsheet only have EB headsigns. Am I missing something? 

[Developer at MBTA]

Hi Jeff,

I'll answer your two questions in order:

One quick question that I think you answered, but just making sure. Will all trains in these areas have the exact lat/long in the spreadsheet or should we be calculating which of those segments the train is closest to?

All trains in these areas will exactly match one of the lat/longs in the spreadsheet. This is because locations in these areas are detected at specific points, which we map to lat/long pairs.

Digging through this a bit more I'm confused about headsigns. For Park WB I would expect headsigns like: Boston College, Cleveland Circle, etc. But the tabs in the spreadsheet only have EB headsigns. Am I missing something?

Headsigns matter only in the following sense: A train that is, for example, on the Eastbound platform at Boylston can be said to be 5 stops away from the Westbound platform at Boylston, if its Eastern terminal is Government Center. However, if its Eastern terminal was Park Street, the same train would only be 3 stops away (because it is going to turn around at Park Street). Western terminals have no affect on this; it doesn't matter whether the train is going to Boston College, Cleveland Circle, or anything else in the Westbound direction; the number of stops away will be the same. Note that on each of the tabs, all of the Westbound track locations have the same "Stops Away" values.

Sincerely,

Developer@MBTA

[Jeff Lopes]

Thanks for the clarification. I played around a bit tonight and it looks like the lat/lon don't exactly match. 

Looking at it on a map, this vehicle looks like it's at North Station: 

{

  "trip_headsign": "North Station",

  "trip_id": "3645_1473480000_1",

  "trip_name": "B train from Boston College to North Station",

  "vehicle": {

    "vehicle_bearing": "230",

    "vehicle_id": "3645",

    "vehicle_lat": "42.36609",

    "vehicle_lon": "-71.06298",

    "vehicle_timestamp": "1473480186"

  }

},

But it doesn't exactly match any of the segments near North Station: 

N_StaEB,42.365434,-71.060395

N_StaEB,42.365629,-71.060574

N_StaWB,42.365432,-71.060399

N_StaWB,42.365627,-71.060576

N_Sta_Before_TurnbackEB,42.366068,-71.060995

N_Sta_Before_TurnbackEB,42.366087,-71.062981

N_Sta_Before_TurnbackEB,42.366536,-71.061876

N_Sta_HaymarketWB,42.363164,-71.058267

N_Sta_HaymarketWB,42.364274,-71.059356

N_Sta_Sci_ParkEB,42.36574,-71.064664

N_Sta_Sci_ParkEB,42.366269,-71.066489

N_Sta_Turnback,42.365623,-71.064324

N_Sta_Turnback,42.365649,-71.064168

N_Sta_Turnback,42.365718,-71.063846

N_Sta_Turnback,42.36602,-71.063157

N_Sta_Turnback,42.366146,-71.062835

N_Sta_Turnback,42.366151,-71.06284

N_Sta_Turnback,42.366152,-71.062843

N_Sta_Turnback,42.366539,-71.061808

My guess is that the realtime API is rounding and it should match: 

N_Sta_Before_TurnbackEB,42.366087,-71.062981

Is it possible to update the spreadsheet with the exact values the realtime API returns? 

[Jeff Lopes]

If anyone is interested in hacking on this together, I've got something started here: https://github.com/jefflopes/stops-away

Once you clone the repo, run "ruby stops_away.rb <station_id>" to see what it currently generates. <station_id> should be one of: place-boyls, place-pktrm, place-gover, place-haecl, place-north, place-spmnl

create_db.rb takes the data from the spreadsheet (in CSV form) and turns it into a SQLite DB. I'm not totally sure my logic there is correct yet, so feedback welcome. I separated out the Westbound stations that were in common between all the sheets into it's own CSV, so there's one more CSV than there were sheets in the original spreadsheet. 

[Developer at MBTA]

Jeff,

Thanks for pointing that out about the rounding. All latitude/longitude values are indeed rounded to 5 places after the decimal for subway locations. A new spreadsheet has been uploaded with rounded values (use the same link above).

Sincerely,

Developer@MBTA

[Jeff Lopes]

Thanks for updating the latitude/longitude values. I think I have something that's working, but validating the data is difficult short of getting someone to stand in each station at the same time. 

Is there anyone from the MBTA that would be willing to compare my output to what's powering the signs? 

You can find my debug output here: https://api.proximitapp.com/stops_away

[Developer at MBTA]

Hi Jeff,

We were able to spend a few minutes comparing your web output to what's going out to the signs, and it looks good from what we can see. We can't guarantee anything, of course. If you do get any specific reports of discrepancies, please let us know with details and we can try to figure out if some data or configuration is missing.

Sincerely,

Developer@MBTA

Developer@MBTA

Developer@MBTA

To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelop...@googlegroups.com.
To post to this group, send email to massdotd...@googlegroups.com.

Visit this group at https://groups.google.com/group/massdotdevelopers.
For more options, visit https://groups.google.com/d/optout.

On Friday, September 9, 2016 at 2:28:29 PM UTC-4, Jeff Lopes wrote:

Digging through this a bit more I'm confused about headsigns. For Park WB I would expect headsigns like: Boston College, Cleveland Circle, etc. But the tabs in the spreadsheet only have EB headsigns. Am I missing something? 

On Fri, Sep 9, 2016 at 2:10 PM, Jeff Lopes <jlo...@monkeycity.org> wrote:

Awesome! Thanks for taking the time to put this together. 

One quick question that I think you answered, but just making sure. Will all trains in these areas have the exact lat/long in the spreadsheet or should we be calculating which of those segments the train is closest to? 

On Fri, Sep 9, 2016 at 1:32 PM, Developer at MBTA <deve...@mbta.com> wrote:

Hi all,

The document describing the "Stops Away" configuration can be found here: https://www.dropbox.com/s/a5xtacqw44ijyaw/Stops_Away_for_developers.xlsx?dl=0

Please note first: "Stops Away" has always been intended as a temporary solution while we improve our ability to provide time-based predictions at stops near terminals. This document is being provided to developers for informational purposes, to allow you all to produce outputs that match what shows up on the station signs, if you wish to do so. We've attempted to include all information that would be necessary to reproduce the "Stops Away" format, but it's possible that we've made mistakes or missed something in the transcription, so please test thoroughly and report any issues in this thread.

The spreadsheet is organized as follows:

Each tab represents one of the four Eastern terminals of the Green Line (Park Street, Government Center, North Station, and Lechmere). This information should be available via the trip_headsign field in the API.

In each tab, the rows represent the segment of track where the train is, and columns represent the location of the relevant sign.

The "Segments" tab allows you to look up a segment name based on the latitude and longitude produced by the API (there are currently a limited set of lat-long pairs that are output when trains are in this area).

Feel free to test this data out and let us know how it works. If there's something we can do to make it easier to reproduce "Stops Away" we will try to help out.

Thanks,

Developer@MBTA

On Tuesday, September 6, 2016 at 4:22:51 PM UTC-4, StefanW wrote:

Superb!!! This is great information - and exactly what I had in mind. This makes my day! I look forward to the additional info / spreadsheet. 

Thank you so much!!

--
You received this message because you are subscribed to the Google Groups "MBTA Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelop...@googlegroups.com.
To post to this group, send email to massdotd...@googlegroups.com.

Visit this group at https://groups.google.com/group/massdotdevelopers.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "MBTA Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelop...@googlegroups.com.
To post to this group, send email to massdotd...@googlegroups.com.

Visit this group at https://groups.google.com/group/massdotdevelopers.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "MBTA Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelop...@googlegroups.com.
To post to this group, send email to massdotd...@googlegroups.com.

[Jeff Lopes]

Awesome, thanks for checking! 

The code backing that URL is here if anyone else is interested: 

https://github.com/jefflopes/stops-away

Disclaimer: I'm an iOS dev, so my ruby is kind of terrible. 

[Jeff Lopes]

I did notice one issue with the data this morning. B trains at Park Street go from 1 stop away directly to boarding. The train disappears when it should be in the approaching state. I looked up one of the trains (by ID) in the vehiclesbyroutes response and it's not there so I don't think it's an issue with the stops away data, but the vehicle data itself. Is it possible it's disappearing as it turns around at Government Center? What's extra weird is that the sign at Park St did have it during the approaching state. 

To unsubscribe from this group and stop receiving emails from it, send an email to massdotdevelopers+unsubscribe@googlegroups.com.
To post to this group, send email to massdotdevelopers@googlegroups.com.

[Jeff Lopes]

I think I found at least one problem. The lat/longs coming back from the realtime API and what's in the spreadsheet still don't match perfectly. 

Here's a vehicle that is probably in the Park_Loop segment: 

"trip": [

  {

    "trip_headsign": "Park Street",

    "trip_id": "3628_1474572600_1",

    "trip_name": "B train from Boston College to Park Street",

    "vehicle": {

      "vehicle_bearing": "30",

      "vehicle_id": "3628",

      "vehicle_lat": "42.3562",

      "vehicle_lon": "-71.06257",

      "vehicle_timestamp": "1474572998"

    }

  },

But here's how the Park_Loop segment is defined in the spreadsheet: 

Park_Loop,42.35621,-71.06257

Looks like the API sometimes rounds to 4 decimal places. Can you double check the lat/longs in the spreadsheet again? 

Thanks,

Jeff

[Jeff Lopes]

I logged all the vehicles between Lechmere and Symphony for a few hours today and tried to map them to the data in the spreadsheet. It looks like 23 lat/longs don't match segments. Upon initial glance it does appear that many of them are simple rounding errors, but others were harder to map to segments. 

Here's the list of all the unique vehicle lat/longs I saw. I included the segment for ones that matched segments in the spreadsheet. 

42.34252,-71.08517,null

42.34293,-71.08469,null

42.34296,-71.08476,null

42.34534,-71.0817,PrudentialEB

42.3454,-71.08181,null

42.34582,-71.08114,Prudential_CopleyEB

42.34587,-71.08122,null

42.34977,-71.07825,CopleyEB

42.34978,-71.07855,null

42.34996,-71.07792,null

42.34997,-71.07748,Copley_ArlingtonEB

42.35018,-71.0771,null

42.35157,-71.07167,ArlingtonEB

42.35164,-71.07172,null

42.3518,-71.07081,Arlington_BoylstonEB

42.35188,-71.07085,null

42.35245,-71.06483,BoylstonEB

42.35301,-71.06454,Boylston_ParkEB

42.35305,-71.06458,null

42.35356,-71.06432,BoylstonWB

42.35551,-71.06316,null

42.35554,-71.06322,Park_Fence_EB

42.35556,-71.06326,Park_BoylstonWB

42.35559,-71.06335,Park_BoylstonWB

42.35614,-71.06243,Park_Govt_CtrEB

42.3562,-71.06257,null

42.35625,-71.06264,Park_Fence_WB

42.35631,-71.06277,Park_Wall_WB

42.35927,-71.05946,Govt_CtrEB

42.35929,-71.05952,null

42.35986,-71.05891,null

42.35989,-71.05897,Govt_CtrWB

42.36234,-71.05804,Haymarket_Govt_CtrWB

42.36238,-71.05804,HaymarketWB

42.36242,-71.05803,HaymarketEB

42.36284,-71.05811,HaymarketWB

42.36316,-71.05827,N_Sta_HaymarketWB

42.36389,-71.05898,null

42.36427,-71.05936,N_Sta_HaymarketWB

42.36494,-71.05996,Haymarket_N_StaEB

42.36543,-71.06039,null

42.36543,-71.0604,N_StaWB

42.36563,-71.06058,N_StaWB

42.36574,-71.06467,null

42.36599,-71.06321,Sci_Park_N_StaWB

42.36607,-71.061,N_Sta_Before_TurnbackEB

42.36609,-71.06298,N_Sta_Before_TurnbackEB

42.36633,-71.0669,Sci_Park_N_StaWB

42.36649,-71.06204,null

42.36668,-71.06772,Sci_ParkWB

42.36672,-71.06772,Sci_ParkEB

42.36689,-71.06815,null

42.36691,-71.06811,Sci_ParkEB

42.36758,-71.06944,Sci_Park_LechmereEB

42.36789,-71.07014,null

42.36857,-71.07137,null

42.36925,-71.07278,Lechmere_Sci_ParkWB

42.36959,-71.07338,null

42.37061,-71.07579,Lechmere

42.37062,-71.07565,Lechmere

42.37093,-71.07721,Lechmere

42.37108,-71.07723,Lechmere

42.37123,-71.07801,Lechmere_Rev

[Jeff Lopes]

Any thoughts on this? I'm a bit blocked currently. Some of them I can figure out by rounding them differently, but others are too far away from any known segments for me to make a reasonable guess. 

[Developer at MBTA]

Hi Jeff,

Thanks for your work on this. Some of the key locations, such as the North Station Turnback, are not currently making it all the way to the public API as they are not considered "revenue service" locations. We will change this as soon as possible so that all of the relevant locations are exposed in the public API. Also, as you note, there was an issue with rounding and some of the values in the spreadsheet are off by one in the last digit. We appreciate your work on trying to get "Stops Away" into your app and hope to resolve these issues as soon as possible.

Sincerely,

Developer@MBTA