New York MTA - Draft GTFS Realtime Spec

[Aaron Donovan]

Hi everyone,

The New York Metropolitan Transportation Authority and MTA New York
City Transit are designing a real time developers' web feed of subway
arrival estimates ("countdown clock data") for the 1, 2, 3, 4, 5 and 6
subway lines. As the MTA and NYC Transit design this web service, we
would like to receive feedback from developers on phase one, a web
feed of GTFS-realtime data, and phase two, a web feed using SIRI.

We have created a working draft GTFS Realtime Spec and posted it at
the link below. Please have a look at this document.

http://httqa.mta.info/developers/pdfs/GTFS-Realtime-NYC-Subway.pdf

If you have any suggestions for how it could be improved, please let
us know. We are all ears. The purpose of this spec is to serve GTFS-RT
developers and ultimately the general public.

Folks from our GTFS-RT team will be monitoring this board and will
available to respond to questions and take down feedback.

This message is cross-posted at our home board, MTA Developer
Resources.
https://groups.google.com/group/mtadeveloperresources
https://groups.google.com/group/mtadeveloperresources/browse_thread/thread/d0a88db9aa552e19

Many thanks,
Aaron Donovan
Spokesman
Metropolitan Transportation Authority

[Csaba Garay]

Hi Aaron,

I'm really glad and excited about your post! 

After a quick look I'd like to comment the following:

"The feed is a full dataset and contains two kinds of entities: trip updates and alerts."

According the the GTFS-realtime spec (here):

"Even though the gtfs-realtime.proto syntax allows multiple entity types to be mixed for a feed, only one type of entity can be used in a particular feed."

This means that you will need to provide 2 separate feeds, one containing trip update only the other alerts only.

Regards,

Csaba Garay

Partner Technology Manager

GTech - Partner Solutions - Google Switzerland GmbH

--
You received this message because you are subscribed to the Google Groups "GTFS-realtime" group.
To post to this group, send email to gtfs-r...@googlegroups.com.
To unsubscribe from this group, send email to gtfs-realtim...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gtfs-realtime?hl=en.

[Adam Ernst]

> According the the GTFS-realtime spec (here):
> "Even though the gtfs-realtime.proto syntax allows multiple entity types to be mixed for a feed, only one type of entity can be used in a particular feed."
>
> This means that you will need to provide 2 separate feeds, one containing trip update only the other alerts only.

Why? That seems like a bizarre requirement, especially since Service Alerts can target Trips. With two separate feeds, there's the possibility that I could get out-of-sync info about a particular trip.

Adam

[Stefan de Konink]

On Fri, 18 May 2012, Aaron Donovan wrote:

> As the MTA and NYC Transit design this web service, we
> would like to receive feedback from developers on phase one, a web
> feed of GTFS-realtime data, and phase two, a web feed using SIRI.

Could you please elaborate why you choose for the FULL_DATASET method? It
plainly doesn't scale. I see you are not using realtime positions, so it
could just 'do' for the small numbers of line published, but for anything
serious I would love to see other DIFFERENTIAL implementations like we did
for The Netherlands.

What are you currently using as 'validator' for the published dataset.
Practically do you trust your code is semantically producing the correct
results or do you have validation tools build?


Stefan

[Brian Ferris]

Personally, I never understood the need to split the different entity types up into different feeds.  Casaba, do you remember the original context behind that decision on the spec?  I'd vote to consider relaxing that requirement if others are interested.

Brian

[Brian Ferris]

Define "doesn't scale".

Based on Mike's numbers, a worst-case GTFS-realtime feed for the entire NYC subway is ~ 500kb uncompressed and ~ 10kb compressed.  I don't think transferring and uncompressing a 10kb file every 30 seconds is going to be too difficult for anyone.  Neither should processing the records.

In the end, asking an agency to support differential mode just makes things more complex to implement for the agency, with what I'd consider to be only marginal improvements in utility for clients.

But we've had this debate at least three different times across as many different mailing lists at this point, so maybe we'll just have to agree to disagree ; )

Brian

--
You received this message because you are subscribed to the Google Groups "GTFS-realtime" group.
To post to this group, send email to gtfs-r...@googlegroups.com.

To unsubscribe from this group, send email to gtfs-realtime+unsubscribe@googlegroups.com.

[Csaba Garay]

My problem with mixing different types is feed consumers need to download the whole feed even if they are only interested in one type of entities.

If you have 600 trip updates that constantly change and 5-6 less frequently changing alerts it just seems a waste of resources on both ends to weed through the data.

If they are in separate feeds you can subscribe to the ones you are interested in.

Moreover alert usage as it is in the mta spec seems to duplicate what's available in trip updates if both absolute times and relative delays are provided.

This would also remove the need for additional alert entity wrapper (duplicating trip_ids as informed entities).

Developers can filter for delay > threshold1 to get the same info as you would provide with the alerts.

I also don't see if there's a predicted delay in 20 minutes from now how would that be modelled with alerts?

Regards,

Csaba

[Vajko, Peter]

Csaba,

> Moreover alert usage as it is in the mta spec seems to duplicate what's
> available in trip updates if both absolute times and relative delays
> are provided.

We won't provide delays, only absolute times.

> Developers can filter for delay > threshold1 to get the same info as
> you would provide with the alerts.

The alert is there for trains displayed on the signs as 'delayed'. However, this is quite independent from the delay (the actual deviation from the schedule), if there is an alert, it means the train is 'stalled', that is, not moving for some reason (which will most likely result in a delay).

You'll see no alert for a trip as long as the train is moving, no matter how high the deviation is.

Peter

This message and any attachments are solely for the use of intended recipients. The information contained herein may include trade secrets, protected health or personal information, privileged or otherwise confidential information. Unauthorized review, forwarding, printing, copying, distributing, or using such information is strictly prohibited and may be unlawful. If you are not an intended recipient, you are hereby notified that you received this email in error, and that any review, dissemination, distribution or copying of this email and any attachment is strictly prohibited. If you have received this email in error, please contact the sender and delete the message and any attachment from your system. Thank you for your cooperation

[tomcola58]

Just to add to what Peter said. The original approach was not to
provide Alert entities. The way our system works when a train is
declared "stalled" for an unknown reason or not moving for a
configurable period of time the arrival time is removed from the sign
and replaced with "Delay". During internal discussion it was realized
that a customer on a Platform could see Delay on the sign and 4 mins
on a mobile device. It was decided that we should communicate that
stalled condition across the interface. These delays are unplanned
events an will the trigger will only be transmitted when the delay
happens. When the train moves again the stall is reset and the the
alert is removed.

[Brian Ferris]

So a slightly different issue to talk about with respect to a NYC MTA Subway realtime feed.

As described in the doc, the realtime info represents a complete snapshot of the system at a particular point in time.  Consider a situation where trips A, B, and C are scheduled in the GTFS, but we only see two active trips in the realtime feed.  If we match those two active trips to scheduled trips A and C, then that implies that trip B isn't running and shouldn't be shown to users.

The trick is you won't ever see an explicit "trip B is cancelled" update or indeed any mention of trip ids A, B, or C because the MTA isn't doing trip matching due to the highly dynamic nature of their system.  The lack of matching trip ids is fine with me, but I'm wondering if we should add something to the spec to indicate "this feed replaces ALL schedule information."

As an example to illustrate my point, there are GTFS-realtime feeds for bus networks where we sometimes don't get a trip update for a bus because it's just not being tracked in the system (broken radio or GPS unit, urban canyon, etc).  In this situation, we don't actually assume that the scheduled trip has been cancelled and shouldn't be shown to the user because there is actually a pretty good chance the vehicle is still running out there some place.

I think the original authors of the GTFS-realtime spec sort of anticipated this and wanted to use the TripUpdate.TripDescriptor.ScheduleRelationship value of REPLACEMENT to indicate this situation.  That said, if you dig down into the spec here, you quickly realize that it gets very complicated in a hurry and it probably doesn't cover the NYC case anyway.

So long story short, here is my proposal:

1) We drop the REPLACEMENT functionality from the spec.  No one is using it and if it's not going to be used in the NYC case, I'd prefer we remove it to make the spec more clear.

2) We consider adding a flag like "boolean trip_updates_are_replacements" (naming to be determined) to the indicate that the feed.  The NYC MTA subway feed could set this flag to indicate that their feed specifies the entire state of the system.

Thoughts?

Brian

On Sat, May 19, 2012 at 2:05 AM, Aaron Donovan <adon...@mtahq.org> wrote:

[Frumin, Michael]

Brian,

 

This idea (“this feed replaces ALL schedule information”) is, in my opinion, exactly what is needed for this use case.  I thought I had communicated that to Sasha a while back, but I guess I failed.

 

Thanks,

Mike

[Brian Ferris]

So is that a +1 for adding this to the NYC MTA feed?

[Vajko, Peter]

Brian,

> 2) We consider adding a flag like "boolean
> trip_updates_are_replacements" (naming to be determined) to the
> indicate that the feed. The NYC MTA subway feed could set this flag to
> indicate that their feed specifies the entire state of the system.

Makes sense. I'd probably add a timeframe attribute as well. Let me explain why.

The feed consumers inevitably will face the problem of removed trips. If trip A is in the GTFS feed but not in the GTFS-RT, at some point of time, as a feed consumer, you need to drop trip A from your prediction.

In order to make that decision, you may want to know for which time window is this feed replaces the trips. You may have some uncertainty with trips on or just below the GTFS-RT time horizon depending on your algorithm but I can't think of any good solution to that.

Currently, the time window for the upcoming NYCT subway GTFS-RT feed is 30 minutes with updates every 5 minutes. What you'll see is that every five minutes a bunch of new trips will show up in the feed. This are the trips scheduled to start in 30 to 35 minutes. Let me give you real life an example, extracted from a log:

2012.04.05;16:20:56:272 SC 01 1651 238/SFT
2012.04.05;16:20:56:366 SC 01 1651+ SFT/242
2012.04.05;16:20:56:428 SC 03 1651+ NLT/148
2012.04.05;16:20:56:662 SC 06 1652 BBR/177
2012.04.05;16:20:56:662 SC 05 1652 FLA/DYR
2012.04.05;16:20:56:662 SC 0S 1652+ GCS/TSS
2012.04.05;16:20:56:662 SC 0S 1653 TSS/GCS
2012.04.05;16:20:56:678 SC 05 1653+ DYR/FLA
2012.04.05;16:20:56:756 SC 02 1653+ NLT/241
2012.04.05;16:20:56:866 SC 03 1654 148/NLT

As you see, at 16:20:56, you get 10 new trips scheduled (SC) to start between 16:51 and 16:54. But the feed is a snapshot generated only every 30 seconds, so it could be tricky to decide at 16:21:10 if you have to drop a GTFS trip starting at 16:55 because it is not in the GTFS-RT feed. But it seems to be a safe bet to drop everything from the static GTFS that is scheduled to start in 30 minutes but is missing from the GTFS-RT feed.

Peter Vajko
Siemens Industry, Inc.
498 Seventh Avenue
16th Floor
New York, NY 10018
Tel: +1 (212) 672-4053
Fax: +1 (212) 672-4001
mailto:peter...@siemens.com

[Brian Ferris]

Maybe instead of a "replacement" flag, we could add an optional "replacement" time range instead.  When specified, it would indicates that the trip updates in the feed replace all schedule service that would be active at any time between the start and end time.  Thus, in your feed, you could specify a time range starting now and extending to ~35 minutes in the future to indicate the exact range of service.

Brian

[Vajko, Peter]

Hello Everyone,

As you already know, we'd add some extensions to the upcoming NYCT Subway feed, (see http://httqa.mta.info/developers/pdfs/GTFS-Realtime-NYC-Subway.pdf).

I know Brian had a post earlier (https://groups.google.com/forum/?fromgroups#!topic/gtfs-realtime/F4FwskhZNyE) on adding extension namespaces.

I don't know how exactly we want to coordinate this, so I just attached the proto file containing the definitions of the NYCT Subway extensions. I simply used the first available extension tags.

Brian, please let me know what tag values you want us to use.

Thanks,

[Brian Ferris]

So here's my two cents on how we should handle extensions to GTFS-realtime in general and for the NYC case specifically.  I've actually already been playing around with a few GTFS-realtime extensions in OneBusAway, which has influenced my thoughts on this.  Here is the proposal:

1) When a developer is interested in extending the GTFS-realtime protocol, we assign them an extension id, picked incrementally from a list of numbers starting at 1000 and going up. Not coincidentally, this corresponds to the tag ids available in the "extension" namespace for each GTFS-realtime message definition.

2) We will maintain a list of assigned ids on the main GTFS-realtime devsite, along with some details about the developer(s) responsible for the extension, links to documentation, etc.

3) Now that the developer has an assigned extension id, they will use that id when extending any and all GTFS-realtime messages.  Even if the developer only plans to extend a single message, the assigned extension id will be reserved for ALL messages.  I think this is a good approach because it will mean we only have to maintain one extension id assignment list, as opposed to 20+ lists for each of the message types.  I'm also not worried about running out of extension ids, since we're starting with 1000 and I'd be MORE than happy to extend the namespace if we get that many developers ; )

4) For a developer extending the spec, instead of adding a single field like a "string" or "int32" with their extension id, the preferred model is to define a new message like "MyAwesomeTripUpdateExtension", extend the underlying GTFS-realtime message with your new message, and then put all your new fields in there.  This has the nice property that you can manage fields within your extension message however you want, without needing to reserve a new extension id from the master list.

So as a quick example, I've put all my OneBusAway extension fields in their own messages and added my extensions using id 1000 (I call dibs!).  You can see the resulting proto definition here:

https://github.com/OneBusAway/onebusaway-gtfs-realtime-api/blob/master/src/main/resources/com/google/transit/realtime/gtfs-realtime-OneBusAway.proto

As another example, let's consider NYC MTA.  We could assign them extension id 1001 and rewrite their proto like the following:

message NyctTripDescriptor {

  optional string train_id = 1;

  optional bool is_assigned = 2; 

}

extend transit_realtime.TripDescriptor {

  optional NyctTripDescriptor nyct_trip_descriptor = 1001;

}

message NyctStopTimeUpdate {

  optional string track = 1;

}

extend transit_realtime.TripUpdate.StopTimeUpdate {

  optional NyctStopTimeUpdate track = 1000;

}

How does this sound to everybody?  In general, I think this approach will minimize the amount of coordination we need to do on the list and give developers more flexible to add new extensions as they see fit.  And of course, the ultimate goal is to move promising extensions back into the core specification.

Brian

[Matt Conway]

I think the section

extend transit_realtime.TripUpdate.StopTimeUpdate {

  optional NyctStopTimeUpdate track = 1000;

}

should read

extend transit_realtime.TripUpdate.StopTimeUpdate {

  optional NyctStopTimeUpdate track = 1001;

}

Matt

[Brian Ferris]

Whoops, you are indeed correct.

[Vajko, Peter]

Brian,

We'll go with your recommendation then.

Is the 1001 for the TripDescriptor and StopTimeUpdate officially assigned now to the NYCT subway? If so, I'll post an updated proto.

[Brian Ferris]

Yeah, that works for me.

Does anyone else see any issues with the proposed extensions policy?  If I don't hear anything in the next 24 hours, I'm going to assume we've got consensus on this and make it official.

Brian

[tomcola58]

I am good with this.

Tom C

On May 29, 10:06 am, Brian Ferris <bdfer...@google.com> wrote:
> Yeah, that works for me.
>
> Does anyone else see any issues with the proposed extensions policy?  If I
> don't hear anything in the next 24 hours, I'm going to assume we've got
> consensus on this and make it official.
>
> Brian
>

> On Tue, May 29, 2012 at 4:02 PM, Vajko, Peter <peter.va...@siemens.com>wrote:
>
>
>
>
>
>
>
> > Brian,
>
> > We'll go with your recommendation then.
>
> > Is the 1001 for the TripDescriptor and StopTimeUpdate officially assigned
> > now to the NYCT subway? If so, I'll post an updated proto.
>
> > Peter Vajko
> > Siemens Industry, Inc.
> > 498 Seventh Avenue
> > 16th Floor
> > New York, NY 10018
> > Tel: +1 (212) 672-4053
> > Fax: +1 (212) 672-4001

> > mailto:peter.va...@siemens.com

[Brian Ferris]

Ok, hearing no complaints, I have updated the site documentation officially outlining extension support:

https://developers.google.com/transit/gtfs-realtime/changes

NYC MTA guys: take a look at the details I added for your entry and let me know if you would like anything changed.

Thanks,

Brian

[Vajko, Peter]

> NYC MTA guys: take a look at the details I added for your entry and let
> me know if you would like anything changed.

Attached our new proto file. I only changed

extend transit_realtime.TripUpdate.StopTimeUpdate {
optional NyctStopTimeUpdate track = 1001;
}

to

extend transit_realtime.TripUpdate.StopTimeUpdate {
optional NyctStopTimeUpdate nyct_stop_time_update = 1001;
}

makes more sense to me.

Peter Vajko
Siemens Industry, Inc.
498 Seventh Avenue
16th Floor
New York, NY 10018
Tel: +1 (212) 672-4053
Fax: +1 (212) 672-4001

mailto:peter...@siemens.com

[Vajko, Peter]

> Based on Mike's numbers, a worst-case GTFS-realtime feed for the entire
> NYC subway is ~ 500kb uncompressed and ~ 10kb compressed.

Just ran a playback recorded on a Friday morning rush hour, NYC subway A division.

Had 255 trains in the feed, 197 of them running (assigned), which is pretty close to the maximum you'll get from the A division.

The feed is ~160k uncompressed, 42k compressed.

[Brian Ferris]

Hey Peter,

Any thoughts about adding a "replacement" TimeRange to the header to indicate the time range for assigned vehicles?

Brian

[Vajko, Peter]

> Any thoughts about adding a "replacement" TimeRange to the header to
> indicate the time range for assigned vehicles?

I think it absolutely makes sense and I'm happy to add it (and any other data we already have) if the community finds it useful.

As I recall you wanted it in the standard GTFS-realtime, not as an extension, and I guess you guys are better at defining generic requirements, Tom and I tend to be more focused on the NYC subway.

[Brian Ferris]

The normal procedure is that we usually add something as an extension first and verify that it actually works the way everyone intended before officially adding it to the spec.  So if it's ok with you, it'd be great to add an extension to your feed which we can try out on our end and make sure everything looks ok.

Brian

[Vajko, Peter]

> before officially adding it to the spec. So if it's ok with you, it'd
> be great to add an extension to your feed which we can try out on our
> end and make sure everything looks ok.

Ok, how about something like

message NyctFeedHeader {
optional TimeRange trips_are_replacements = 1;
}

extend FeedHeader {
optional NyctFeedHeader nyct_feed_header = 1001;
}

I'd omit trips_are_replacements.start as it does not make sense in our case. Trips_are_replacemnts.end would be now + 30 minutes.

[Brian Ferris]

Not a big deal, but I'd propose "trip_replacement_period" as the name.  Thoughts?

[Vajko, Peter]

> Not a big deal, but I'd propose "trip_replacement_period" as the name.

Sure, thanks, much better.

[Vajko, Peter]

I had a closer look at our system and realized that while the trip_replacement_period is now 30 minutes for all of our trips, it can be different for each route (it depends on where the real time data is coming from). So a really clean solution would be to provide this per route, but it may be an overkill.

At this moment this is not a problem, but as we add more lines, we may not be able to stick with 30 minutes for those. Thoughts?

message TripReplacementPeriod {
// the replacement period for this route
optional string route_id = 1;
// The start time is omitted, the end time is currently now + 30 minutes for
// all routes of the A division
optional transit_realtime.TimeRange replacement_period = 2;
}

// NYCT Subway extensions for the feed header
message NyctFeedHeader {
// For the NYCT Subway, the GTFS-realtime feed replaces any scheduled
// trip within the trip_replacement_period.
// This feed is a full dataset, it contains all trips starting
// in the trip_replacement_period. If a trip from the static GTFS is not
// found in the GTFS-realtime feed, it should be considered as cancelled.
// The replacement period can be different for each route, so here is
// a list of the routes where the trips in the feed replace all
// scheduled trips within the replacement period.
repeated TripReplacementPeriod trip_replacement_period = 1;
}

extend transit_realtime.FeedHeader {

optional NyctFeedHeader nyct_feed_header = 1001;
}

Peter Vajko
Siemens Industry, Inc.
498 Seventh Avenue
16th Floor
New York, NY 10018
Tel: +1 (212) 672-4053
Fax: +1 (212) 672-4001
mailto:peter...@siemens.com

[Brian Ferris]

Hey Peter,

This proposal sounds reasonable to me.  Sorry for not providing feedback sooner, I've been out travelling for the past week.

Brian

[Aaron Donovan]

Hi everyone,

At the links below, please find the current (beta) version of the MTA New York City Transit GTFS-RT spec (in pdf), and NYCT proto file (in txt). The revisions to these specifications were in response this discussion thread and one over at the MTA Developer Resources Google Group. 

This is the version we plan to develop, test and make available later this year. During our beta phase, we would make additional changes as necessary (hopefully no major changes will be needed, if any at all).

Thank you to all who helped us arrive at this document, for your insight and support. Everyone on our team feels this approach worked very well and helped us reach what we expect will be a very useful spec.  

http://httqa.mta.info/developers/internal/GTFS-Realtime-NYC-Subway-Updated-Spec.pdf

http://httqa.mta.info/developers/internal/nyct-subway-proto.txt 

-Aaron Donovan

Spokesman

Metropolitan Transportation Authority

State of New York

[Aaron Donovan]

Hey everyone,

Thanks again for your help earlier. Following up on that, the attached zip file has a 10-minute sample of real time train movement data from one afternoon last month on the 1, 2, 3, 4, 5 and 6 subway lines and the 42nd Street Shuttle, using the spec we'd all helped to develop.

Please have a look and play around with it and let us know if there are any unforeseen issues. Your feedback here on this board would be very helpful to us. But please don't blog about this data release or launch any public demo projects without notifying us first. 

Thanks,

Aaron 

[Vajko, Peter]

> zip file has a 10-minute sample of real time train movement data from
> one afternoon last month on the 1, 2, 3, 4, 5 and 6 subway lines and
> the 42nd Street Shuttle, using the spec we'd all helped to develop.

Actually, this is the result of a lab test with only two simulated trains running. There are way too many trains in the feed because it contains all scheduled trains (the ones with the is_assigned = false) too, but still realistic at least for the two running.

Expect less trains in total but of course a lot more of them running (is_assigned = true) in the real thing.

Both running trains are line 1 trains, 083500_1..S03R (01 1355 242/SFT) and 083800_1..N03R (01 1358 SFT/242), the southbound one even stalled somewhere, I think around Times Square, you should be able to see the alerts in gtfs.15 and gtfs.16.

[Aaron Donovan]

Hi All,

Attached please find the latest revised GTFS-RT specification and associated test data. The significant enhancements with this release include the inclusion of Vehicle Positioning data which can be used to determine a “stalled train” condition.  We also expanded track information to include both schedule and actual track.  This can be used to determine when a train is not operating according to it scheduled plan.

Aaron Donovan

Media Liaison

Metropolitan Transportation Authority

[Brian Ferris]

Sorry for not having to look at this earlier, but I did have one comment.  I noticed that you are going to start specifying VehiclePosition data just so that you can use the timestamp field to indicate when the vehicle last moved.  I think this deviates from the spec a bit, as the timestamp is really a measure of when you last received data from the vehicle, not a measure of when the vehicle last moved.  Or is it really the case that "we haven't heard from a train since time X" implies "train hasn't moved since time X"?

That said, I'm wondering if the information in the feed already captures stalled vehicle information?  Aka if the arrival times for the train keep creeping up, then it probably indicates a stalled vehicle.

Brian

--
You received this message because you are subscribed to the Google Groups "GTFS-realtime" group.

To view this discussion on the web visit https://groups.google.com/d/msg/gtfs-realtime/-/8rlRMxiL-ysJ.

[Frumin, Michael]

Brian, in fixed-block signaling systems (i.e. non-CBTC) there is no communication from the vehicle per se.  All there is is communication from the signaling system, and that only reports changes when the train crosses from one signal block to another.  There is nothing else it can detect.

 

Thoughts?

 

Thanks,

Mike

[Brian Ferris]

Ok, given that clarification, then using the timestamp seems reasonable.  And given that using the timestamp seems reasonable, it seems like a shame that you have to include a separate VehiclePosition record (which developers then have to go back and match to a TripUpdate) just for the timestamp field.  My proposal is that we just add a timestamp field to TripUpdate directly.

As a supporting data-point, OneBusAway already has a timestamp extension for TripUpdate:

https://github.com/OneBusAway/onebusaway-gtfs-realtime-api/blob/master/src/main/resources/com/google/transit/realtime/gtfs-realtime-OneBusAway.proto

which is both produced by some of the GTFS-realtime converters (SIRI, NextBus, etc) and consumed by the OBA transit data service.  Given that you are looking for a similar field, my proposal is that we just add it to the spec directly.

Thoughts?

[Vajko, Peter]

Brian,

> TripUpdate) just for the timestamp field. My proposal is that we just
> add a timestamp field to TripUpdate directly.

Yes, that was our first thought but then we realized we have already a timestamp in the VehiclePosition defined as "moment at which the vehicle's position was measured" which is exactly what we have.

The OneBusAway example defines the TripUpdate timestamp as the "moment at which the trip update was computed" which is in our case not true, I feel our timestamp is more related to the vehicle (train) than to the trip update and would have to be defined differently.

Nevertheless, if the VehiclePositions does not make sense from the feed consumer perspective, I have no problem omitting them and adding a timestamp to the TripUpdate instead.

> developers then have to go back and match to a TripUpdate

BTW, the current implementation just throws in a VehiclePosition entity right after the TripUpdate in case the timestamp is already known, so you have TripUpdates and VehiclePositions alternating in the feed.

[Brian Ferris]

In practice, the OneBusAway tools are using the field to capture the time of the last update from the transit vehicle.  That said, I can see how there might be a semantic difference between "the time of the last vehicle update" and the "the time of the last trip update".  That is, if you haven't received an update from a train, then you will probably update arrival times along the train's trip to reflect the stalled status of the vehicle.  Thus you could argue that any TripUpdate timestamp should reflect the time at which you updated those values, even if there wasn't a vehicle update to back it up.

Is that a distinction worth making?

[Vajko, Peter]

> status of the vehicle. Thus you could argue that any TripUpdate
> timestamp should reflect the time at which you updated those values,
> even if there wasn't a vehicle update to back it up.
>
> Is that a distinction worth making?

Probably not. So we want to go with the timestamp for the TripUpdate then?

[Brian Ferris]

I'm +1.  I'd even go so far as to add this to the official spec now, since this has come up in a couple of places.  Something like:

// Moment at which the vehicle's real-time progress was measured.  In POSIX time (i.e., the number of seconds since January 1st 1970 00:00:00 UTC). 

optional uint64 timestamp = 6;

Anyone opposed?

[tomcola58]

Brian

I am also in agreement with updating the specification to include "// Moment at which the vehicle's real-time progress was measured".  If this could be finalized before December even better.  We are testing out better feed and are planning on finalizing it before the end of the year and would like to include this change at that time.

[Brian Ferris]

Ok, let's start the clock on adding this to the official spec.  If there are no major complaints in the next week, we will add the trip-update timestamp proposal to the spec next Tuesday, October 9th.

Thanks,

Brian

To view this discussion on the web visit https://groups.google.com/d/msg/gtfs-realtime/-/hrcjQ5KrkQ8J.

[Brian Ferris]

Ok, hearing no complaints, this has been added to the spec:

https://developers.google.com/transit/gtfs-realtime/reference#TripUpdate

Hopefully the NYC MTA guys can consider using this directly.  I will also see about updating some of the OneBusAway GTFS-realtime tools to support this as well.

Thanks,

Brian