[GTFS Informations] - GTFS VS SIRI VS OTHER EXISTING

[Benjamin Batista]

Hello,

I'm new in the transit GTFS "community" and I would like to know few things.

I don't understand why some companies are using GTFS and other TransModel, etc ...

What is the huge differences between all this "norme"?

How GTFS is more stronger and how GTFS is "weak" ? 

Thank you 

[Tony Laidig]

Take a look at Sean Barbeau's presentation here:
http://www.slideshare.net/sjbarbeau/apta-transitech-2013-open-transit-data-a-developers-perspective
It's a good introduction to the questions you're asking.

[Benjamin Batista]

Sorry I didn't see your answer. I already contact him by Twitter and he already gave me this presentation ahah.

There is something that I don't understand :

Slide n°11 : Why using TCIP, GTFS-RT and SIRI together? because it's a FireHose?

Thank you

[Joris Wu]

Hi Benjamin,

As a mass-user of transit data ( covering as much of the world as possible ) I would say the greatest strength of GTFS is that it is the most widespread in terms of public availability. Probably by far if counting by the number of agencies. Weaknesses of any standard tend to be subjective. My personal percieved weaknesses are :

- Not descriptive enough  : quite a handful of actual behaviours and patterns cannot be described well

- Not well suited for integration : stopos are referred by coordinates. The actual combination of multiple datasets in to a single integrated one is a fuzzy operation.

- Poor support for internationalisation : timezones, international names are not truly anticipated.

I have never perceived GTFS as consumer-friendly. You go through quite some hoops to get the actual network out with all its quirks. And not all producers interpret the standard identically.
But for both producers and consumers, it is most important what others work with. TransModal and TransExchange may appear superior, yet did not made it to the masses.

Joris

[Michael Frumin]

Here is my take from experience:

TCIP is of little value for communicating from inside the agency to outside the agency.  But it has models for tons of things internal to the agency that SIRI and GTFS will never have.  So, use it internally if you don't already have another standard.

For communicating to outside the agency, if you are going for Fire hose, use GTFS and GTFS-RT.  Otherwise, use SIRI.

It's really that simple, right (Sean, Toby, Joris, etc)?

--
You received this message because you are subscribed to the Google Groups "Transit Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to transit-develop...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sjba...@gmail.com]

TCIP is of little value for communicating from inside the agency to outside the agency.  But it has models for tons of things internal to the agency that SIRI and GTFS will never have.  So, use it internally if you don't already have another standard.
For communicating to outside the agency, if you are going for Fire hose, use GTFS and GTFS-RT.  Otherwise, use SIRI.
It's really that simple, right (Sean, Toby, Joris, etc)?

That's pretty much my take as well.  I usually refer to TCIP as part of an open architecture - in other words, internal at the agency, if you want to be able to swap out one internal component with another, TCIP is probably the standard you should be looking at.  Again, this is relevant for the agency making decisions about what technology they should be implementing internally - doesn't necessarily have any implications for what is shared publicly.

If an agency is interested in sharing data externally with 3rd parties outside the agency (i.e., open data) in bulk (i.e., firehose), GTFS for static data and GTFS-rt or SIRI are your best options.  Practically speaking, GTFS-rt is probably the first choice of most transit agencies today, because you can directly share it with apps like Google - to my knowledge Google doesn't support consuming SIRI directly as of today.  GTFS-rt also has explicit connections with GTFS, the de facto open data format for schedule data, so there are fewer chances for ID fields not matching up, etc.  You could still match GTFS to SIRI realtime but because they are two different standards, there are more chances for something to get lost in translation.  Main downside of GTFS-realtime at this point, IMHO, is that the GTFS-realtime spec is still relatively loose and there are several gray areas that need to be tightened up.  If you're interested in the details of this, see https://github.com/google/transit/pull/19.

And to clarify - you can implement an open architecture internally using something like TCIP without sharing any of that information with the public (i.e., open data).  And, you can have open data and share information publicly in formats like GTFS and GTFS-rt without having an internal open architecture.  Ideally you have both an open architecture and open data, which gives the agency the largest flexibility and smallest chance of facing vendor lock-in.  Worst case is having neither open architecture nor open data, which means you're likely purchasing a massive monolithic AVL/scheduling software system and will be locked into that same vendor until the end of life of the system.  And, at end of life, you'd likely have to replace all components of that system with a completely new system.

For slide 11/12 - it's just illustrating where the various standards are useful in the larger architecture of the entire end-to-end system - typically you'd chose one standard for internal open architecture (e.g., TCIP), and then one standard for static firehose open data (GTFS), and one standard for real-time firehose open data (GTFS-realtime OR SIRI).  For real-time firehose open data, as mentioned above, GTFS-realtime is becoming a common first choice.

Sean

To unsubscribe from this group and stop receiving emails from it, send an email to transit-developers+unsub...@googlegroups.com.

[Nisar Ahmed]

At 511 SF Bay, we have adopted GTFS and GTFS-Realtime as firehose and SIRI/NeTEx as API standards.

 

--Nisar

To unsubscribe from this group and stop receiving emails from it, send an email to transit-develop...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Transit Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to transit-develop...@googlegroups.com.

[Benjamin Batista]

Thank you all for your answers that's very helpful !

So we have to make our own company data transit with TCIP (that is only inside the company), and then use GTFS for open data (mobile application for example) and convert TCIP to GTFS-RT/SIRI ?

And do you know what are the preferred platforms for the GTFS usage? I think I'll do a mobile application or a web-service (or both) but for the moment I'm trying to understand everything about it before.

"  Ideally you have both an open architecture and open data, which gives the agency the largest flexibility and smallest chance of facing vendor lock-in.  Worst case is having neither open architecture nor open data, which means you're likely purchasing a massive monolithic AVL/scheduling software system and will be locked into that same vendor until the end of life of the system.  And, at end of life, you'd likely have to replace all components of that system with a completely new system."

I don't understand this part about "vendor lock-in".

[Sean Barbeau]

And do you know what are the preferred platforms for the GTFS usage? I think I'll do a mobile application or a web-service (or both) but for the moment I'm trying to understand everything about it before.

If you're looking to build a mobile app based on transit data, first step would be to see if the transit agency offers a "faucet" type API, that let's you query things like "what are the upcoming arrival times at stop 1234?".  MTA in NY has an API of this type based on the SIRI standard - http://bustime.mta.info/wiki/Developers/SIRIIntro.

Other agencies will offer proprietary (i.e., specific to one AVL vendor) faucet APIs, such as NextBus or Clever Devices.  The OneBusAway open-source project offers a set of these APIs as well for the cities where it is deployed - http://onebusaway.org/onebusaway-deployments/, http://developer.onebusaway.org/modules/onebusaway-application-modules/current/api/where/index.html.  Many of these cities also offer the same SIRI-based API that MTA in NY offers (MTA's system is based on OneBusAway software).

If the agency doesn't have a faucet API, then you will have to create your own by setting up your own server that transforms GTFS/GTFS-realtime data into an API - there is a list of such projects here - https://github.com/luqmaan/awesome-transit#apis.

Vendor lock-in is when you are forced to use the same vendor that originally deployed a system to replace/upgrade components, and you do not have a choice to use any other vendor.  This is because the component is proprietary to that single vendor.  Vendor lock-in is typically very expensive because there is no competition for individual components of the system - to switch vendors, you'd need to replace the entire system.

Standards try to avoid vendor lock-in by designing interfaces that multiple vendors can implement.  When interaction between components are standardized you should be able to swap out a component from one vendor with another vendor's component that conforms to the same standard.  This allows open competition in the marketplace and typically results in less costly components.

Sean

[Benjamin Batista]

Ok thank you Sean,

So, we have GTFS/Neptune/NeTEx,TransXChange,Transmodel as static transit data.
It's better to use GTFS because it's the most widespread (so we have more data)

We have three servers, one for the agencie (internal at the agencie so we have to use TCIP to collect and manage data from the BUS for example), one for use GTFS-RT ( or SIRI) and the other one for the API/web service?

[Benjamin Batista]

Why did you choose SIRI/NeTEx as API standards?

Thank you

To unsubscribe from this group and stop receiving emails from it, send an email to transit-developers+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Transit Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to transit-developers+unsub...@googlegroups.com.

[Andrew Byrd]

On 26 Apr 2016, at 16:26, Benjamin Batista <benjich...@gmail.com> wrote:

Ok thank you Sean,

So, we have GTFS/Neptune/NeTEx,TransXChange,Transmodel as static transit data.
It's better to use GTFS because it's the most widespread (so we have more data)

Put simply:

Transmodel is a vocabulary that standardizes the meaning of things like “StopArea” and “JourneyPattern” and the relationships between them. 

TransXChange and NeTEx are officially-sanctioned data exchange formats that use the Transmodel vocabulary to encode transit data as XML in different ways.

GTFS is an unofficial, collaborative industry standard for exchanging transit schedule data in simple CSV tables, in enough detail to provide good trip planning and timetables to public transport riders.

In more detail:

I have only ever seen TransXChange data in the UK. Apparently Australia also provides it, and in my experience it’s not straightforward to convert to other formats (as mentioned in other messages).

NeTEx is an emerging format that is not used much yet. Many people (particularly those who have invested a lot of money and time in developing it) expect NeTEx to become the standard way to publish transit data, and for this to be required by law. Many other people do not expect it to catch on outside large rail operators due to its immense complexity, which I have been told is the result of design by committee where several countries made it possible to express all their existing formats within NeTEx.

At least in Europe, when you refer to GTFS as a “standard” some people are quick to correct you, saying that only CEN “normes” are genuinely standards, making NeTEx the real state-sanctioned and mandated standard, and characterizing GTFS as some sort of attempt to subvert the legitimate authority of the state. In this sense NeTEx is a comparable to TCIP in the US.

While these other standards were being mulled over for years in advance of ever being used, gradually expanding to include anything you might ever want to represent, GTFS was created very rapidly to handle immediate concrete needs. GTFS evolves through collaborative discussion, but there is a rule against extending GTFS to address things that have not been encountered in the field and implemented on both the data producer and consumer end. 

GTFS concentrates on passenger information (trip planning, displaying arrival times) while the other formats include a lot of operational details that are mostly of interest to the transit operators. In a rather unorthodox move for its time, GTFS also intentionally avoided XML, which I believe has greatly contributed to its popularity.

So GTFS is an eminently pragmatic, unofficial industry standard data exchange format for passenger information. The other formats are officially sanctioned norms used by a few large operators or countries to exchange much more detailed operational information, which makes these formats overly complex for most end users.

-Andrew Byrd

[Barbeau, Sean]

That’s a good description.  GTFS is a de facto standard, while TransXChange, NeTEx, TCIP, and SIRI are all de jure standards.  Given enough time, if GTFS-realtime becomes dominant in the market, it could also be considered a de facto standard.

 

Sean

--
You received this message because you are subscribed to a topic in the Google Groups "Transit Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/transit-developers/ZBk6_lLP0Lk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to transit-develop...@googlegroups.com.

[Benjamin Batista]

Thank you Sean and Andrew 

To unsubscribe from this group and all its topics, send an email to transit-developers+unsub...@googlegroups.com.

[Michael Smith]

"The great thing about standards is that there are so many to choose from!" - a great humorous quote attributed (most likely wrongly) to Mark Twain.

Andrew's summary is excellent and very helpful. I can only reiterate that there are standards and there are "standards". Standards are usually created by committee, are cumbersome, and obsolete by the time they are official. And since they are obsolete the standard then needs to be changed and is therefore not truly a standard anymore. SIRI is a prime example. Created by committee, XML, overly complicated, designed before smartphones were ubiquitous. So now there is a new SIRI standard that is JSON and RESTful, and very incompatible with the original standard. I've even seen a transit agency require both the old and the new versions of SIRI.

And then there are defacto "standards" such as GTFS and GTFS-realtime. Not ideal but they use good technologies (CSV and protobuffers) and they have really great adoption rates. Sure we can quibble about those formats because they have limits. But they are incredibly useful.

I expect that some governments/bureaucracies will continue to develop and require problematic standards. But I recommend that systems you work on first use the better defacto "standards" and only deal with other standards when specifically required to do so.

Mike

--

You received this message because you are subscribed to the Google Groups "Transit Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to transit-develop...@googlegroups.com.

[Benjamin Batista]

Sorry for disturbing you again ahah..

How the server update the GTFS-RT?

They access to the protocol buffer and then change the vehicle information about the vehicles'id?

Thank you

[Benjamin Batista]

And also how to visualize a .pb files?

[Sean Barbeau]

The .pb protocol buffer file is typically served via an HTTP server like any other file.  To update it, it is simply replaced with the updated version.

Many tools that publish protobuf files also offer the option to show the information in plain text format for debugging purposes.

For example, OneBusAway offers a GTFS-realtime Export API:
http://developer.onebusaway.org/modules/onebusaway-application-modules/current-SNAPSHOT/api/gtfs-realtime.html

By default, these methods return GTFS-realtime data encoded as a binary protocol buffer, per the GTFS-realtime spec. To make it easier to debug your system, we also provide a simple way to see a textual representation of each GTFS-realtime feed as well. Simple switch the “.pb” extension in the URL with “.pbtext”.

The GTFS-realtime generation tool we built:
https://github.com/CUTR-at-USF/HART-GTFS-realtimeGenerator

...also offers a plaintext option:

• Protobuf - http://realtime.prod.obahart.org:8088/trip-updates
• Plain text - http://realtime.prod.obahart.org:8088/trip-updates?debug

Kurt Raschke also built a tool where you can dump any protobuf file to plain text:

https://github.com/kurtraschke/gtfs-rt-dump

Sean

[Sean Barbeau]

And if you're talking about visualizing vehicle positions on a map from a protobuf file, check out this project:
https://github.com/OneBusAway/onebusaway-gtfs-realtime-visualizer/wiki

[Benjamin Batista]

Thank you that's exactly what I wanted: gtfs-rt-dump is very useful thank you

And I already used gtfs-realtime-visualizer and I wanted to know how the .pb files looks like to understand more about it.



-

[Benjamin Batista]

Hello it's me again.

Sorry for disturbing you but I just want to know, what's the link between GTFS-RT and GTFS? Do I need to use it together ? If yes, how can we use it together? I don't understand I think it's two thing totaly diferent but can we interact GTFS with GTFS-RT?

Thank you  

[Barbeau, Sean]

The IDs in the GTFS data should match the IDs in the GTFS-realtime data.  For example, there is a trip_id = X in the GTFS data that says a bus is scheduled to arrive at 9:00am.  Then, there would be an update in the GTFS-realtime data saying that trip_id = X is running 5 minutes late.  Consumers of the data can then show this information, with an ETA of 9:05am.

 

Sean

--
You received this message because you are subscribed to a topic in the Google Groups "Transit Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/transit-developers/ZBk6_lLP0Lk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to transit-develop...@googlegroups.com.

[Benjamin Batista]

Oh thank you Sean.

You really help me with all your answers, for the moment I just made an API that collect information from the database for GTFS static and then a web service who create the GTFS file.
Thank you again.

One question again, in siri you have different version and each version you have to collect some data ( but it depends on the version), do you have the same with GTFS-RT? Because at the begining of GTFS-RT file there is a field about version 1.0

Thank you

To unsubscribe from this group and all its topics, send an email to transit-developers+unsub...@googlegroups.com.

[Barbeau, Sean]

GTFS-realtime version field currently isn’t used – it’s always v1.0.

To unsubscribe from this group and all its topics, send an email to transit-develop...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "Transit Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/transit-developers/ZBk6_lLP0Lk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to transit-develop...@googlegroups.com.

[Benjamin Batista]

Ok :)

I think I understand GTFS-RT, there is a kind of API/library to create a GTFS-RT protobuffer.

For example VehicleDescriptor.Builder vehicleDescriptor = VehicleDescriptor.newBuilder(); is to create the Vehicle location feed.
Am I right?

Thank you

Le mercredi 29 juin 2016 15:59:04 UTC+1, Sean Barbeau a écrit :

GTFS-realtime version field currently isn’t used – it’s always v1.0.

 

From: transit-d...@googlegroups.com [mailto:transit-d...@googlegroups.com] On Behalf Of Benjamin Batista
Sent: Wednesday, June 29, 2016 9:43 AM
To: Transit Developers
Subject: Re: [transit-developers] Re: [GTFS Informations] - GTFS VS SIRI VS OTHER EXISTING

 

Oh thank you Sean.

You really help me with all your answers, for the moment I just made an API that collect information from the database for GTFS static and then a web service who create the GTFS file.
Thank you again.

One question again, in siri you have different version and each version you have to collect some data ( but it depends on the version), do you have the same with GTFS-RT? Because at the begining of GTFS-RT file there is a field about version 1.0

Thank you

Le mercredi 29 juin 2016 14:09:52 UTC+1, Sean Barbeau a écrit :

The IDs in the GTFS data should match the IDs in the GTFS-realtime data.  For example, there is a trip_id = X in the GTFS data that says a bus is scheduled to arrive at 9:00am.  Then, there would be an update in the GTFS-realtime data saying that trip_id = X is running 5 minutes late.  Consumers of the data can then show this information, with an ETA of 9:05am.

 

Sean

 

From: transit-d...@googlegroups.com [mailto:transit-d...@googlegroups.com] On Behalf Of Benjamin Batista
Sent: Wednesday, June 29, 2016 8:39 AM
To: Transit Developers
Subject: [transit-developers] Re: [GTFS Informations] - GTFS VS SIRI VS OTHER EXISTING

 

Hello it's me again.

Sorry for disturbing you but I just want to know, what's the link between GTFS-RT and GTFS? Do I need to use it together ? If yes, how can we use it together? I don't understand I think it's two thing totaly diferent but can we interact GTFS with GTFS-RT?

Thank you  

--
You received this message because you are subscribed to a topic in the Google Groups "Transit Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/transit-developers/ZBk6_lLP0Lk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to transit-developers+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Transit Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/transit-developers/ZBk6_lLP0Lk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to transit-developers+unsub...@googlegroups.com.