proposal: add a "feed info" file to express metadata about the feed
Joe Hughes
Add a new file that contains information about the feed itself, rather
than the services that the feed describes.
Motivation:
GTFS currently has an agency.txt file to provide information about the
agencies/operators that operate the services described by the feed.
However, the publisher of the feed is sometimes a different entity
than any of the operators (in the case of regional aggregators). In
addition, there are some fields in agency.txt that are really
feed-wide rather than agency-wide settings now that we allow multiple
agencies per feed. Finally, It would be useful to have an identifier
that a feed publisher can use to determine which version of their feed
is currently being used by a client.
Proposal:
A new optional feed_info.txt file will be added to the specification,
with the following fields.
feed_publisher_name (required):
The feed_publisher_name field contains the full name of the
organization that publishes the feed. (This may be the same as one of
the agency_name values in agency.txt.) GTFS-consuming applications
can display this name when giving attribution for a particular feed's
data.
feed_publisher_url (required):
The feed_publisher_url field contains the URL of the feed publishing
organization's website. (This may be the same as one of the
agency_url values in agency.txt.) The value must be a fully qualified
URL that includes http:// or https://, and any special characters in
the URL must be correctly escaped. See
http://www.w3.org/Addressing/URL/4_URI_Recommentations.html for a
description of how to create fully qualified URL values.
feed_timezone (required):
The feed_timezone field specifies the timezone in which the times in
the feed will be given. Any stop/stations which don't have a
stop_timezone specified are also assumed to be located in this
timezone. For feeds containing feed_info.txt, this value is used
instead of the agency_timezone values in agency.txt. Please refer to
http://en.wikipedia.org/wiki/List_of_tz_zones for a list of valid
values.
feed_lang (required):
The feed_lang field contains a two-letter ISO 639-1 code for the
default language used for the text in this feed. This setting helps
GTFS consumers choose capitalization rules and other language-specific
settings. Please refer to
http://www.loc.gov/standards/iso639-2/php/code_list.php for a list of
valid values. This value overrides any agency_lang values in
agency.txt
feed_version (optional):
The feed publisher can specify a string here that indicates which
version of their feed this is. GTFS-consuming applications can
display this value to help feed publishers determine whether the
latest version of their feed has been incorporated.
Comments?
Joe Hughes
Google
Fabien Viger
I like all fields that Joe is proposing to add, and I'd propose adding
an other field:
feed_valid_until (optional):
The feed can be marked by providers as valid until a certain date,
while still providing extrapolated schedule data past that date. It
helps clients monitor the validity of a GTFS feed over time, while
allowing graceful degradation when a feed expires: the client can
choose to use the schedule data past the feed_expiration_date, but it
will be easy to show a warning to users about the questionable
accuracy of this data. The format should be the same as dates in
calendar.txt, i.e.YYYYMMDD, the value should be the last day of
validity of the feed.
I'd also like to propose to widen the scope of the feed_info.txt file,
maybe renaming it as appropriate, to allow adding fields that let
clients specify more feed-wide information. I heard some requests
about modelling transfers more accurately.
transfer_safety_buffer (optional):
The time, in seconds, to reserve for each transfer in addition to the
walking time. This only applies to 'default' transfers, i.e transfers
that aren't explicitly listed in transfers.txt, but can be inferred
from the stop positions.
preferred_transfer_safety_buffer (optional):
Ditto, but applying to transfers listed in transfers.txt with type 0
timed_transfer_safety_buffer (optional):
ditto, but applying to transfers listed in transfers.txt with type 1
Note that transfers listed in transfers.txt with type 2 should not
need a safety buffer, since the exact transfer time (including the
safety buffer) is already provided, according to the current GTFS
spec.
Joe, let me know if you think that the part about transfers should be
part of another GTFS proposal. In that case, I'll open one, with my
apologies.
On Jan 30 2008, 8:23 pm, "Joe Hughes" <joe.hughes.c...@gmail.com>
wrote:
> Summary:
> Add a new file that contains information about thefeeditself, rather
>
> Motivation:
> GTFS currently has an agency.txt file to provide information about the
> agencies/operators that operate the services described by thefeed.
> agencies perfeed. Finally, It would be useful to have an identifier
> that afeedpublisher can use to determine which version of theirfeed
>
> Proposal:
> A new optional feed_info.txt file will be added to the specification,
> with the following fields.
>
> feed_publisher_name (required):
> The feed_publisher_name field contains the full name of the
>
> feed_publisher_url (required):
> The feed_publisher_url field contains the URL of thefeedpublishing
> organization's website. (This may be the same as one of the
> agency_url values in agency.txt.) The value must be a fully qualified
> URL that includes http:// or https://, and any special characters in
>
> feed_timezone (required):
> The feed_timezone field specifies the timezone in which the times in
> timezone. For feeds containing feed_info.txt, this value is used
>
> feed_lang (required):
> The feed_lang field contains a two-letter ISO 639-1 code for the
> agency.txt
>
> feed_version (optional):
> version of theirfeedthis is. GTFS-consuming applications can
> display this value to helpfeedpublishers determine whether the
> latest version of theirfeedhas been incorporated.
>
> Comments?
>
> Joe Hughes
--
You received this message because you are subscribed to the Google Groups "General Transit Feed Spec Changes" group.
To post to this group, send email to gtfs-c...@googlegroups.com.
To unsubscribe from this group, send email to gtfs-changes...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gtfs-changes?hl=en.
Arno Eigenwillig
> I support the idea of having a simple file defining feed-wide values.
you proposed.
> feed_valid_until (optional):
> The feed can be marked by providers as valid until a certain date,
> while still providing extrapolated schedule data past that date. It
> helps clients monitor the validity of a GTFS feed over time, while
> allowing graceful degradation when a feed expires: the client can
> choose to use the schedule data past the feed_expiration_date, but it
> will be easy to show a warning to users about the questionable
> accuracy of this data. The format should be the same as dates in
> calendar.txt, i.e.YYYYMMDD, the value should be the last day of
> validity of the feed.
to agreement between feed producer and consumer; the spec should not
restrict this. Besides display with warning, another use could be:
compute a transit route with and without expired data; if the latter
result is x% worse than the former, display that the result to the
query is expired.
> I'd also like to propose to widen the scope of the feed_info.txt file,
> maybe renaming it as appropriate, to allow adding fields that let
> clients specify more feed-wide information. I heard some requests
> about modelling transfers more accurately.
are a wide topic. I'd prefer to keep that separate. Let us use this
proposal to establish feed_info.txt, and let's have a separate
discussion about transfers, because they involve much more than
feed-wide parameters. (Also, it may make sense to group them in
another file, separate from thie general "data about the data in feed"
from feed_info.txt.)
> On Jan 30 2008, 8:23 pm, "Joe Hughes" <joe.hughes.c...@gmail.com>
> wrote:
>> feed_lang (required):
>> The feed_lang field contains a two-letter ISO 639-1 code for the
>> default language used for the text in thisfeed. This setting helps
Following Tom's proposal for agency_lang from May 2009, we should go
for full BCP 47 language codes right away.
Arno
PS: Can anyone recommend a Python library for validating/parsing BCP
47 language codes? If yes, then please let me know off-thread; it
would help getting this into transitfeed and the feedvalidator.
Max Campos
I'd prefer to have a feed_version that's restricted to being an integer, and then require that the integer always increase with newer versions. That way you can programmatically determine if datafile A is newer than datafile B, in a feed-independent way.
Ex: Maybe some agency would use YYYYMMDDHHMMSS
20100515150000
20100520180000
… or some agency could just use 1, 2, 3 as their version numbers.
- Max
> feed_version (optional):
> The feed publisher can specify a string here that indicates which
> version of their feed this is. GTFS-consuming applications can
> display this value to help feed publishers determine whether the
> latest version of their feed has been incorporated.
>
> Comments?
>
> Joe Hughes
>
> You received this message because you are subscribed to the Google Groups "Google Transit Feed Spec Changes" group.
Arno Eigenwillig
In addition to my response from May 7, I'd like to also propose a
feed_valid_from field, matching the feed_valid_until field proposed by
Fabien. Feed validity is naturally limited on boths ends, so it makes
sense to record that as well. This has an important practical use in
case an agency cannot provide service during some unforseen blackout
window: By setting feed_valid_from to today's date and the start_dates
in calendar.txt to a later date, an agency can express clearly that
there is no service before the later date. Without the
feed_valid_from date, a GTFS consumer might erroneously assume the
feed becomes valid only at that later date, and remain stuck at an
older version of the feed that still had service for the blackout
window.
Also, I believe we should make these columns required (but allow empty
values as an escape of last resort) to emphasize this is not an
optional add-on but something that is "morally required" from
high-quality feed producers.
The revised proposal reads as follows.
feed_valid_from (required), feed_valid_until (required):
The feed provides complete and reliable schedule information for
service in the period from the beginning of the feed_valid_from day to
the end of the feed_valid_until day. Both days are given as dates in
YYYYDDMM format as for calendar.txt, or left empty if unavailable. The
feed_valid_until date must not precede the feed_valid_from date if
both are given. Feed providers are encouraged to give schedule data
outside this period to advise of likely future service, but feed
consumers should treat it mindful of its non-authoritative status.
We at Google Transit anticipate a rather coarse-grained usage of these
fields: warn partners if they provide a new feed that's not valid yet
(because it would break queries for "depart: now") or if the validity
of the feed ends too soon. For that, we don't need to be
super-specific on fine points like trips that run on the
feed_valid_until date but extend past 24:00:00. Does anyone on the
group have an application that would require to define exactly what
the feed validity dates mean for these corner cases? If so, which
interpretation would you suggest? If not, let's keep it simple (as
above).
Arno
> I like all fields that Joe is proposing to add, and I'd propose adding
> an other field:
>
> feed_valid_until (optional):
> The feed can be marked by providers as valid until a certain date,
> while still providing extrapolated schedule data past that date. It
> helps clients monitor the validity of a GTFS feed over time, while
> allowing graceful degradation when a feed expires: the client can
> choose to use the schedule data past the feed_expiration_date, but it
> will be easy to show a warning to users about the questionable
> accuracy of this data. The format should be the same as dates in
> calendar.txt, i.e.YYYYMMDD, the value should be the last day of
> validity of the feed.
Google Switzerland GmbH
Neil
to keep track of changes and updates to the schedule data. It could
also be helpful to show the version number in an unobtrusive fashion
on the Google trip planner itself, if nothing else than for testing
purposes.
On May 16, 4:11 am, Max Campos <li...@bpsw.biz> wrote:
> Well since this has been reopened, I'd like to suggest one tweak -
>
> I'd prefer to have a feed_version that's restricted to being an integer, and then require that the integer always increase with newer versions. That way you can programmatically determine if datafile A is newer than datafile B, in a feed-independent way.
>
> Ex: Maybe some agency would use YYYYMMDDHHMMSS
> 20100515150000
> 20100520180000
>
> … or some agency could just use 1, 2, 3 as their version numbers.
>
> - Max
>
>
>
>
>
>
>
> > feed_version (optional):
> > The feed publisher can specify a string here that indicates which
> > version of their feed this is. GTFS-consuming applications can
> > display this value to help feed publishers determine whether the
> > latest version of their feed has been incorporated.
>
> > Comments?
>
> > Joe Hughes
>
> > --~--~---------~--~----~------------~-------~--~----~
> > You received this message because you are subscribed to the Google Groups "Google Transit Feed Spec Changes" group.
> > To post to this group, send email to gtfs-c...@googlegroups.com
> > To unsubscribe from this group, send email to gtfs-changes...@googlegroups.com
Brian Ferris
David Turner
GTFS data is largely uncopyrightable, being facts formatted
uncreatively. It might make agencies feel good to say that the data is
only available for noncommercial use, but it's usually not enforcable,
but will serve to discourage creative people who don't know this.
> > For more options, visit this group
> athttp://groups.google.com/group/gtfs-changes?hl=en.
>
>
> --
> You received this message because you are subscribed to the
> Google Groups "General Transit Feed Spec Changes" group.
> To post to this group, send email to
> gtfs-c...@googlegroups.com.
> To unsubscribe from this group, send email to gtfs-changes
> http://groups.google.com/group/gtfs-changes?hl=en.
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "General Transit Feed Spec Changes" group.
> To post to this group, send email to gtfs-c...@googlegroups.com.
> To unsubscribe from this group, send email to gtfs-changes
Brian Ferris
David Turner
things, on their terms and your jurisdiction.
But click-throughs are not enforcable "at a distance" -- that is, if A
downloads and redistributes the data, and B gets it from A, then B is
(generally) not bound by the terms of the click-through.
This proposal would make that proposition more difficult.
John L
Joe Hughes
a significant side debate, I propose that leave it out of the initial
feed_info extension, in the interests of arriving at something we can
all agree on sooner.
Incidentally, there was some previous discussion about encoding
licensing information in the feeds in this thread:
https://groups.google.com/group/gtfs-changes/browse_thread/thread/9746dedf888f04ec/b9f66e3146120976?lnk=gst&q=license#b9f66e3146120976
Cheers,
Joe
Arno Eigenwillig
> I'd prefer to have a feed_version that's restricted to being an integer, and then require that the integer always increase with newer versions. That way you can programmatically determine if datafile A is newer than datafile B, in a feed-independent way.
>
> Ex: Maybe some agency would use YYYYMMDDHHMMSS
> 20100515150000
> 20100520180000
>
> … or some agency could just use 1, 2, 3 as their version numbers.
text field, for example, North County Transit District from
http://code.google.com/p/googletransitdatafeed/wiki/PublicFeeds and
San Diego MTS, so there is a cost attached to fordbidding this now.
On the other hand, now is the last time to do it, if ever.
Can anyone from the feed producer side comment on the issue?
Is there an up-and-running example of a feed producer and a feed
consumer using this ordering of versions in practice?
If not, I suggest we keep a free-format feed_version for now and work
towards getting this proposal into the spec; it has been open for too
long already. We can always add a separate field like
feed_version_number later on.
Marcy
inert hop to agency URL allows me to VERY quickly view which version
is in production or preview
i love metadata but I would need to view it to confirm it and all of
those steps take time
Sadly the QA team does not like the inert hop approach and in
production it is (often) removed but if allowed & not distracting I
found I can mouseover the agency url & VOILA I see http://myagency.com#051410
and know this version was posted a few weeks ago
hoping simplicity might be a virtue
Devin Braun
no heartburn changing it.
We have begun to use GTFS files as our standard for offering
information to our outside vendors who do work with us. Having a
version number (in whatever format) is important because we will have
to be on the same page when offering scheduling information to our
passengers. Although, none of these projecs are live yet so we can't
give an example.
Devin
San Diego MTS
Max Campos
That version number is then used in the following places:
1) In the filename on disk. ("google_transit.zip" files aren't particularly helpful in identification -- TriMet_20100529_20100529.zip)
2) In the database name once the dataset has been imported (ex: TQ-TriMet-20100529-20100529)
3) Once the dataset is generated into a schedule database for my application (TransitQ), the version number is embedded inside of the schedule database so that the application can tell when the schedule data has been updated by the user (this is necessary to update the user's favorite stop lists, etc)
4) The version number is also used in the filename of the schedule database that the end-user downloads (http://www.transitq.com/download). They use this to determine if there is a new schedule available or not.
5) The version number is used in the filename that's stored on the Palm (which is independent of the *.pdb filename and *.zip filename)
6) The version number is used to differentiate datasets on my GTFS browsing site
- Look how ugly it is to determine version on this page: http://gtfs.transitq.com/
- It's then embedded into the URL once you select a dataset (http://gtfs.transitq.com/TriMet_20100430_20100430/)
7) It looks like GTFS data exchange goes through a similar process. They generate filenames like lane-transit-district_20100528_0227.zip.
It seems senseless to have each GTFS consumer reinvent the wheel with their own version number scheme. We should have an ascending version number that (even better) fits some standard form (ex: YYYYMMDDHHMMSS) so that it could easily be placed into filenames, database names, web URLs, used for keeping archives, etc.
When it comes to version numbers, I think we should instead be asking ourselves why it makes sense to have a non-ascending version number. I can't think of any place I've seen a non-ascending version number that wasn't really annoying. Does the Windows version numbering scheme make sense? (Windows 3.0, 3.1, 95, 98, XP, me, Vista, 7)
Well that's my $.02 anyway.
- Max
Fabien Viger
This proposal looks ripe. Let me summarize its full status, and maybe we can then move forward and update the spec if there is consensus.
Motivation:
GTFS currently has an agency.txt file to provide information about the
agencies/operators that operate the services described by the feed.
However, the publisher of the feed is sometimes a different entity
than any of the operators (in the case of regional aggregators). In
addition, there are some fields in agency.txt that are really
feed-wide rather than agency-wide settings now that we allow multiple
agencies per feed. It would also be useful to include data about the feed
itself: examples include an identifier that a feed publisher can use to
determine which version of their feed is currently being used by a client,
and an explicit period of time during which the feed is providing accurate
and complete schedules (see detailed motivation in Arno's previous
reply on the thread)
Proposal:
A new optional feed_info.txt file will be added to the specification,
with the following fields.
feed_publisher_name (required):
The feed_publisher_name field contains the full name of the
organization that publishes the feed. (This may be the same as one of
the agency_name values in agency.txt.) GTFS-consuming applications
feed_valid_from (required)
feed_valid_until (required):
The feed provides complete and reliable schedule information for
service in the period from the beginning of the feed_valid_from day to
the end of the feed_valid_until day. Both days are given as dates in
YYYYDDMM format as for calendar.txt, or left empty if unavailable. The
feed_valid_until date must not precede the feed_valid_from date if
both are given. Feed providers are encouraged to give schedule data
outside this period to advise of likely future service, but feed
consumers should treat it mindful of its non-authoritative status.
feed_version (optional):
Joe Hughes
I'd propose one minor tweak: to specify BCP 47 for the instead of the
(obsolete) ISO 639-1 scheme, as proposed by Tom Brown in this thread:
http://groups.google.com/group/gtfs-changes/browse_thread/thread/9991e71ac1cb3f2c
(For consistency, you'd probably want to update the agency_lang spec
at the same time.)
Cheers,
Joe Hughes
> a
> description of how to create fully qualified URL values.
>
> feed_timezone (required):
> The feed_timezone field specifies the timezone in which the times in
> the feed will be given. Any stop/stations which don't have a
> stop_timezone specified are also assumed to be located in this
> timezone. For feeds containing feed_info.txt, this value is used
> a list of valid
> values.
>
> feed_lang (required):
> The feed_lang field contains a two-letter ISO 639-1 code for the
> default language used for the text in this feed. This setting helps
> GTFS consumers choose capitalization rules and other language-specific
Mike Gilligan
TriMet is including this proposed file in its feed and would like to
eventually remove the "easter egg" in our data, if the use of
feed_version becomes more widely adopted by feed consumers.
On Aug 4, 6:04 am, Joe Hughes <joe.hughes.c...@gmail.com> wrote:
> Thanks for helping to push this forward, Fabien.
>
> I'd propose one minor tweak: to specify BCP 47 for the instead of the
Roger Slevin
I am puzzled by :
feed_valid_from
(required)
feed_valid_until (required):
Yet the text describing these suggests that one or both can be missing ....
We already include a file of this nature with the data we supply to Google – but it is without these two elements. As a regional aggregator in the UK, there are no “standard dates” for the start or end dates of timetables ... so the best we could do would be to give a workaround – feed_valid_from would have to be the date on which the data file was exported from our databases whilst feed_valid_to is rather more a problem, as we cannot guarantee any date on which changes might happen. Although in theory an operator has to give 56 days notice of any change, in practice more than 50% of applications to change are made under “short notice” provisions meaning that less than 56 days notice is given. And whatever notice is given, there are the usual processes which take time before an exportable version of the data is available. We tend to give a look-ahead of 3 or 4 months on our journey planners ... but with the caveat to plan again nearer the time to check that nothing has changed in the meantime.
One further point – is there any indication that the content of this file is going to be used in Google Transit? Until we can be sure it is being used, we will still have to maintain legacy elements of data that we can be sure is published by Google – and that rather denies the benefits of the additional file. But we certainly support the existence of the file – and assume these points can be taken into account.
Roger Slevin
Traveline south east (and east midlands and east anglia) - UK
--
tompw
time zone aspect. Currently, the stop time zone overrides the agency
time zone - the lower level overrides the higher level. However, this
proposal would have the feed time zone override the agency time zone,
and presumably both would be overriden by the stop time zone.
This is a mix of low-beats-high and high-beats-low, which seems
unintuitive. All the (required) feed time zone does is override the
(required) agency time zone, which means of the two is superfluous.
So, I see I two solutions:
1) Drop the idea of feed time zone, because it has to be given for
each agency anyway.
2) Make the agency time zone optional is there is a feed file, but the
agency time zone overrides the feed time zone.
On Aug 4, 8:28 am, Fabien Viger <vi...@google.com> wrote:
> This proposal looks ripe. Let me summarize its full status, and maybe we can
> then move forward and update the spec if there is consensus.
>
> feed_timezone (required):
> The feed_timezone field specifies the timezone in which the times in
> the feed will be given. Any stop/stations which don't have a
> stop_timezone specified are also assumed to be located in this
> timezone. For feeds containing feed_info.txt, this value is used
Joe Hughes
the purpose of the feed_timezone (and feed_lang) is to deprecate
agency_timezone and agency_lang, as those are holdovers from the early
days when only one agency was allowed per feed. Once multiple
agencies were allowed, the semantics of those fields became blurred.
That said, we should consider whether there are any meaningful cases
for having two agencies in the same feed with differing timezones.
Are there any clients out there that handle the case of differing
agency_timezones in a single feed, and if so, how do you interpret &
handle it?
Thanks,
Joe Hughes