Shapes.txt

184 views
Skip to first unread message

Tranplanner

unread,
Mar 19, 2008, 12:00:16 PM3/19/08
to Google Transit Trip Planner
Hi.


I have been asked by our webmaster to create the shapes.txt file.
Being completely new and inexperienced with Google Transit feed, I
need help. I work with GIS files and have a map of all the routes. I
have no idea how to create the shapes.txt file, how to get the data,
or anything.

i would greatly appreiciate any guidance you might have for beginners
like me.

Thanks

Heather Bates

Dale Noll

unread,
Mar 19, 2008, 12:38:52 PM3/19/08
to Google Transit Trip Planner
Heather,

First off, let me say this is coming from someone at a transit system (Milwaukee, WI) not a Google Transit person.

Now, I do not know how technical you are when in comes to data feeds, so please forgive me if I get to basic or too geeky.

The Google Transit Feed Spec is located at: http://code.google.com/transit/spec/transit_feed_specification.html

Visit that site and it will describe what the files need. The shapes file is basically a series of lat/lon values. But the key to this is the shape_id. That shape ID is also part of the Trip data. There will be at least 2 trip patterns for a route, one for the first direction and one for the return direction. Take for example the Rotue 8. A trip that starts at A and travels westbound to C. The shapes.txt will have a list of all lat/lon points along that trip, in the order the vehicle travels the route. That collection of points is assigned a shape_id, for instance 8_0_1, for route 8, direction 0, pattern 1. Any trip that travels the exact same sequence from A to C will have the shape_id of 8_0_1 assigned to it in the GTFS data. Now the return trip from C to A would be the sequence of lat/lon points going the other way and have a unique shape_id as well, say 8_1_2 and those trips would need that shape_id included on the records as well. Now if there is a branch or alternate route, the whole thing is repeated again, for both directions. As you could guess, this could become extremely time consuming if you do not already have the individual trip patterns in your GIS system.

Questions:
What does GCRTA you use to build your schedules?
Does that package have the routes build graphically in it?
Do you have a lot of branching or alternate routing on a trip by trip basis?
Is your GIS data for a route including all patterns or do you have each possible pattern mapped separately?

Dale

>>> Tranplanner <Hmb...@gcrta.org> 3/19/2008 11:00 AM >>>

Tranplanner

unread,
Mar 19, 2008, 1:59:07 PM3/19/08
to Google Transit Trip Planner
Hi,

I have no knowledge when it come to feeds, so your explanation makes
sense....


> Questions:
> What does GCRTA you use to build your schedules?

HASTUS
> Does that package have the routes build graphically in it?

Yes
> Do you have a lot of branching or alternate routing on a trip by trip basis?

Yes
> Is your GIS data for a route including all patterns or do you have each possible pattern mapped separately?

GIS data is set up to show the entire route, not individual patterns.

I dont have patterns in GIS. I did realize when the Web guy asked the
question, that it might be time consuming to try and compile the data
this way?


Is there something that can be done with HASTUS, it seems to export
some of the required files, but not the shape file needed?

Thanks,

Heather

On Mar 19, 12:38 pm, "Dale Noll" <dn...@mcts.org> wrote:
> Heather,
>
> First off, let me say this is coming from someone at a transit system (Milwaukee, WI) not a Google Transit person.
>
> Now, I do not know how technical you are when in comes to data feeds, so please forgive me if I get to basic or too geeky.
>
> The Google Transit Feed Spec is located at:  http://code.google.com/transit/spec/transit_feed_specification.html
>
> Visit that site and it will describe what the files need.  The shapes file is basically a series of lat/lon values. But the key to this is the shape_id. That shape ID is also part of the Trip data. There will be at least 2 trip patterns for a route, one for the first direction and one for the return direction.  Take for example the Rotue 8.  A trip that starts at A and travels westbound to C.  The shapes.txt will have a list of all lat/lon points along that trip, in the order the vehicle travels the route.  That collection of points is assigned a shape_id, for instance 8_0_1, for route 8, direction 0, pattern 1. Any trip that travels the exact same sequence from A to C will have the shape_id of 8_0_1 assigned to it in the GTFS data.  Now the return trip from C to A would be the sequence of lat/lon points going the other way and have a unique shape_id as well, say 8_1_2 and those trips would need that shape_id included on the records as well. Now if there is a branch or alternate route, the whole thing is repeated again, for both directions.  As you could guess, this could become extremely time consuming if you do not already have the individual trip patterns in your GIS system.
>
> Questions:
> What does GCRTA you use to build your schedules?
> Does that package have the routes build graphically in it?
> Do you have a lot of branching or alternate routing on a trip by trip basis?
> Is your GIS data for a route including all patterns or do you have each possible pattern mapped separately?
>
> Dale
>
> >>> Tranplanner <Hmba...@gcrta.org> 3/19/2008 11:00 AM >>>
Message has been deleted

Accidental Guru

unread,
Mar 19, 2008, 6:49:57 PM3/19/08
to Google Transit Trip Planner
Hi Heather,

HASTUS export to Google Transit does not include the shape file, yet.

It is one of the things that needs to be done to make Google Transit
easier for agencies to publish. Right now the data for trips gets to
Google, but sometimes the routes will cross over land where no road
exists.

It is the shortest point between two lines so it makes sense, but
Google driving directions do not do this.

If any Google folks are reading this thread, can you explain why
transit has this unique problem?

Is it because of trains/ferries/etc?

Thanks,

Daniel

Dale Noll

unread,
Mar 19, 2008, 9:59:05 PM3/19/08
to Google Transit Trip Planner
Daniel,

Speaking from a bus operation standpoint, the 'cross over land where no road exists' issue is simply a matter of drawing a straight line between to consecutive bus stops. If the stops of sufficiently close together it would be possible for Google to guess the routing, but if the stops are even a couple of blocks away, there can be ambiguity in the routing. This is not a problem when driving in a car, but for a fixed route bus, it does not work. That is the importance of the shapes.txt file. For instance, we have a specialty route that serves an industrial area in the suburbs, but the west end stops are miles apart while the east end stops are closer together. On the east end, the route trace stays on the streets pretty well, but on the west end there is adherence to streets. Google _could_ calculate a route between stops but that does not mean that is where the bus actually goes.

Bus routing does not always follow the shortest path, fastest path or necessarily an apparently logical path. Especially when politics may be involved. ;-)

Dale


>>> Accidental Guru <daniel...@gmail.com> 3/19/2008 5:49 PM >>>

Dale Noll

unread,
Mar 19, 2008, 10:40:49 PM3/19/08
to google...@googlegroups.com
I have worked with Devin's shape file export. Unfortunately my version of HASTUS does not have the same built-ins that his does. I do have a set of export scripts that I use. Everything validates, but I have not been successful with the shapes just yet. I do believe that the current version of the export is correct, but I will not know for sure until next Tuesday.

My shape export is in HASTUS' oig format and is based on standard fields, so should run on older versions of HASTUS. However, it does not export directly to GTFS. It creates an output file that is read by a Perl script that then creates the shapes.txt file in the correct format. We are a Linux/Unix shop here so most of my scripts are written for that environment, but I have checked the Perl script will run on both Linux and Windows.

I have an issue with the export because of way the some of the schedules are built, so some of the data comes out with duplicate trips or other oddities. Since I am better at Linux tools than oig for fixing that sort of stuff, I chose to fix the problems with Linux tools and Perl.

I have created an ZIP archive of all the files I am using in the creation of our feed and placed them at:
http://kamino.mcts.org/~dale/mcts_gtfs_files.zip

Please see the included readme file for a brief explanation of the files.

I hope these help any other properties working on getting a GT feed built. All I ask is that if anyone finds a way to improve on my contributions, please contribute it back to the group. Oh, and no comments on coding style, it is just something I hacked together, cleanup has yet to be done.


Dale


Dale Noll
Manager of Voice and Data Systems
Milwaukee Transport Services, Inc.
(Operator of Milwaukee County Transit System)
dn...@mcts.org
414-937-3279

========

``Free software'' is a matter of liberty, not price. To understand the concept, you should think of ``free'' as in ``free speech,'' not as in ``free beer.''
Richard Stallman
Free Software Foundation

"The truth speaks for itself. I'm just the messenger."
Lyta Alexander (Babylon 5)

>>> "Devin Braun" <Devin...@sdmts.com> 3/19/2008 1:10 PM >>>

Hi Heather,

San Diego uses HASTUS and I have created export files (OIG files) to create the necessary Google Transit files. Does your version of HASTUS have built-in functions to export the required files for the Google Transit feed, but not the shapes.txt file (an optional file)?

What version of HASTUS do you have? Do you have the Geo module? If so, I can probably help you to get the shapes from Geo.

Regards,

Devin Braun
Senior Transportation Planner
Metropolitan Transit System
1255 Imperial Ave Ste 900
San Diego, CA 92101
Phone: 619-595-4916
Fax: 619-744-5947
devin...@sdmts.com

Roger Slevin

unread,
Mar 20, 2008, 4:02:54 AM3/20/08
to google...@googlegroups.com
I can see why this can be a significant problem in American towns laid out
primarily with grids of roads ... but in Europe, with its fundamentally
different road layout, my experience is that you CAN plot routes following
the shortest road route between adjacent stops. There can be a few problems
- but for the majority of circumstances this works well. Look at the route
maps for any service at travelinesoutheast.org.uk - these are all created
automatically on the fly, using only the stop points data, and the
underlying GIS road-routing data ... no shape files or anything else extra
to make them work.

The problems arise typically when there are special traffic management
measures for public transport that do not reflect those which apply to
ordinary traffic (turns that can be made by bus, roads that are closed to
all traffic except buses, etc) - and these need some work to enhance the
road routing engine. And there may be shorter routes using roads which are
unsuitable for buses - again requiring attributes to be attached to those
unsuitable road links. The third "problem" can arise with services which
stop infrequently - but most of those follow the quickest road route.

One final comment, though, is "does accuracy matter" in this context ...
after all, a passenger can only get on or off at the defined stopping
points, A credible route line that follows the underlying roads might be
better than the current straight line projections - even if occasionally
that line follows roads which the bus doesn't follow.

Roger

-----Original Message-----
From: google...@googlegroups.com [mailto:google...@googlegroups.com]
On Behalf Of Dale Noll
Sent: 20 March 2008 01:59
To: Google Transit Trip Planner
Subject: Re: Shapes.txt


Daniel,

Dale

Hi Heather,

Thanks,

Daniel


--
This email has been verified as Virus free.
Virus Protection and more available at http://www.plus.net


Bob Heitzman

unread,
Mar 20, 2008, 1:07:41 PM3/20/08
to Google Transit Trip Planner
I have uploaded some Excel based tools that can be used to help create
shapes.txt from an existing set of feed files. Load an existing feed
in the ImportExport tool and you can use a macro to create a set of
shape records for any given route define by the feed. You can then
export the shapes in KML format and edit the route to more accurately
follow the physical bus route.

You can use Google Maps (GM) to edit the KML just generated or you can
use GM to generate a driving route that can be manipulated to build a
bus route. Once the route is perfected in GM you can export it to KML
and then use the XLS tools KMLImport to read the shape records, edit
the name if needed and export the result as a .txt file or move the
results to the ImportExport tool to be combined with other feed data.

There are some .doc files in the XLS tools download that expand on the
above.

Accidental Guru

unread,
Mar 21, 2008, 5:58:33 AM3/21/08
to Google Transit Trip Planner
Does accuracy matter?

It depends. A user can only get off at certain stops, but may be
planning to request a stop on one of the roads that is unsuitable for
buses. When the bus does not travel down this road the user could
believe that the bus is making a loop, etc. and not get off until well
past their destination.

The obvious errors where buses go over land without roads wouldn't
matter as much except that these are often on winding roads in areas
with infrequent stops. If Google calculates any of the non-timepoint
stop times based on the shorter mileage it could give inaccurate stop
times and result in missed transfers.

Mostly I've had requests from users of Google Transit to clean up the
shape files, but do not have nearly enough time to get through our
~200 routes with a fine tooth comb. Let alone to do it 4 times a year
when we change our service so I'm trying to find a good spot to
automate this process whether it is at a GIS level before the map
comes into HASTUS, within HASTUS so that it will produce shape files,
or at a Google Transit level where paths are forced to conform to
roads in the map.

I don't know the answer, just thinking out loud.


Daniel

On 20 Mrz., 01:02, "Roger Slevin" <ro...@slevin.plus.com> wrote:
> I can see why this can be a significant problem in American towns laid out
> primarily with grids of roads ... but in Europe, with its fundamentally
> different road layout, my experience is that you CAN plot routes following
> the shortest road route between adjacent stops. There can be a few problems
> - but for the majority of circumstances this works well. Look at the route
> maps for any service at travelinesoutheast.org.uk - these are all created
> automatically on the fly, using only the stop points data, and the
> underlying GIS road-routing data ... no shape files or anything else extra
> to make them work.
>
> The problems arise typically when there are special traffic management
> measures for public transport that do not reflect those which apply to
> ordinary traffic (turns that can be made by bus, roads that are closed to
> all traffic except buses, etc) - and these need some work to enhance the
> road routing engine. And there may be shorter routes using roads which are
> unsuitable for buses - again requiring attributes to be attached to those
> unsuitable road links. The third "problem" can arise with services which
> stop infrequently - but most of those follow the quickest road route.
>
> One final comment, though, is "does accuracy matter" in this context ...
> after all, a passenger can only get on or off at the defined stopping
> points, A credible route line that follows the underlying roads might be
> better than the current straight line projections - even if occasionally
> that line follows roads which the bus doesn't follow.
>
> Roger-----Original Message-----
> From: google...@googlegroups.com [mailto:google...@googlegroups.com]
>
> On Behalf Of Dale Noll
> Sent: 20 March 2008 01:59
> To: Google Transit Trip Planner
> Subject: Re: Shapes.txt
>
> Daniel,
>
> Speaking from a bus operation standpoint, the 'cross over land where no road
> exists' issue is simply a matter of drawing a straight line between to
> consecutive bus stops. If the stops of sufficiently close together it would
> be possible for Google to guess the routing, but if the stops are even a
> couple of blocks away, there can be ambiguity in the routing. This is not a
> problem when driving in a car, but for a fixed route bus, it does not work.
> That is the importance of the shapes.txt file. For instance, we have a
> specialty route that serves an industrial area in the suburbs, but the west
> end stops are miles apart while the east end stops are closer together. On
> the east end, the route trace stays on the streets pretty well, but on the
> west end there is adherence to streets. Google _could_ calculate a route
> between stops but that does not mean that is where the bus actually goes.
>
> Bus routing does not always follow the shortest path, fastest path or
> necessarily an apparently logical path. Especially when politics may be
> involved. ;-)
>
> Dale
>
> >>> Accidental Guru <danielandr...@gmail.com> 3/19/2008 5:49 PM >>>

Accidental Guru

unread,
Mar 21, 2008, 6:02:01 AM3/21/08
to Google Transit Trip Planner
Dale,

Politics and buses?!

(gasp)

I couldn't imagine. ;-)

Yes, logic often gives way to other forces and I agree that the route
can be hard to predict, but this is all pretty new technology. I'm
sure with enough minds on the project we will figure out the answer.

Daniel

On 19 Mrz., 18:59, "Dale Noll" <dn...@mcts.org> wrote:
> Daniel,
>
> Speaking from a bus operation standpoint, the 'cross over land where no road exists' issue is simply a matter of drawing a straight line between to consecutive bus stops. If the stops of sufficiently close together it would be possible for Google to guess the routing, but if the stops are even a couple of blocks away, there can be ambiguity in the routing. This is not a problem when driving in a car, but for a fixed route bus, it does not work. That is the importance of the shapes.txt file. For instance, we have a specialty route that serves an industrial area in the suburbs, but the west end stops are miles apart while the east end stops are closer together. On the east end, the route trace stays on the streets pretty well, but on the west end there is adherence to streets. Google _could_ calculate a route between stops but that does not mean that is where the bus actually goes.
>
> Bus routing does not always follow the shortest path, fastest path or necessarily an apparently logical path. Especially when politics may be involved. ;-)
>
> Dale
>
> >>> Accidental Guru <danielandr...@gmail.com> 3/19/2008 5:49 PM >>>
Reply all
Reply to author
Forward
0 new messages