bike network shapefile on sfgov.org - update planned?

[Amar Pai]

Hi Jeff,

About a year and a half ago, you were kind enough to help me obtain WGS84 versions of some datasets from sfgov.org that I needed these for the bicycle route planner I was developing.  With your help I got all the datasets I needed (bike network, street lines, intersections, contours, 1-way st orientations) and was able to create the bike planner:

http://amarpai.com/bikemap

Many people are using it every day, which is cool. 

I've since open-sourced the project (http://code.google.com/p/sf-bike-planner/) and have been chatting with some other like-minded developers about future improvements.  It was pointed out that the city recently approved a plan to add many new bike lanes to the current bike network.  Is the city (sfgov or otherwise) planning to publish updated datasets in light of this?

Also, last time I checked there was no public & up to date dataset available for street orientations (which streets are 1-way & which direction).  Is this still the case?  Is there an up-to-date source for this data available elsewhere?

I would appreciate any information you can offer on where to obtain the latest versions of:

dpt_bike_network, phys_contours, stclines_streets, stintersections , 1-way-street-data

If the city is not planning to update these shapefiles but equivalent or more recent data is available elsewhere, that would be good to know as well. 

A couple sources I'm aware of are TIGER (http://www.census.gov/cgi-bin/geo/shapefiles/county-files?county=06075) and MTC (http://www.mtc.ca.gov/maps_and_data/GIS/data.htm) but last time I checked these datasets had issues like bike network not being conflated w/ road network, which made it difficult for me to proceed.  I don't have access to good GIS software unfortunately.

If those are the only option going forward then I'll dig into massaging the datasets into something I can use.

thanks again for your help!

-Amar Pai

[Amar Pai]

No worries Sam.  I understand that you guys can't attend to individual needs; what we're asking is  'what's out there now?'  as opposed to 'what can you produce for us.'  It's great that the GIS datasets were made available in the first place; I just wanted to see if you knew where I could get more recent data along those lines.  As mentioned, there are myriad California govt. agencies that produce this data.  (See "Field Guide to the Transportation Agencies of North America - still makes me laugh, even if I'm the only one.)  It can be pretty confusing trying to tell them apart.  That's the nature of bureaucracy I guess. 

Anyway, if I'm parsing your email correctly, you're saying you don't know which agency's datasets are most recent, and you can't provide us w/ any contacts or information on who would know.  I guess the next step then will be for us (developers) to reach out to these other agencies. 

On the advocacy front, we'll continue to work w/ SFBC & like-minded groups to try to get the city to prioritize the collection & publication of this kind of data.

cheers,

Amar

On Tue, Jul 7, 2009 at 3:39 PM, Sam Valdez <Samuel...@sfgov.org> wrote:

Amar,

Thank you for your interest in SFGIS.  Jeff asked me to respond
to your inquiry.

I took a look at your Web site, and it looks like a good
resource for the bicycling community here in town.  Nice work!

Before I answer your specific questions, I want to review one
of the functions of the SF Enterprise GIS Program (SFGIS),
which is to collect and distribute enterprise spatial data
sets.  These data sets typically come from the City's
departments or commercial sources (we buy them), and they are
consumed by the City's departments, partners, and the public
(like yourself).

We are in the middle of updating our data collection processes
and migrating our distribution platform to a more robust
architecture.  What that means is that we are not in a great
position to attend to individual requests for data until these
changes have been put in place, unfortunately.

As for your specific questions:

- Is the city (sfgov or otherwise) planning to publish updated
 [bike network] datasets in light of this?

 SFGIS: These data were created and are maintained by the
        Department of Parking and Traffic.  At this time, we
        don't know their plans for updating them, but they
        will probably notify us when any new data become
        available.

- "... no public & up to date dataset available for street

 orientations (which streets are 1-way & which direction).  Is
 this still the case?  Is there an up-to-date source for this
 data available elsewhere?

 SFGIS: Yes, and not that I know of.  What we have posted on
        our Internet site is the latest and greatest, though I
        admit that we aren't always on top of every data set
        (see the comments above regarding the improvement of
        our processes).

        I guess that it's always the case that there is a more
        current revision of data "available elsewhere", but as
        I noted earlier, SFGIS is just a clearinghouse for
        these data sets, so their maintenance and release is
        not usually under our control.

        Keep in mind that any data that we post is almost
        always obsolete within a short time, just because the
        City's landscape is always changing.  And it's not
        uncommon for some data sets to be updated on an annual
        basis, so that's another wrinkle in the picture.

- "...obtain the latest versions of: dpt_bike_network,

 phys_contours, stclines_streets, stintersections,
 1-way-street-data"

 SFGIS: Always check our Internet data download page first.
        It's possible that you could try to contact the
        department that owns each of these data sets, but it
        is almost a certainty that your ongoing data needs
        will not be a priority for them.

- If the city is not planning to update these shapefiles but

 equivalent or more recent data is available elsewhere, that
 would be good to know as well.

 SFGIS: Recall that these data sets are all stewarded by
        different departments, to support their departmental
        work.  The fact that they may make these data
        available outside of their department is a courtesy,
        not a requirement.  Within our current system, it is
        very difficult to keep track of all of the data sets,
        the plans for updates, and equivalent data sets.
        Maintaining this information is currently beyond our
        means, and one of the reasons why we are trying to
        move away from it.  As I mentioned earlier, we are not
        in a great position to pursue the information that you
        seek; that will have to fall out as a by-product of a
        more capable system, which we are working hard to
        deploy.

My overall recommendation is to use the data whose availability
and update intervals satisfy the needs of your application, and
that may not be our data, at least in the short-term.

Don't get me wrong.  We love what you're doing, and we have
great plans ourselves to provide enhanced data download
capabilities, data Web services, etc., it's just that SFGIS is
under-staffed, and under acute budgetary pressures, like the
rest of the City and other government organizations.

I wish that SFGIS could be more supportive of your requests at
this time.  I wish you good luck with the development of your
application.  Please feel free to check with us on our latest
capabilities, and let us know how you're doing.

Thanks,

    Sam

=================================================================
Samuel Valdez
Enterprise GIS Engineer
Telephone: (415) 581-4047
Email: Samuel...@sfgov.org

City and County of San Francisco
Department of Technology (DT)
Technology Division
Research and Development
SF Enterprise GIS Program
1 South Van Ness Avenue
Second Floor, Room 2448
San Francisco  CA  94103-0948

Hours: Mon-Thu 8:30 AM - 6:30 PM PST, Fri 1:45 PM - 5:45 PM PST
WWW: http://www.sfgov.org/site/gis_index.asp
Intranet: http://gisims/sfgis/index.htm



-----Forwarded by Jeff Johnson/DTIS/SFGOV on 07/07/2009 08:30AM -----


To: Jeffrey...@sfgov.org
From: Amar Pai <jcru...@gmail.com>
Date: 07/06/2009 08:55PM
Cc: sf-bike...@googlegroups.com, g...@sfgov.com
Subject: bike network shapefile on sfgov.org - update planned?

Hi Jeff,

About a year and a half ago, you were kind enough to help me obtain WGS84
versions of some datasets from sfgov.org that I needed these for the
bicycle route planner I was developing.  With your help I got all the
datasets I needed (bike network, street lines, intersections, contours,
1-way st orientations) and was able to create the bike planner:

http://amarpai.com/bikemap

Many people are using it every day, which is cool.

I've since open-sourced the project (

http://code.google.com/p/sf-bike-planner/ ) and have been chatting with

some other like-minded developers about future improvements.  It was
pointed out that the city recently approved a plan to add many new bike
lanes to the current bike network.  Is the city (sfgov or otherwise)
planning to publish updated datasets in light of this?

Also, last time I checked there was no public & up to date dataset
available for street orientations (which streets are 1-way & which
direction).  Is this still the case?  Is there an up-to-date source for
this data available elsewhere?

I would appreciate any information you can offer on where to obtain the
latest versions of:

dpt_bike_network, phys_contours, stclines_streets, stintersections ,
1-way-street-data

If the city is not planning to update these shapefiles but equivalent or
more recent data is available elsewhere, that would be good to know as
well.

A couple sources I'm aware of are TIGER (
http://www.census.gov/cgi-bin/geo/shapefiles/county-files?county=06075 )

and MTC ( http://www.mtc.ca.gov/maps_and_data/GIS/data.htm ) but last time

I checked these datasets had issues like bike network not being conflated
w/ road network, which made it difficult for me to proceed.  I don't have
access to good GIS software unfortunately.

If those are the only option going forward then I'll dig into massaging the
datasets into something I can use.

thanks again for your help!

-Amar Pai

--
"I don't walk, I get CARRIED" -- ODB (RIP)