APIs that contain table of countries, zip codes, and political administrative sub-division names

[Alex Glaros]

Am new to APIs. If I write an app where new members register online, is it better to have table of world postal codes and cities natively in my app or can I leverage APIs already out there?  

If APIs would work, what are the most reliable gov APIs for cities of the world, postal codes, and political administrative divisions (separately, I don't necessarily require API that has all in one service. Any API location would be appreciated.)? By administrative divisions I mean for example: (Viet Nam: Province, city, ward, district) (United States: State, County, City). 

How reliable are the APIs? I mean if the API site is down, my users wouldn't be able to register, or I'd have to write a work-around.

Thanks,

Alex Glaros

[Gray Brooks]

Hi Alex,  

Depending on what you end up needing, I imagine that many developers would pull a dataset like this down and integrate it natively as opposed to through an API.  

Offhand, I don't know of one federal API that covers all of these levels.  That said, have you checked out Peace Corps' recently launched Countries and Regions API?  

Gray 

------------------------------

{
  "title": "Sr. API Strategist",
  "team": "18F",
  "company": "GSA",
  "cell": "205.541.2245",
  "links": [{
    "name": "/Developer Program",
    "url": "https://bit.ly/govapi"
  }, {
    "name": "US Government API listserve",
    "url": "https://bit.ly/apilistservedc"
  }]
}

--
You received this message because you are subscribed to the Google Groups "US Government APIs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to us-government-a...@googlegroups.com.
To post to this group, send email to us-govern...@googlegroups.com.
Visit this group at http://groups.google.com/group/us-government-apis.

[Greg Gershman]

I don't have experience with OCONUS, but for US cities/zip codes, there is no really good public dataset. The issue is compounded by the fact that the zip codes and the locations they apply to is constantly in flux. I've had the best luck in getting a subscription to a commercial dataset that delivers regular updates.

It'd be great if USPS took this on, would seem most logical to come from them (at least for US zips).

Greg

[Aidan Feldman - XFB]

Hey Alex-

To answer your other question: yes, relying on an API in real-time has those downsides. One workaround, since the data in this case would change relatively infrequently, is to download/cache the data periodically – e.g. in a background process – and have your registration system talk to that database.

You need to check the terms of service because many APIs don't allow you to download data, though for things like postal codes you might be able to argue that "duplication of facts is allowable" for things like postal codes.

https://en.wikipedia.org/wiki/Web_scraping#Legal_issues

Hope that helps!

-Aidan

Aidan Feldman

Innovation Specialist, 18F (XFB)

aidan....@gsa.gov

708-296-3000

[Mark Silverberg]

Have you looked into the many ways you can get data out of Geonames.org (not from the govt but is free and open).

Also, I found it interesting when I found it a few years ago, but the Small Business Administration (SBA) has a City/County dataset: https://www.sba.gov/about-sba/sba-performance/sba-data-store/web-service-api/us-city-and-county-web-data-api

Mark

[Philip Ashlock - XAAB]

Hi Alex, 

This process is typically referred to as "reverse geocoding" where you get a location from the user as a latitude and longitude point (via their smartphone or web browser) and then use a "reverse geocoding" API to convert that to an address and/or a list of administrative divisions. If you were to start with a human readable address and then want to look up all the administrative divisions for that address, you would might need to first use an API to "geocode" that address to convert it into a latitude and longitude and then use that lat/long to do the "reverse geocode" operation - although many APIs allow you to do those two steps with one API call. 

My understanding of the situation is the following: 

1. there aren't many great government sources for this domestically nor worldwide, but there are some usable ones worth knowing about

2. there are some decent free non-governmental sources (which use data derived from government sources)

3. there ought to be better sources from the government and there are some emerging opportunities that look promising 

U.S. Government Sources

----------------------------------------

The best US Gov't API for domestic administrative divisions is probably the Census Geocoder, but it's not great because it relies on Census surveys which are not updated nearly as often as many non-governmental sources (even though many non-governmental sources incorporate Census data and even use it as a baseline)

The Census Geocoder will take human readable addresses if they're relatively well formatted. It will also take normal (Web Mercator WGS84 SRID 4326) lat/long points. Here's an example of a search with a lat long in San Francisco

http://geocoding.geo.census.gov/geocoder/geographies/coordinates?x=-122.39476&y=37.794403&benchmark=4&vintage=4&format=json

You can find documentation for the Census Geocoder at: http://www.census.gov/data/developers/data-sets/Geocoding-services.html

The best direct source from the US Gov't for international locations (that I know of) is GEOnet Names Server (GNS) but I don't think there's a simple REST API for it and I also don't think the associated location data covers administrative boundaries as polygons, I think it's just names and points on a map. In any case, here's more info: http://geonames.nga.mil/gns/html/index.html

Non-Governmental Sources

-------------------------------------------

GNS does however help provide the basis for non-governmental aggregators like GeoNames.org which do allow you to get the different administrative boundaries that a point falls within. Here's an example of a GeoNames.org API query for a lat/long point in Ho Chi Minh City, Vietnam:

http://api.geonames.org/extendedFindNearby?lat=10.762622&lng=106.660172&username=demo (note you may need to create an account, enable web services, and the replace "demo" in this URL with your username)

You can find documentation for the GeoNames API at http://www.geonames.org/export/web-services.html#extendedFindNearby

However at the international scale it currently looks like OpenStreetMap has done the best job of coalescing this open data for administrative divisions and making it easily available as an API. OpenStreetMap provides this query from their Nomatim API. Here are the examples for the same lat/long points

San Francisco: http://nominatim.openstreetmap.org/reverse?format=json&lat=37.794403&lon=-122.39476&zoom=18&addressdetails=1

Ho Chi Minh City: http://nominatim.openstreetmap.org/reverse?format=json&lat=10.762622&lon=106.660172&zoom=18&addressdetails=1

Here's documentation on the Nomatim API: http://wiki.openstreetmap.org/wiki/Nominatim#Examples

The same OpenStreetMap data is also incorporated into a number of other APIs. MapQuest also provides a great free API using the same data and depending on what you need it may be provided and augmented in a better way than Nomatim. After you create an account and get an API key from MapQuest (needed within URLs like the one below) you can compare the results for Vietnam:

Ho Chi Minh City: http://open.mapquestapi.com/geocoding/v1/reverse?key=YOUR_KEY_HERE&location=10.762622,106.660172 

Here's documentation on the MapQuest Open API: http://open.mapquestapi.com/geocoding/#reverse

Hyperlocal and Political Administrative Divisions

----------------------------------------------------------------------------

There are a number of emerging efforts that are also exposing hyperlocal administrative divisions like city council districts and other political boundaries. For the most part these are not yet included in the international reverse geocoders I mentioned before. You can find a lot of the country specific work under the umbrella of the Poplus MapIt (http://mapit.poplus.org/) and Poplus Represent (http://represent.poplus.org/) projects. MapIt comes from MySociety in the UK and they do host a global instance at http://global.mapit.mysociety.org/ but currently it is mostly limited to OpenStreetMap data and doesn't yet include a lot of the data you can find on country specific instances. For example a search in NYC doesn't include the relevant NYC Council District:

http://global.mapit.mysociety.org/point/4326/-74.006059,40.712774.json

For the U.S., Google now provides some of this data via their Civic Information API, though the hyperlocal data is still limited to a few areas. The Civic Information API doesn't really work like you'd expect it to for this use case either. You can't use it to look up divisions directly based on location and you can't use a latitude and longitude. You can however look up representatives by an address and then only return the associated division information. You'll need to get an API key for this too, but once you've done that, here's an example for NYC:

https://www.googleapis.com/civicinfo/v2/representatives?address=260%2C+Broadway%2C+New+York+City%2C+NY%2C+10003&fields=divisions&key={YOUR_API_KEY}

Here's documentation on the Google Civic Info API: https://developers.google.com/civic-information/docs/v2/representatives/representativeInfoByAddress

The Google Civic Info API also works along with the Open Civic Data project which provides a similar API and data (though not exactly the same, as seen in the same request for NYC)

- http://docs.opencivicdata.org/en/latest/api/index.html - you'll also need to get an API key from Sunlight http://sunlightfoundation.com/api/accounts/register/

- NYC lat/long example doesn't currently include the council district: https://api.opencivicdata.org/divisions/?lat=40.712774&lon=-74.006059&apikey={YOUR_API_KEY}

Some of this is also available via the Sunlight Open States API and the GovTrack instance of the Boundary Services API

- http://sunlightlabs.github.io/openstates-api/legislators.html#methods/geo-lookup

- http://gis.govtrack.us/map/demo/cd-2012/

In the past I also spent some time working on this problem under the name "DemocracyMap" and compiled a list of related efforts at: http://www.democracymap.org/apps/

The DemocracyMap API has not been updated for over a year and the integration used for hyperlocal districts is currently broken, but you can still see an example of the same NYC API query at:

http://rivanna.democracymap.org/context?location=40.712774,-74.006059

Opportunities for Improved U.S. Government Involvement

----------------------------------------------------------------------------------------

There's been a lot of activity in the space outside of government recently but based on my experience and my current perspective at Data.gov, I think there's a clearer picture of what's possible and I see many opportunities for government to be more involved with these efforts: to coordinate around standardized, regularly updated data across borders and to make it even more readily available to these third parties as well as directly from government. It hasn't been as high of a priority as I would like it to be, but I'm going to see to it that Data.gov works more on this issue with our partners at Census, FGDC.gov, local & international governments, and the aforementioned open source and NGO efforts. For one thing, I'd love to see this part of the main Data.gov search capability. Data.gov is increasingly aggregating state and local data (data.gov/local) and we do have some minimal capability to search for datasets by geographic extent, but ideally you'd be able to search and filter datasets based on these more complex geographic administrative boundaries. This would also tie into efforts like GeoWebDNS (http://wiki.open311.org/GeoWeb_DNS/) or LoST (https://tools.ietf.org/html/rfc6451) which is part of the federated infrastructure for things like Next Generation 911 that the FCC is working on. 

Philip Ashlock
Chief Architect, Data.gov
U.S. General Services Administration

[Alex Glaros]

great group; lots of smart effort towards connectivity. 

thanks to all and to Philp for posting your vision of where things are headed

 

I'll try to finish and make public, relational tables of political administrative divisions of the world. It's part of an open source app. Table design will be finished in a few months but not sure how long populating will take.  The design has a different purpose from addressing. Instead will support things like government crowdsourcing ideas from citizens.

thanks,

Alex Glaros

[Steven Clift]

Definitely check put misc. Poplus.org components to see what you can leverage or share.

Note that I forwarded this convo here: http://bit.ly/poplusgroup

--

[Steven Clift]

That's check out ... not "put"