[Mobilizing Historic Geodata] - Project Tracks, Project Ideas, and Sweet, Sweet Data Starter Kits

14 views

Skip to first unread message

David Riordan

unread,

May 30, 2013, 1:07:46 PM5/30/13

to nypl-ga...@googlegroups.com

MOBILIZING HISTORIC GEODATA: HACK NYC’S PAST WITH NYPL LABS

Project Tracks, Project Ideas, and Sweet, Sweet Data Starter Kits

For the past few years, with a lot of help from the NEH, we’ve been working on a historical gazetteer of New York, called the New York City Chronology of Place, built by the brilliant geo-hackers at Topomancy and derived in part from NYPL’s historical map collection. Topomancy’s been working on a similar tool with the Library of Congress, but while their system is geared around institutional metadata creation and remediation, ours has a public focus. In that spirit, we invite you to dig into New York City’s past with help from our APIs.

We have a few ideas of areas we could focus on:

Applications: Building mind-blowing user-facing proofs of concepts and prototypes of what can be done with the historical geodata and services we’ve got today, while showing kind of things we’ll need to build and data sources we’ll have to provide in the future (to view the past like we view today... whoaaaaa).

Making geo-data out of other data: Like data? Good. Love data? We love you! We’ve got various historic geodata sets and things that could potentially be geodata. Building tools to look for named entities and geoparse texts, working with semi structured and unstructured data. Metadata warriors, arise!

Geo-integration and visualization: Wanna run batch queries to find places that no longer exist in thousands of metadata records or plot amazing data you’ve got? Wanna build an urban-scale GutenKarte? You’re in the right place!

Project Ideas

These are just ours - you should discuss, criticize, and most importantly suggest your own. We also want your help figuring out which of these ideas we should pursue and which we should jettison.

Historical Time Travel Receipt (HTTR)

Think Historical Foursquare. Check in anywhere in Manhattan, get back the history of the place.

This idea came from a conversation* investigating historical checkin and how could it be done with the georectified maps from NYPL’s collection... The very simple explanation that HTTR is a basic mobile app that allows a user to check in at CONTEMPORARY locations and have detailed OLD maps of those places sent to a predetermined email... thus not overtaxing a tileserver in the event it’s popular.

We also have grand delusions that we could do more. We’d love to not only have it pull in maps, but also data from other sources so that it not only collates a series of historical maps of the locations you visit and check in to, but, based on those checkin location, this application queries various APIs (e.g. NYPL Repository, National Digital Newspaper Project, NYTimes or you name it), and sends you a series of digital objects we think might be related to your checkin/journey.

Historical Newspaper Geosearch

We’ve spent lots of time scraping data from old maps and integrating it into the New York City Chronology of Place, so we want to put it to good use. One way to do that is to use the NYCcop API to facilitate search of the newspaper APIs we love. We want to make all the old newspapers searchable by the click of a map, so we can see all the articles, for example on the construction of the Croton Reservoir or the building and conflagration that took down the Crystal Palace where Bryant Park now sits.

Old Map Scraper (Polygon extraction)

One of the holy grails of working with old maps is scraping meaningful (i.e. not garbage) data from maps analogous to how OCR does it for texts. It doesn’t have to be structured data, though if you can do that, please, by all means, go ahead. The basic idea is we’ve got a georectified map, we scrape points, lines or polygons and spit them out for use elsewhere.

Mike Resig, brother of John of the House of jQuery, has provided us with a set of instructions and a proof of concept to show us that this is possible and our own Mauricio Giraldo’s been doing some work toward this.

City Directory Geo Visualization

We’ve got a great set of 65k fully addressed (with lots more attributes) building footprints that we hand transcribed (!) from William Perris’ Maps of the City of New York (Manhattan) from 1852-4. We’ve also scanned a city directory published by John Doggett in 1854. We’ve already OCR’d it and done a fair deal of parsing, so it’s ready to zip up with the Perris data.

Historical Geoparsing

A predicate for conducting geoparsing is a gazetteer. Run a body of unstructured texts against a list of places with associated Lat/Long and have returned locations within texts. Historical geoparsing adds the temporal element. Since we’ve built the NYCcop, we’d like to test drive it against whatever data we can find about NYC history. The dream of the Topomancy team has been to build it to parse The Power Broker.

Historic (chrono|geo)parsing (Stokes Iconography)

You may know the nearly hundred year old Stokes Iconography of Manhattan Island 1609-1898, an incredibly dense, exhaustively researched tome of Manhattan people, history and geography, a six volume hypertext-before-it-existed epic homage to the city. It is so deeply rooted in place and time that every page littered with discarded and forgotten points, lines and polygons from the past. And copies exist in digital format, some with OCR. This thing begs to be creatively re-instantiated and given back to the world, as a mappable, or timelined history, or whatever you might imagine.

Gazetteer + NYPL Repository:

We’ve got this thing full of metadata records (like a million) about things in the library (sadly not books). It’s got an API. The team that built that system and API will be hacking along side us. Let’s geoparse it!

Mapping books:

A few weeks ago, I met this amazing group of folks who were going through their favorite NYC books and geocoding any quote related to a place. Schuyler’s GutenKarte did this for public domain texts a few years ago, but never at urban scale. Let’s tackle urban scale book geocoding.

STARTER DATA KITS

NEWSPAPERS