Little update re etl nyc parsers progress

1 view
Skip to first unread message

Tim Waters

unread,
Nov 30, 2012, 9:29:22 AM11/30/12
to nypl-ga...@googlegroups.com
Hi folks, just a little update with in particular about the ETL parsers

We are setting up a dedicated box for elastic search, all set up and
configured much better, than just with out of the box settings.

Parsers done, committed.
Datasets uploaded to server
JSON dumps ready for ES import on server
New: addresses are saved as objects in Place doc - see place mapping in code

Some docs http://topomancy.com/support/projects/gazetteer/wiki/NyplShapefilesTodo.

First step: add one of the buildings datasets (buildings or
buildings_b or the FinalCleaned one).

Tim

todo:

OSM for NYC (incl. OSM with buildings, if named or addressed)

1.separate ES index (and data dirs) from LC for new nypl and nypl dev
2.define index names for nypl instances
3.generate history from dumps
4.import data
5.import history

David Riordan

unread,
Dec 1, 2012, 2:25:54 PM12/1/12
to Tim Waters, nypl-ga...@googlegroups.com
This is incredibly exciting! Do you have it running anywhere we can access?

-Dave

David Riordan | Product Manager, NYPL Labs | @NYPL_Labs



--



Reply all
Reply to author
Forward
0 new messages