WNY OpenData Running on Infochimps Platform
3 views
Skip to first unread message
Dan Magnuszewski
Apr 1, 2011, 2:31:22 PM4/1/11
to wnyop...@googlegroups.com
All,
I had a call with Jacob Perkins, an engineer at Infochimps, this morning to discuss how we can leverage their platform and got into some technical detail on how we're going to proceed. First off, here's their GitHub repo for their Hackboxen:
https://github.com/infochimps/hackboxen-public
The framework is built in Ruby and uses Rakefiles with a specific directory structure so that it can be easily built and incorporated into their production system. The nice part is that we can write the ingestors, within the hackbox engine, in any language we want - which is cool, because I have a bunch of Perl already written.
Moving forward, I'm going to get a proof of concept written (in Perl, obviously) and see how things go. We're the first group to take on this type of partnership, so we'll be figuring out some of the good and bad parts of process along the way. They're still not quite there for automating APIs for all data, so we'll initially just start populating as much data as possible. Hopefully by this point they'll be ready to automate our APIs. Depending on their timeline, we'll wait for them or possibly help them write the APIs for our stuff, but for now, we'll focus on consolidating the data into sane formats.
...more details to come over the next week or so.
-Dan
I had a call with Jacob Perkins, an engineer at Infochimps, this morning to discuss how we can leverage their platform and got into some technical detail on how we're going to proceed. First off, here's their GitHub repo for their Hackboxen:
https://github.com/infochimps/hackboxen-public
The framework is built in Ruby and uses Rakefiles with a specific directory structure so that it can be easily built and incorporated into their production system. The nice part is that we can write the ingestors, within the hackbox engine, in any language we want - which is cool, because I have a bunch of Perl already written.
Moving forward, I'm going to get a proof of concept written (in Perl, obviously) and see how things go. We're the first group to take on this type of partnership, so we'll be figuring out some of the good and bad parts of process along the way. They're still not quite there for automating APIs for all data, so we'll initially just start populating as much data as possible. Hopefully by this point they'll be ready to automate our APIs. Depending on their timeline, we'll wait for them or possibly help them write the APIs for our stuff, but for now, we'll focus on consolidating the data into sane formats.
...more details to come over the next week or so.
-Dan
Reply all
Reply to author
Forward
0 new messages