EveryBlock / OpenBlock type website for DFW

[steevithak]

Does anyone else miss EveryBlock? I'm thinking about starting a project to build a replacement, at least for the Dallas / Fort Worth area and I'd like to gauge the interest level among potential contributors. 

If you aren't familiar with EveryBlock it was a geolocation-based community data aggregator. You would log in and select communities you wanted to follow. For example, I followed Deep Ellum. EveryBlock had a vast set of data sources that it aggregated include local city data. All the data was indexed by location and presented based on the user's selected neighborhood. So every day I could log in and see updates. For example, by following Deep Ellum I would see this type of stuff each day:

flickr photos

new business openings

police reports

yelp reviews of business 

restaurant inspection reports

meetups 

construction permits

etc...

In each case, I'm seeing only the subset of data that happened in my neighborhood in the past day or so. There were both traditional blog-like views as well as map views of the data and you could also just ask for a daily email of the content. It was shut down at the beginning of 2013 for the usual reasons; corporate overlords couldn't figure out how to make money with it. 

The EveryBlock website was designed entirely with open source software and relied on open GIS and map data. The software was called OpenBlock and was released under a free software license prior to EveryBlock's shutdown. Unfortunately, it's pretty crufty and has lots of esoteric requirements (e.g. a specific old version of Python and specific versions of certain libraries) that makes it a real pain to set up. However, there's an Amazon EC2 instance available that's preconfigured, so that's one option for a quick start. The backend has a plugin architecture for "scrapers", which are the modules that go out and retrieve the data. Some scrapers use nice APIs (like Flickr and Open311 APIs) but others are literally screen scrapers. The surviving source code doesn't have any of the DFW scraper modules. I'm trying to get in touch with the authors to see if they could be made available under a free license. If not, we'd have to rebuild the scraper modules from scratch.

In the long term, the entire system would need a major rewrite in order to move it to an up-to-date GNU/Linux distro. OpenBlock is currently implemented in Python (and, as I said, only works on a specific version, not the current 3.x version of Python). Because of the number of APIs it talks to, it would probably make more sense to switch to a language where we can get ready-to-use, supported API libraries like PHP or even PERL. 

It would also take some $$$ to cover the hosting and bandwidth costs. A quick-launch would have to use the Amazon EC2 instance but once we had new-and-improved code, we could run it on cheaper, faster hosting elsewhere. We could probably raise enough to cover a few years of hosting with a Kickstarter or IndieGoGo campaign. Another option would be to establish a 501(c)(3) and try to convince local cities to throw in a few $$ annually.

Here's a list of the type of skills needed to make this project happen:

1. GNU/Linux sysadmin skills

2. Python coders

3. PHP or PERL coders depending how the rewrite plans evolve

4. Open Street Map work

5. DFW GIS data entry (community boundaries, police precincts, zip codes, etc)

6. Identification of data sources in each community

7. HTML/CSS/Javascript and design skills for the UI

So that's it. I'm looking for comments, thoughts, ideas, volunteers, large cash donations, or whatever. :)

-Steve

[ext-randa...@nokia.com]

Steve, I could get on board for this and help any way I can.  Rusty on coding but willing to get back up to speed... I also can cover graphic design and database modeling work.

 

FYI, this could also work with a project I have in mind: a meetup venue finding service.  Something like Yelp + Cvent + more, and focused on free spaces.  I need the same skillset for this project (hubfish.com) but am leaning toward applying for Bizspark and Appcampus.  Those require exclusive Microsoft tech for 6 months, primarily focused on Windows Phone, but I'm betting with some planning we could find synergies...

 

Randall (Randy) Arnold

Developer Ambassador

 

Mobile: +1 817 739 6806

Email: ext-randa...@nokia.com

Twitter: @NokiaDevNorthTX

LinkedIn: http://www.linkedin.com/in/texrat

Nokia US

---

From: data-li...@googlegroups.com [data-li...@googlegroups.com] on behalf of ext steevithak [srain...@ncc.com]
Sent: Friday, July 12, 2013 11:35 AM
To: data-li...@googlegroups.com
Subject: EveryBlock / OpenBlock type website for DFW