Hack Day Brainstorming

[Jim Klucar]

Just a thread so people can post an intro and whatever they might
think of doing at hack day. Spawn new threads if replies get focused
on a topic.

So to start it off, I'm a developer with a wide background. I have
written a lot of code, but mostly low level stuff for hardware and
high-performance computing. Recently I've been doing higher level
application software, and started to tinker with GWT and I thought
this would be a good opportunity to do something useful while
learning. I also have a degree in Applied Math, so data analysis is
right up my alley.

As for the day's events, David's LinkedIn graph looks interesting, but
I don't know what information he's trying to glean from it. I have
done some graph theory, so I could help implementing algorithms to
find sub-graphs of interest, anomalous nodes, path finding, etc.

The snow plow idea is also interesting. I wonder if the Socrata back-
end could allow a plow driver's phone to push coords from a phone app
(with authentication) into the database, then we could update the
graph quickly during an event.

I was thinking of building a geo-index of the data tables that would
allow an efficient search for entries occurring within an arbitrary
rectangle on a map. I think this may be too back-end database work
though. If we had it though, building an app that pulled in data as
you walked around the city would be relatively easy.

[Jonathan Julian]

> I was thinking of building a geo-index of the data tables that would
> allow an efficient search for entries occurring within an arbitrary
> rectangle on a map. I think this may be too back-end database work
> though.  If we had it though, building an app that pulled in data as
> you walked around the city would be relatively easy.

I think we can do this with the PostGIS extension to Postgres. Ed
Schmalzle and I were planning on pulling the data (parking citations
to start) into a Postgres db to allow for location-awareness. We could
then build a public RESTful api to expose nearby data points given a
user's coordinates. Is this a worthwhile effort?

> --
> Event Info at: http://CivicHackDay.org
>
> You received this message because you are subscribed to the "Civic Hack Day" Google group.
> To post to this group, send email to civic-h...@googlegroups.com
> To unsubscribe from this group, send email to
> civic-hack-da...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/civic-hack-day?hl=en?hl=en
>

[Mark Headd]

Also have a look at GeoCouch - the Geo-enabled fork of CouchDB:

https://github.com/vmx/couchdb

I used it as the basis for this small project that uses shapefile data from Baltimore:

http://www.voiceingov.org/blog/?p=2234

Mark 

[Jim Klucar]

Sure. What I don't like about this though is that you have to re-host
the data, so inevitably it will become stale compared to the city's
site. If I were to re-host it, I'd be tempted to do it with Hadoop's
HBase. http://hbase.apache.org/

On Feb 9, 11:50 pm, Jonathan Julian <jonathan.m.jul...@gmail.com>
wrote:

[Jonathan Julian]

Re-hosting (I'm thinking of it as warehousing) feels like a necessity
at the moment, to allow for relational and location-aware queries.
Also querying across datasets, like show me 311 calls, crime, parking
tickets, and real estate all at once for a particular location.
Hopefully on Saturday we can uncover solutions for these types of
problems. Maybe they are temporary.

And I'm not worried about stale data right now - I can refresh it as
fast as it can be downloaded.

On Feb 10, 8:15 am, Jim Klucar <klu...@gmail.com> wrote:
> Sure. What I don't like about this though is that you have to re-host
> the data, so inevitably it will become stale compared to the city's
> site.  If I were to re-host it, I'd be tempted to do it with Hadoop's

> HBase.http://hbase.apache.org/

[Mark Headd]

Looks like a bunch of geo data sets were added today, providing information on the location of all sort of city assets and other facilities:

http://discuss.baltimorecity.gov/topic/more-locational-data-sets-in-openbaltimore

If anyone wants a walk through of how to populate a GeoCouch instance with shapefile data, let me know.  It's super simple and puts the data in a really easy to use format.

Props to the City for getting this data in there so quickly - this type of data is great for building all sorts of mobile apps.

See you all tomorrow.

Mark

[Chris Metcalf]

You can actually do some simple geo queries from right within the
Socrata API. It's not as sexy as what you can do with GeoCouch but it
does the job in a pinch and it doesn't require re-loading the data.

I've added a rough writeup of the "within_circle" operator to our dev
site, in the "querying datasets" section (scroll to the bottom):

http://dev.socrata.com/querying-datasets

I'll provide a code sample for people to play with tomorrow as well.

Thanks,

Chris Metcalf
Technical Program Manager /
Developer Evangelist

chris....@socrata.com
(206) 340-8008 x114 (office)
(734) 276-1100 (cell)
http://www.socrata.com

On Feb 11, 5:19 pm, Mark Headd <mhe...@voxeo.com> wrote:
> Looks like a bunch of geo data sets were added today, providing information on the location of all sort of city assets and other facilities:
>

> http://discuss.baltimorecity.gov/topic/more-locational-data-sets-in-o...