Mapping for community organizers

[Eric van Zanten]

Folks:

I just wanted to share a project I've been working on that was inspired by a conversation I had at the recent Google sponsored hack day (Safer Communities?, something like that). 

Basically, after talking to Patrick Barry from LISC Chicago, we decided that it would be pretty neat to come up with a tool that would allow community organizers to analyze the type of crime that is going on in their, usually pretty specific, area of focus. Since a lot of times these areas don't really have any parallel relation to other boundaries (wards, community areas, etc) and can change, the best approach might be to give users the ability to draw their own shapes. So, that's what I attempted with this:

http://www.crimearound.us

Using the tools on the left, you can define a custom area (either an arbitrary polygon or a box) and once you get done drawing it, the app will load all the crime reports from 7-14 days ago. If you want more or to define a different date range, you can use the tools at the bottom of the map. You can also filter out the different types of crime using the checkboxes down there, too. Once the dots are there, you can click them to see info about each report. And once you've defined what you want to look at, you can click the "Get a Report" link at the top which will generate an Excel spreadsheet showing all the detail about the stuff you're looking at.

I went with the Tribune's definition for those three categories (violent, property and quality of life crimes). You can read more about that over here: 

http://crime.chicagotribune.com/chicago/about#crime-type-definition

I'm hoping to document this thing a bit better in the coming days/weeks. I've also got a pretty nifty JSONP endpoint (which means you can use it from a client-side Javascript app as well as a server-side app) on my data set to offer to developers. Ideas I've had as to where to take this include:

1) Offering other types of info (not just crime reports)

2) Offering other output formats for reports (maybe a PDF version?)

3) Allowing for more robust filters

4) Allow users to save and share their shapes

5) Color the dots based on type of crime report

If any of you guys have any suggestions, I'd love to hear them. Again, I've mainly thought of this as a community organizer tool not really something that would interest the general public (although there's no reason to hide it from them). 

Thanks,

Eric 

[Juan-Pablo Velez]

Interesting approach, and nice work! There are definitely tons of directions to go in. 

To pick between them, and to make sure this ends up being useful to community organizers, I have a meta-suggestion - talk to one.

That will let you validate some of your design assumptions (need to draw arbitrary polygons, only showing recent crimes, etc.) and get an immediate sense of what additional features to add. I'm sure LISC would be happy to connect you. A few community organizers come to the open gov hack night (opengovhacknight.eventbrite.com) as well.

Great start,

- Juan-Pablo Velez
312-218-5448
jpv...@gmail.com

--
You received this message because you are subscribed to the Google Groups "Open Government Chicago-land" group.
To unsubscribe from this group and stop receiving emails from it, send an email to OpenGovChicag...@googlegroups.com.
To post to this group, send email to OpenGov...@googlegroups.com.
Visit this group at http://groups.google.com/group/OpenGovChicago?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Eric van Zanten]

Yeah, thanks. Seems kind of obvious. As I mentioned, this did arise from a conversation with Patrick Barry from LISC. I guess I'm just in the "proof of concept" phase and, yeah, next step is to get it into the hands of people who can use it to start making it actually work.

I'll attempt to make it to a hack night (Tuesdays aren't always open for me) to get some face time with some other folks who might be interested in getting their hands on it.

As an interesting side-note, a direction something like this could go in would be to augment the already awesome looking Local Data project and allow for community organizers to take data from that app and stuff it into this one where they could compare it along side other metrics. 

Anyways, thanks for the support. Maybe I'll see you on an upcoming Tuesday.

Eric 

[Derek Eder]

Eric,

This is great work! Drawing arbitrary polygons is the 'Holy Grail' of features for a lot of people, and you just went ahead and made it! Awesome job. I'll reiterate what Juan said and go out and see how people (perhaps starting with LISC) plan to use it. I'd also love to contribute to this at an upcoming hack night.

As far as where you're getting your data from, it looks like you're hitting this URL: http://crime-weather.smartchicagoapps.org/api/crime/ Is this a FOURTH API by Smart Chicago for crime in Chicago (the others being Socrata, Tribune and ClearPath)? It looks like it is whats enabling this killer arbitrary polygon feature (something the others don't have). Is the usage of such an API documented anywhere?

Derek

--
Derek Eder

@derekeder

derekeder.com

derek...@gmail.com

[Juan-Pablo Velez]

Derek, I'm pretty sure that's a MongoDB database Eric is hosting on smart chicago's servers. So he's copying the data into mongo - possibly from one of the APIs - and then dumping it into Mongo where he can do the spatial queries that power the map.

- Juan-Pablo Velez

312-218-5448
jpv...@gmail.com

[Eric van Zanten]

Yeah, I kinda went back and forth about whether or not we needed yet another crime endpoint but I was interested in making this sucker run mainly as a client-side app so I wanted to make it consume a JSONP endpoint. 

Docs are forthcoming. In a nutshell it's kind of a mashup between the way that Django's tastypie uses the Django ORM filters and the operators that Mongo would expect for a query. So a querystring might look something like this:

?callback=[something you specify]&location__geoWithin=[JSON.stringified GeoJSON]&date__lte=[unix timestamp]&date__gte=[another timestamp]

It should work with any logical field filter combo specified at the top of the app.py file over here:

https://github.com/evz/crimeapi 

I haven't really screwed around with anything other than the way that the crimearound.us app uses the filters (date + location + type) but it *should* work.

Yeah, but docs are coming soon.

Eric

[Derek Eder]

Awesome! I'll poke around ;)

[David Eads]

You're welcome to use the Chicago Tribune Crime API (see
http://blog.apps.chicagotribune.com/2013/03/22/announcing-the-chicago-crime-api/).
We're committed to maintaining and expanding this, so we'd be happy to
work with you.

David

On Mon, May 20, 2013 at 9:29 AM, Eric van Zanten

David Eads - 773.354.2285
News applications developer, Chicago Tribune
(http://www.chicagotribune.com/news/data/)
Founder, FreeGeek Chicago (http://freegeekchicago.org)

"I get up every morning determined to both change the world and have
one hell of a good time. Sometimes this makes planning my day
difficult." -- E.B. White

[Pallavi Anderson]

Awesome job, and everything Derek said!

Are you hosting on Amazon?

For the spatial queries, are you using GeoDjango or direct GDAL queries or something else entirely (Cascading / Scala, perhaps)?

How many crime records can you query within a reasonable HTTP response time?

How many GeoJson entities can you display on a map without compromising UI responsiveness (slow panning / zooming)?

I've worked on several experiments to test these limits and am happy to sync up at a hack night or another time (Tuesdays are almost impossible for me) and share notes.

Pallavi

@pkaushik

[Eric van Zanten]

Actually considered that but didn't end up using it for one big reason: I can't give it a GeoJSON blob and get everything within it. Are you thinking of adding that in there?

Eric

[Eric van Zanten]

Amazon: Yes, through the help of the Smart Chicago Collaborative

Backend is pretty much a raw MongoDB interface. 

Haven't done much performance testing but I'd love to see someone break it and help me fix it again :P

I'm hoping to make it to the OpenGov thing on the 23rd. Maybe we can talk then?

Eric

[Pallavi Anderson]

Great, I am planning to attend the OpenGov thing on Thursday! See you then.

Eric

--

You received this message because you are subscribed to the Google Groups "Open Government Chicago-land" group.

To unsubscribe from this group and stop receiving emails from it, send an email to OpenGovChicag...@googlegroups.com.
To post to this group, send email to OpenGov...@googlegroups.com.
Visit this group at http://groups.google.com/group/OpenGovChicago?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
@pkaushik

[Patrick Barry]

Let me add to the applause for this one. Great work, Eric. 

We're excited about this tool at LISC Chicago because several of our neighborhood partners have direct applications for it in connection with their safety work. We will be helping them define their areas of interest (the polygons or rectangles) and their timeframes for comparisons over time. Each neighborhood has different interests so as they get into the work they'll surely have ideas on how to best sift the data (in particular by type of crime).

We'll definitely be interested in the ability to save the polygons and date filtering, which I suppose requires a log-in function. Is there another open tool out there with a log-in and save-search function (like the City data portal) that could be brought into this one? 

We'll also be interested in exporting PDFs or any other method to provide graphic views of the data/map.

Eric

--

You received this message because you are subscribed to the Google Groups "Open Government Chicago-land" group.

To unsubscribe from this group and stop receiving emails from it, send an email to OpenGovChicag...@googlegroups.com.
To post to this group, send email to OpenGov...@googlegroups.com.
Visit this group at http://groups.google.com/group/OpenGovChicago?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Patrick Barry
6743 N. Newgard Ave.
Chicago, IL 60626
773-973-2169 primary
773-484-7954 cell
773 751-5512 fax

[Greg Sanders]

Amazing stuff. Once again this shows how much value follows from the release of granular data by a large jurisdiction like the City, and how much talent and energy are catalyzed when data flows freely.  Of course this has me pondering how we'll someday (soon I hope) live in a region where granular raw data is readily available from the remaining 283 municipalities...  Excellent work. Thanks  --Greg Sanders

[Eric van Zanten]

Pallavi was asking about how MongoDB does its geospatial indexing at the hack night this evening so I thought I'd post this back to the group in case anyone else is interested in the technical details there:

http://blog.mongodb.org/post/50984169045/new-geo-features-in-mongodb-2-4

[Eric van Zanten]

Folks:
In case anyone is interested, I made a few updates to the crimearound.us tool earlier today. There's now a basic address search and I've made a couple other cosmetic changes.

More to come soon.

Eric

[Juan-Pablo Velez]

Looks great. How are you doing the geocoding?

- Juan-Pablo Velez

312-218-5448
jpv...@gmail.com

[Eric van Zanten]

Mapquest

[Randy Baxley]

I drew a shape and get no report.

I entered 200 W State and get no report.