Awesome. How did you do this? It's pretty darn cool = the pincode sub-site I mean.
http://pincode.net.in/ has 152,000 pin codes, and we could use
wikiscraper or a custom program to get them.
Then use Google Geocoder which has a simple API, just pass
sensor=false&address=[city], India [pincode]. You'll most likely get a
few results, wtih lat/lng, put that into the database, let the crowd
come in and choose the PO directly or choose of of them options.
One way to handle overlaps is a) announce it and b) create an api forpeople to use (say GetGeoAddress given PinCode). If you know of any of
the overlaps, why don't we list 'em and let's contact everyone and get
the pieces flowing.
Btw, I couldn't be on the skype meet. I'm trying to do stuff to
visualize the budget, am based in Gurgaon, write on financial stuff
and economics in India at www.capitalmind.in, was a programmer in a
past life and am getting back to it, and I constantly work at
visualizing financial data on the blog and privately for investing
ideas. (One of the pieces I'd done was
http://blog.investraction.com/2010/02/bharti-zain-deal-analysis.html -
but that's more or less static.)
>> Awesome. How did you do this? It's pretty darn cool = the pincode sub-site
>> I mean.
>
> Python, AppEngine, Google Maps API. The source is
> at https://github.com/sanand0/pincode
Absolute coolness. I am learning python, and used version control in
the zamaana where SVN was a relief from Visual Sourcesafe, so please
excuse my ultra-newness to the concept.
>> http://pincode.net.in/ has 152,000 pin codes, and we could use
>> wikiscraper or a custom program to get them.
>
> Actually, this is a list of 152,000 POST OFFICES, not PIN Codes. There are
> multiple post offices per PIN code. But you're right, we should be able to
> get this.
True, even if we got one post office per pin code, and geo tagged it
property it's way more than anything else available! Pincodes are
effectively areas, and I wish we would get a lat/lng polygon but
that's expecting WAAAAY too much from the postal dept right now.
One interesting way to get lat-lang coordinates is from cell phone
towers or have a phone app to directly post that info back into
pincode.datameet.org, for GPS enabled phones. ("What's the Pincode of
your current location" [send]) But it may be overkill.
>> Then use Google Geocoder which has a simple API, just pass
>> sensor=false&address=[city], India [pincode]. You'll most likely get a
>> few results, wtih lat/lng, put that into the database, let the crowd
>> come in and choose the PO directly or choose of of them options.
>
> Good point. But re-using the Google Geocoder data will run into licensing
> problems. I'd rather people did this manually, or we used OpenStreetMap like
> Arun suggested.
Really? Even if we used just the latitude/longitude portions? Quite
interesting.
>> One way to handle overlaps is a) announce it and b) create an api for
>> people to use (say GetGeoAddress given PinCode). If you know of any of
>> the overlaps, why don't we list 'em and let's contact everyone and get
>> the pieces flowing.
>
> Don't know who else is doing it. But fair point on the API. Want to give it
> a shot? Just fork the code and have a play :-)
Will do. Will check out the code too.
> Neat stuff! Tableau is pretty good. In fact, was thinking of having a few
> tutorials in the next datameet. Would you mind giving a 15 minute tutorial
> on using Tableau Public?
Sure - happy to. Whenver we decide on the meet, I need to get a small
dataset together on the concept. It's quite simple, or I haven't used
it enough :)
On a slightly different note: I want to make this for India.
http://www.nytimes.com/interactive/2008/05/03/business/20080403_SPENDING_GRAPHIC.html
I have all the data for India, I know how it's organized etc. But I
have no idea how to start creating a flash (or even Java) based
Voronoi treemap, especially not something that's mapped back into a
circle. Anyone know? Want to collaborate?
Am trying to get into born-again programming, but here's JS source for
on-the-fly voronoi calculations.
http://www.raymondhill.net/voronoi/rhill-voronoi-demo2.php. If you
really like this kinda stuff, you shoudl also know about CGAL.org -
that stuff is a little overhead zone for me at the moment.
Cheers,
Deepak Shenoy
Company: http://www.marketvision.in
Blog: http://www.capitalmind.in
Twitter: @deepakshenoy
On a slightly different note: I want to make this for India.
http://www.nytimes.com/interactive/2008/05/03/business/20080403_SPENDING_GRAPHIC.html
I have all the data for India, I know how it's organized etc. But I
have no idea how to start creating a flash (or even Java) based
Voronoi treemap, especially not something that's mapped back into a
circle. Anyone know? Want to collaborate?
- The pincode displayed in webpage should be editable for entering new pincode. Right now I can do it only through the URL!
- Counters - Total Pincode available, Total pincode mapped, Total pincode pending, 10 Randomly picked pincode that needs inputs.
- Counters - Total Pincode available, Total pincode mapped, Total pincode pending, 10 Randomly picked pincode that needs inputs.
We don't have the full list of PIN codes at the moment, which is part of the problem. But yes, once we have that, this is a definite to-do.
--
-----------------------------------------------------------------------------------
http://blog.dhananjaynene.com twitter: @dnene
I missed the skype session and am curious what are you guys using this
pincode data for?
--
On Mon, Mar 7, 2011 at 4:28 PM, Arun Ganesh <arung...@gmail.com> wrote:
> Attached a visualization sample of geocoded placenames with pincodes
> currently in osm. AFAIK this would be the best free datasource currently
> available.
>
> -Arun
> On Mon, Mar 7, 2011 at 1:33 PM, Ananth Mani
> <ananthar...@reportbee.com> wrote:
>>>
>>>
>>>>
>>>> Counters - Total Pincode available, Total pincode mapped, Total pincode
>>>> pending, 10 Randomly picked pincode that needs inputs.
>>>
>>> We don't have the full list of PIN codes at the moment, which is part of
>>> the problem. But yes, once we have that, this is a definite to-do.
>>
>> Idea of counter is to reflect the current status (like progress) rather
>> than completed status.
>> ~ Ananth
>
>
> --
> j.mp/ArunGanesh
>
-----------------------------------------------------------------------------------
http://blog.dhananjaynene.com twitter: @dnene
Really? Even if we used just the latitude/longitude portions? Quite
interesting.
I've pulled the data out in a format compatible with the AppEngine bulkuploader (osm.csv).Comparing that with the GeoNames data, we have 2,790 PIN codes in OSM that are new, and I'll import them today. (Arun, this data is CC-SA compatible, right?)We also 2002 common PIN codes. When I compare the two, some them have significant differences (over 1,000km in some cases). I'm attaching the differences (diff-in-data.csv).Arun (or anyone else), any guidance on why we have these differences? Which one do you think is more accurate?
RegardsAnandOn Mon, Mar 7, 2011 at 7:54 PM, Arun Ganesh <arung...@gmail.com> wrote:
Attached is the osm xml data of all Indian placenames extracted from the openstreetmap database. I recommend exploring the data and attribute tags using josm. For visualizations like what I created you can play with maperitive. Be warned that the data may have inconsistent tags due to various users contributing data. It can be corrected and updated on the osm server using josm (but be very careful and make sure you know what you are doing).The data of interest, if you look into the .osm file is the postal_code tag. I have extracted all nodes with a place attribute, and not all of them have the pincode data.References:http://wiki.openstreetmap.org/wiki/Map_Features - Common osm tag valueshttp://wiki.openstreetmap.org/wiki/.osm - osm xml format documentationhttp://wiki.openstreetmap.org/wiki/Osmosis - command line utility to process .osm data--
j.mp/ArunGanesh
Just a point here - the Datameet data set is CC-BY licensed and this
AND is CC-BY-SA licensed. They don't play nicely - you can include AND
in the Datameet set only if the Datameet set is CC-BY-SA licensed too.
Only reason I picked CC-BY is because http://www.geonames.org/ is CC-BY and that's where my initial dataset is from. I can convert that to a CC-BY-SA, right?Am tempted not to use CC-BY-SA. Attribution is fine, but forcing others to share alike... commercial use would be restricted.Arun, any chance OSM can be used without ShareAlike?
The one way around this is to keep the OSM data set independent of
everything else, display it independently and return changes to OSM as
mandated by the SA license. The other option, is to ask AND if they
can independently license it to data meet under a vanilla BY license.
--
Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to a topic in the Google Groups "datameet" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/21uKP0AzrjY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to datameet+unsubscribe@googlegroups.com.