Karnataka Polling Station Boundaries scraped from KSRSAC

[Vivek Matthew]

Hello folks,

I had scraped the polling station boundaries for Karnataka from the KSRSAC site (https://kgis.ksrsac.in/pollinginfo/). The data is up on this GitHub repo: https://github.com/Vonter/india-polling-stations

There are a total of 60366 polling stations. On the repo there is a Karnataka.geojson with the boundaries for all the 60366 polling stations but it is a huge 156MB file. So I have also divided it into separate GeoJSONs for each of the 224 Assembly Constituencies (~1MB each) and each of the 31 districts (~6MB each) in Karnataka.

I'll soon be adding Parliamentary Constituency level GeoJSONs. Converting the GeoJSONs into other formats is also in the pipeline. If anyone knows of other states that have similar sites which could be scraped to generate polling station boundary GeoJSONs, I would be happy to try and do so.

When I was searching for Karnataka polling station boundary geodata a few days back (out of interest in more granular population density geodata) I didn't find it anywhere online. So I hope that someone else out there searching for it will find this data useful now that it has been scraped from KSRSAC.

Regards,
Vivek

[Nikhil VJ]

Hi Vivek, 

Great work!

it looks like the big files are all in Github LFS

Example: content of Karnataka.shp : 

version https://git-lfs.github.com/spec/v1
oid sha256:e32b9cfaa855993cabba3de1f860b248cac98cb53b1a5bf50b8913dda33ad044
size 106013972

Can you tell how to download that, and add the same in the repo's readme ?

--
Cheers,
Nikhil VJ
https://nikhilvj.co.in

--
Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAH8O8LdHebEeQcRWLjXf6t2UK9JHAZ10yR8O_MseD9ji5kygYw%40mail.gmail.com.

[sreeram kandimalla]

Github releases supports exporting 2GB files and supports http range requests. Maybe convert the data into pmtiles or flatgeobuf and serve directly from there?

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAH7jeuO-Y5_rFumEqTCbgqm3pNoWVH%3Dv7oeZQ-ssL_2efMbG6w%40mail.gmail.com.

[Vivek Matthew]

Hi Nikhil,

I've moved the large files out of Git LFS and stored them as compressed .zip files instead. This was my first time using Git LFS and I didn't realize that it doesn't work in a straightforward way, apparently GitHub even charges for LFS bandwidth usage in excess of 1 GB [1]. So I have also removed the LFS files from the Git history, if anyone has pulled those files, you may need to re-clone the repo again. The README.md has been updated with details regarding the compressed files.

Also thanks for pointing this out! I would not have realized this problem otherwise.

Regards,

Vivek

[1] - https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-storage-and-bandwidth-usage#tracking-storage-and-bandwidth-use

[Anil kumar]

HI, Vivek

Can u make a video or share the details of how to scrape the boundaries (https://kgis.ksrsac.in/pollinginfo/) form web.

--

Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAH8O8LdHebEeQcRWLjXf6t2UK9JHAZ10yR8O_MseD9ji5kygYw%40mail.gmail.com.

--

With regards,

Anilkumar y.s

"Be happy only happy"

[Vivek Matthew]

Thanks for the suggestion Sreeram. I have created a GitHub release with the Karnataka data in PMTiles and FlatGeobuf formats: https://github.com/Vonter/india-polling-stations/releases/tag/karnataka

I don't have much experience with GIS so apologies if these files don't work as expected, but I was able to load the FlatGeobuf file into QGIS successfully. Not too sure how to test if the PMTiles file is valid, but I think it should be correct too.

Regards,

Vivek

[sreeram kandimalla]

You did better than I have.. I haven't tested any of the ideas yet.. but my research suggested that this is the best way to share geospatial data. I will try to build a webpage to show the advantages of what you just did when I find time. Pmtiles doesn't yet have support in GDAL yet.. so most existing tools won't work with it yet.. it is basically mbtiles done better and cloud native by default. The author is still cleaning up the specification. 

And side note to people sharing Indian geospatial data on GitHub.. truncate the coordinates to 5 decimal points as the data storage is not in India. 

To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/91ddc04f-6302-4136-a0f8-553750851705n%40googlegroups.com.