Mumbai Slum Clusters Map from SRA

[Arun Ganesh]

Just rectified the PDF map of slum clusters from the Slum Rehabilitation Authority of Mumbai:  View map

Data Source: SRA Mumbai 2016 -> MapWarper -> Mapbox

The map also includes Mumbai village boundaries if anyone wants to digitize them. The underlying GIS data looks quite good and aligns quite well with satellite imagery. Wish they would release the shapefiles.

[Sarath Guttikunda]

Good afternoon, Arun.

Is there is a similar map for the Greater Delhi region - slums/jugees?

With regards,

Sarath

--
Dr. Sarath Guttikunda
http://www.urbanemissions.info

http://www.indiaairquality.info

--
Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org
---
You received this message because you are subscribed to the Google Groups "datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Arun Ganesh]

On Wed, Nov 22, 2017 at 3:54 PM, Sarath Guttikunda <sgutt...@gmail.com> wrote:

Good afternoon, Arun.

Is there is a similar map for the Greater Delhi region - slums/jugees?

Not that I know of. If anyone can find a raster/pdf I can help rectify and load it onto a map just like this. 

[Nikhil VJ]

Hi Arun,

Great work!

Here's a tool you can use to fade this over different backgrounds, including satellite imagery:

https://answerquest.github.io/overlap-custom.html?url=http://mapwarper.net/maps/tile/25733/{z}/{x}/{y}.png

You can load any mapwarper url or mapbox tileset over it by changing the permalink, or pasting the schema at bottom.

https://answerquest.github.io/overlap-custom.html

See the code here

--
Cheers,
Nikhil VJ
+91-966-583-1250
Pune, India
Website <http://nikhilvj.cu.cc>

DataMeet Pune chapter <https://datameet-pune.github.io/>
Self-designed learner at Swaraj University <http://www.swarajuniversity.org>

Contribute <https://www.instamojo.com/@nikhilvj/>

[Nikhil VJ]

Correction : You can see the mumbai slums map by just passing the mapwarper id:

https://answerquest.github.io/overlap-custom.html?mapwarper=25733

--
Cheers,
Nikhil VJ
+91-966-583-1250
Pune, India
Website <http://nikhilvj.cu.cc>

DataMeet Pune chapter <https://datameet-pune.github.io/>
Self-designed learner at Swaraj University <http://www.swarajuniversity.org>

Contribute <https://www.instamojo.com/@nikhilvj/>

[Arun Ganesh]

Supercool Nikhil, this is really handy for mapwarper maps. Something to think about is using GL maps for the tool, this will allow inserting the overlay in between the basemap layers, so you could have the roads+labels display over the map. 

Created a ticket for this, for maybe one of my weekend hacks.

[nishadh]

@Sarath, this link gives link for map of JJ (Jhuggie Jhompri Squatter Settlements/Clusters) clusters in Delhi, apparently the map is hosted in google earth engine which is currently non functional and data lost unrecoverable.

@Arun, the pdf map is having vector details embedded in it, vectors from pdf file can be retrieved as follows, which would avoid time consuming (ray tracing based) digitisation of pdf maps, however the resultant vector have to be geo referenced further.

1. The pdf can be converted into scalable vector graphics(svg) by using inkscape command line interface, as the size and complexity of vectors in the the pdf file, this step is time consuming and resultant file size goes upto 80 MB

   inkscape thepdfmap.pdf --export-plain-svg=thepdfmap.svg
   

2. The resultant svg file can be opened in inkscape GUI. Using ungroup (use keyboard shortcut- Shift+Ctrl+G) facility of Inkscape the various layers in the svg can be removed, here the layers would be the map layout as one layer, map legends as one layer, then the map boundary as another layers etc
3. To get individual layer (such as slum polygons), other layers can be deleted by selecting those layers. Save the resultant single layer as file format Desktop Cutting Plotter(Autocad DXF) in Inkscape

4. The DXF file can be opened in QGIS and vector can be saved as shape file or Geojson(slumPdf.geojson), the vector file from DXF in QGIS is polyline and it can be converted into polygon as follows

   from shapely.geometry import Polygon
   import numpy as np
   import geopandas as gp
   import pandas as pd
   ee=gp.read_file('slumPdf.geojson')
   ee['lineNo']=np.arange(0,len(ee.index))
   xx=[]
   yy=[]
   lineno=[]
   for index, row in ee.iterrows():
        for pt in list(row['geometry'].coords):
             xx.append(pt[0])
             yy.append(pt[1])
             lineno.append(row['lineNo'])
   db=pd.DataFrame()
   db['xx']=xx
   db['yy']=yy
   db['lineno']=lineno
   

5. The above code reads the geojson file and collect x and y coordinates of all vectors’ line strings, I guess this step is the right place to apply geo referencing of vectors

   db1=db.drop_duplicates('lineno')
   linenos=db1.lineno.tolist()
   linenos1=[]
   geoslist2=[]
   for lineno in linenos:
        db2=db[db.lineno==lineno]
        if len(db2.index)>=3:
               db2['geos'] = db2[['xx', 'yy']].apply(tuple, axis=1)
               geoslist=db2.geos.tolist()
               geoslist1=Polygon(geoslist)
               geoslist2.append(geoslist1)
               linenos1.append(lineno)
               #print geoslist
        else:
               #print len(db2.index)
               pass 
   
   sd=pd.DataFrame()
   sd['polyno']=linenos1
   sd['geometry']=geoslist2
   dfa = gp.GeoDataFrame(sd, columns=['geometry','polyno'], index=sd.index)
   dfa.to_file('slumPdfPoly.geojson',driver='GeoJSON')
   

6. The above code convert, individual coordinates into polygon, here the line’s having only two coordinates sets are omitted as it can’t be converted into polygon

7. The resultant polygon have to be georeferned further, here with attaching the line geojson file collected from pdf file for further experimentation.
8. Requesting your ideas for geo referencing vectors and also there is some issue with generated polygon in its geometry.

​

[nishadh]

Using Affine transformations, vectors can be georectified as explained here. It requisites few modification from the above codes. The control points are collected from MapWarper, under its export tab, provided kml file was used. The kml is opened in google earth and the outer most outline of the pdf map was used and GCPs are taken from it

mx=[72.763313,73.000312,72.995965,72.758095]
my=[18.885526,18.888553,19.279235,19.276757]

1. The geojson from the pdf file was saved with two layers one for slum vectors and GeoLyr vector which is the map outline box present in the pdf file.

   import numpy as np
   from skimage.transform import AffineTransform
   import math
   import geopandas as gp
   mx=[72.763313,73.000312,72.995965,72.758095]
   my=[18.885526,18.888553,19.279235,19.276757]
   dest=[[mx[0],my[0]],[mx[1],my[1]],[mx[2],my[2]],[mx[3],my[3]]]
   ee=gp.read_file('slumPdf.geojson')
   ee1=ee[ee['Layer']=='GeoLyr']
   xx=[]
   yy=[]
   lineno=[]
   for index, row in ee1.iterrows():
     for pt in list(row['geometry'].coords):
            xx.append(pt[0])
            yy.append(pt[1])
   source1=[[xx[0],yy[0]],[xx[2],yy[0]],[xx[2],yy[1]],[xx[0],yy[1]]]
   source1=np.array(source1)
   dest = np.array(dest)
   aft3 = AffineTransform()
   aft3.estimate(source1, dest)
   

2. Apply affine transformation on the collected pixelx and pixely from the gejson, first line removes the outerline layer which is used for geo referencing

   eegof=ee[ee['Layer']!='GeoLyr']
   eegof['lineNo']=np.arange(0,len(eegof.index))
   xx=[]
   yy=[]
   lineno=[]
   layer=[]
   for index, row in eegof.iterrows():
     for pt in list(row['geometry'].coords):
          xx.append(pt[0])
          yy.append(pt[1])
          lineno.append(row['lineNo'])
          layer.append(row['Layer'])
   db=pd.DataFrame()
   db['xx']=xx
   db['yy']=yy
   db['lineno']=lineno
   db['layer']=layer
   a0=aft3.params[0,0]
   a1=aft3.params[0,1]
   a2=aft3.params[0,2]
   b0=aft3.params[1,0]
   b1=aft3.params[1,1]
   b2=aft3.params[1,2]
   db['longitude']=(a0*db['xx'].astype(float))+(a1*db['yy'].astype(float))+a2
   db['latitude']=(b0*db['xx'].astype(float))+(b1*db['yy'].astype(float))+b2
   

3. Then convert the polylines into polygons as follows

   db1=db.drop_duplicates('lineno')
   linenos=db1.lineno.tolist()
   linenos1=[]
   geoslist2=[]
   for lineno in linenos:
     db2=db[db.lineno==lineno]
     if len(db2.index)>=3: 
           db2['geos'] = db2[['longitude', 'latitude']].apply(tuple, axis=1)
           geoslist=db2.geos.tolist()
           geoslist1=Polygon(geoslist)
           geoslist2.append(geoslist1)
           linenos1.append(lineno)
     #print geoslist
     else:
          #print len(db2.index)
          pass 
   sd=pd.DataFrame()
   sd['polyno']=linenos1
   sd['geometry']=geoslist2
   dfa = gp.GeoDataFrame(sd, columns=['geometry','polyno'], index=sd.index)
   dfa.to_file('slumClusters.geojson',driver='GeoJSON')
   

4. The resultant slumCluster geojson file is uploaded in https://github.com/nishadhka/Municipal_Spatial_Data/tree/master/Mumbai

​

[Arun Ganesh]

Nishadh, this is excellent! Super useful technique to georeference any vector map pdfs, like the masterplans for Bengaluru.

Have updated the Slum map with these outlines (in red): Map | Data

The output seems really good and matches the satellite image almost perfectly. I did have to translate it by a few pixels, but the scale is perfect.

[nishadh]

Thanks, There is issues in polygon generation, missing to cut the polygon in small lines, please see the attached pdf map area and its geojson output. Has to rectify it, regarding the georefrencing, it is based on the MapWarper coordinates, improvement can be looked into affine transformation method and base pdf map referencing.   

There is dxf reading library in Python and using that the whole process can be made automatic, in the case of Bengaluru master plan, it is very much possible as it is from same source. Will discuss about it in that thread.