CartoDB support for GeoTiff (or ArcInfo Grid ASCII)

[F Lengyel]

Hi,

The CartoDB interface indicates that GEOTiff is supported. I can upload a georeferenced 

GeoTiff file to CartoDB, but the interface reports that the uploaded table isn't georeferenced.

Here are the details:

The GeoTiff file was obtained with gdal_translate: 

$cat fwd.sh

#!/bin/bash

gdal_translate -of GTiff \

        -a_srs EPSG:4326 \

        -a_nodata -9999 \

        ramankutty_cropland2000_frac_5m.asc africa_cropland.tif

gdalinfo shows that the correct WSG1984 projection was used:

$ gdalinfo africa_cropland.tif 

Driver: GTiff/GeoTIFF

Files: africa_cropland.tif

Size is 4320, 2160

Coordinate System is:

GEOGCS["WGS 84",

    DATUM["WGS_1984",

        SPHEROID["WGS 84",6378137,298.257223563,

            AUTHORITY["EPSG","7030"]],

        AUTHORITY["EPSG","6326"]],

    PRIMEM["Greenwich",0],

    UNIT["degree",0.0174532925199433],

    AUTHORITY["EPSG","4326"]]

Origin = (-180.000000000000000,89.989020000000011)

Pixel Size = (0.083328250000000,-0.083328250000000)

Metadata:

  AREA_OR_POINT=Area

Image Structure Metadata:

  INTERLEAVE=BAND

Corner Coordinates:

Upper Left  (-180.0000000,  89.9890200) (180d 0' 0.00"W, 89d59'20.47"N)

Lower Left  (-180.0000000, -90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"S)

Upper Right ( 179.9780400,  89.9890200) (179d58'40.94"E, 89d59'20.47"N)

Lower Right ( 179.9780400, -90.0000000) (179d58'40.94"E, 90d 0' 0.00"S)

Center      (  -0.0109800,  -0.0054900) (  0d 0'39.53"W,  0d 0'19.76"S)

Band 1 Block=4320x1 Type=Float32, ColorInterp=Gray

  Min=0.000 Max=1.000 

  Minimum=0.000, Maximum=1.000, Mean=0.109, StdDev=0.215

  NoData Value=-9999

  Metadata:

    STATISTICS_MAXIMUM=1

    STATISTICS_MEAN=0.10855641062885

    STATISTICS_MINIMUM=0

    STATISTICS_STDDEV=0.21470044269803

Florians-iMac:gdal-asc2gtiff flengyel$ 

To ensure that nothing was lost, the reverse translation was done (values were converted to Float64,

but otherwise the contents were the same:

$cat rev.sh

#!/bin/bash

gdal_translate -of AAIGrid \

        -a_srs EPSG:4326 \

        -a_nodata -9999 \

        africa_cropland.tif africa_cropland.asc

This also produced an xml metadata file:

$ more africa_cropland.asc.aux.xml 

<PAMDataset>

  <PAMRasterBand band="1">

    <ColorInterp>Gray</ColorInterp>

    <Metadata>

      <MDI key="STATISTICS_MAXIMUM">1</MDI>

      <MDI key="STATISTICS_MEAN">0.10855641062885</MDI>

      <MDI key="STATISTICS_MINIMUM">0</MDI>

      <MDI key="STATISTICS_STDDEV">0.21470044269803</MDI>

    </Metadata>

  </PAMRasterBand>

</PAMDataset>

Question: is there a way to get raster data into CartoDB in some form?

I've tried loading the .asc files in QGIS, which worked, and converting those 

to ESRI shapefiles in QGIS, but this produces polygons with none of the 

raster data. Does someone have an explicit procedure for producing a 

vector coverage that might work? Maybe I should write a python program...

Thanks,

FL

[Andrew Hill]

Hi FL,

Thanks for writing. In fact, we should probably remove the file format from the list of supported filles, as we don't technically support tiffs yet. I'll file a ticket for it now. 

In fact, we do allow import of geotiffs, but after they are imported you will have a raster band in your table, but not a valid the_geom. This makes sense, because the_geom is for vectors. The georeferencing warning is being produced because you the_geom is empty, that is all. 

We will keep working to support rasters better in the future, but for now it is pretty basic. You can use SQL and the rast column though to do some fun raster stuff within cartodb still, take a look at the PostGIS 2.0 Raster methods here for some ideas,

http://www.postgis.us/downloads/postgis20_raster_cheatsheet.pdf

Cheers,

a

--
 
---
You received this message because you are subscribed to the Google Groups "cartodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cartodb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Andrew W. Hill, PhD, Senior Scientist 

www.vizzuality.com | @andrewxhill

Map, analyze and build applications with your data

www.cartodb.com

[Florian Lengyel]

For the record, I've written an ArcInfo Grid ASCII to ESRI Shapefile "vectorization" program (code on GitHub--still under development) and used it to produce this map:  http://cdb.io/YdKps7 Perhaps I could have used GRASS in three steps, but I'm still new to GIS.

[Andrew Hill]

Hi Florian,

That's a great map. Within CartoDB, you can also vectorize using the ST_DumpAsPolygons with the rast column you noticed before. So, it would go something like this (from memory, could be typos),

SELECT ST_Multi((ST_DumpAsPolygons(rast, 1)).geom) as the_geom, (ST_DumpAsPolygons(rast, 1)).val as value FROM table_name 

Then with the result, run a Table from query to get what you have now.

see the doc here too,

http://www.postgis.org/documentation/manual-2.0/RT_ST_DumpAsPolygons.html

Cheers,

a

[Florian Lengyel]

Andrew,

Now that the GeoTIFF option is removed from CartoDB, if I am not mistaken, I would not be able to run this query unless I had my own cartodb instance on my own server so that I could have access to postGIS...

Raster datasets, like the Ramankutty cropland map of the last post, make more efficient use of space. So one question is whether I could use within CartoDB the following: instead of multiple layers of oriented vector grid-squares each with a single attribute, a single layer of such polygons with multiple attributes.  I would need programmatic access to the CartoDB.CSS symbology to interactively display the attributes I wanted. I'm suspect though, that this would complicate the Javascript handing of layers (such as the assignment of interactivity...).*

(I should ask my histogram question also in another thread, because if I could have access to PostGIS, I could run its histogram functions instead of running them in a RESTful endpoint service I wrote to provide histograms, because I don't have access...)

FL

*I'm still unseasoned in GIS.

[Andrew Hill]

Hi Florian,

 

Now that the GeoTIFF option is removed from CartoDB, if I am not mistaken, I would not be able to run this query unless I had my own cartodb instance on my own server so that I could have access to postGIS...

No, we just don't support GeoTIFF officially - meaning we can not guarantee that it will work etc. But you can still upload them and manipulate the rast column if you wish to try. 

Raster datasets, like the Ramankutty cropland map of the last post, make more efficient use of space. So one question is whether I could use within CartoDB the following: instead of multiple layers of oriented vector grid-squares each with a single attribute, a single layer of such polygons with multiple attributes.  I would need programmatic access to the CartoDB.CSS symbology to interactively display the attributes I wanted. I'm suspect though, that this would complicate the Javascript handing of layers (such as the assignment of interactivity...).*

I don't really follow this request? Keep the questions coming, sorry if I don't follow right away.

Cheers

[Florian Lengyel]

On Sat, Feb 16, 2013 at 1:12 PM, Andrew Hill <and...@vizzuality.com> wrote:

Hi Florian,

 

Raster datasets, like the Ramankutty cropland map of the last post, make more efficient use of space. So one question is whether I could use within CartoDB the following: instead of multiple layers of oriented vector grid-squares each with a single attribute, a single layer of such polygons with multiple attributes.  I would need programmatic access to the CartoDB.CSS symbology to interactively display the attributes I wanted. I'm suspect though, that this would complicate the Javascript handing of layers (such as the assignment of interactivity...).*

I don't really follow this request? Keep the questions coming, sorry if I don't follow right away.

What I mean is this: the current datasets I have in the asrc account were derived from raster data sets by conversion to ESRI shapefiles with grid squares centered at each raster pixel. The size of the polygonal grid squares are determined by resolution of the raster. Each polygon centered at (lon, lat) has a single attribute equal to the pixel at (lon lat)  [I'm glossing over notation: shapefiles flip the coordinates in the diagonal--they want (lat, lon) in geographic coordinates, whereas ArcInfo Grid ASCII rasters use (x,y)=(lon,lat).]

Diagrammatically  this means that a table imported from an ESRI shapefile

the_geom                        value       

[ grid sq at (lat, lon) ]   -> single pixel at (lat lon) 

(This is from a cropland raster file.)

So for different rasters, I need many such "vectorized" or "grid poly'd" tables:

First table:

the_geom                        value       

[ grid sq at (lat, lon) ]   -> single pixel from 1st raster at (lat lon) 

Second table:

[ grid sq at (lat, lon) ]   -> single pixel from 2nd raster at (lat lon) 

...

Nth table:

[ grid sq at (lat, lon) ]   -> single pixel from Nth raster at (lat lon) 

This is how the data is organized in http://asrc.cuny.edu/crossroads/arims/map.html

But if raster data has to be represented in vector format this way, then it would be more efficient not to duplicate the_geom for every dataset, but to include multiple values within a single table:

the_geom                        value1    .....                        valueN       

[ grid sq at (lat, lon) ]   -> pixel from 1st  ...              pixel from N-th 

                                           raster at (lat lon)              raster at (lat, lon)

This rearrangement makes some assumptions that can be elided for the moment--such as that I have a grid square coverage at the highest resolution (1 min--the cropland example I gave was at 5min) and that I have sampled from a lower resolution. 

My question is whether CartoDB's symbology handling through CartoDB CSS can be modified through Javascript (through some library) to select which of the attributes to view, if this is possible...

[It seems like a substantial reorganization of the code though-- I can indicate how this is done currently.]

Florian

[Andrew Hill]

Yes, absolutely. You can see that happening here, with the tile_style parameter passed via layer updates,

http://developers.cartodb.com/tutorials/query_by_distance.html

cheers,

a