Raster data: All tiles come up all black

[Gregor at GreenInfo]

I am trying to serve up a set of GeoTIFFs via TileStache using the
Mapnik renderer, but all of the tiles are coming up all black. Any
ideas on what could cause this?

Relevant parts of the napfile are below: A layer using a GeoTIFF in
Google Web Mercator 900913, and the RasterSymbolizer. I would think
this is all that's required, yes?

<Style name="hillshade">
<Rule>
<RasterSymbolizer/>
</Rule>
</Style>

<Layer name="hillshade-grdn37w112_13.tif" status="on">
<StyleName>hillshade</StyleName>
<Datasource>
<Parameter name="type">raster</Parameter>
<Parameter name="file">hillshade-grdn37w112_13.tif</Parameter>
<Parameter name="lox">-12467844.813</Parameter>
<Parameter name="loy">4300539.426</Parameter>
<Parameter name="hix">-12356398.499</Parameter>
<Parameter name="hiy">4439184.225</Parameter>
</Datasource>
</Layer>

I have verified that the TIFFs aren't in fact all black, they do work
in qGIS just fine. Any ideas on this?

[Michal Migurski]

Hi Gregor, I'd love to see the whole file! Maybe even the whole TIFF if you're willing to share. I use GeoTIFFs regularly with Tilestache and while I know that it definitely works, I also know there are a lot of gotchas.

-mike.

----------------------------------------------------------------
michal migurski- mi...@stamen.com
415.558.1610

[Greg Allensworth]

On 11/30/2011 3:28 PM, Michal Migurski wrote:
> Hi Gregor, I'd love to see the whole file! Maybe even the
> whole TIFF if you're willing to share.

This stuff isn't secret, just public data, so sharing wouldn't be a
problem. I'll email you off-list, though, as it'll be a fair volume of
text and a few hundred MB of TIFFs at temporary locations.

--
Greg Allensworth, Web GIS Developer
BS A+ Network+ Security+ Linux+ Server+
GreenInfo Network - Information and Mapping in the Public Interest
564 Market Street, Suite 510 San Francisco CA 94104
PH: 415-979-0343 x302 FX: 415-979-0371 email: gre...@greeninfo.org
Web: www.GreenInfo.org www.MapsPortal.org

Subscribe to MapLines, our e-newsletter, at www.GreenInfo.org

[Dane Springmeyer]

On Nov 30, 2011, at 3:26 PM, Gregor at GreenInfo wrote:
>

> <Layer name="hillshade-grdn37w112_13.tif" status="on">

^^ no 'srs' value is defined there which means Mapnik will assume it is the same srs as the map. So, it is safest to properly define that srs with the proj4 string for spherical mercator, for both the map and layer.

> <StyleName>hillshade</StyleName>
> <Datasource>
> <Parameter name="type">raster</Parameter>
> <Parameter name="file">hillshade-grdn37w112_13.tif</Parameter>
> <Parameter name="lox">-12467844.813</Parameter>
> <Parameter name="loy">4300539.426</Parameter>
> <Parameter name="hix">-12356398.499</Parameter>
> <Parameter name="hiy">4439184.225</Parameter>
> </Datasource>
> </Layer>
>

I see you are using Mapnik's native raster.input plugin. You likely want to be using the 'gdal' plugin, as it will auto-calculate the extent for you, supports overviews, and supports reading for specific bands.

Dane

[Greg Allensworth]

On 11/30/2011 4:24 PM, Dane Springmeyer wrote:
> I see you are using Mapnik's native raster.input plugin.
> You likely want to be using the 'gdal' plugin

I changed over to the GDAL plugin. This did relieve the problem I had
with a debugging header causing the tile's first load to fail. But it
didn't help with the all-black tile output.

The TIFFs and in the same SRS as the map: Google Web Mercator. The Map
and Layer definitions are here, in case you see an error in the Map's SRS.

<Map bgcolor="&watermaskcolor;" srs="+proj=merc +a=6378137 +b=6378137
+lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null
+no_defs +over">

<Style name="hillshade">
<Rule>
<RasterSymbolizer/>
</Rule>
</Style>

<Layer name="grdn32w110_13.tif" status="on">
<StyleName>hillshade</StyleName>
<Datasource>
<Parameter name="type">gdal</Parameter>
<Parameter name="file">grdn32w110_13.tif</Parameter>
</Datasource>
</Layer>
<Layer name="grdn32w111_13.tif" status="on">
<StyleName>hillshade</StyleName>
<Datasource>
<Parameter name="type">gdal</Parameter>
<Parameter name="file">grdn32w111_13.tif</Parameter>
</Datasource>
</Layer>
<Layer name="grdn32w112_13.tif" status="on">
<StyleName>hillshade</StyleName>
<Datasource>
<Parameter name="type">gdal</Parameter>
<Parameter name="file">grdn32w112_13.tif</Parameter>
</Datasource>
</Layer>
<Layer name="grdn32w113_13.tif" status="on">
<StyleName>hillshade</StyleName>
<Datasource>
<Parameter name="type">gdal</Parameter>
<Parameter name="file">grdn32w113_13.tif</Parameter>
</Datasource>
</Layer>

</Map>

[Dane Springmeyer]

On Nov 30, 2011, at 5:19 PM, Greg Allensworth wrote:

> I changed over to the GDAL plugin. This did relieve the problem I had with a debugging header causing the tile's first load to fail. But it didn't help with the all-black tile output.

Okay, how is mapnik built? (and what version)? Another possibility is that there is a linking problem whereby duplicate libtiff versions (the statically linked one in gdal vs the system one libmapnik is likely linked against) are causing problems.

Ideally post a reduced test case to a new issue at https://github.com/mapnik/mapnik/issues and I can take a look tomorrow.

Dane

[Michal Migurski]

Can you, on the same machine, use gdalwarp on the command line to generate a tile by hand?

Something like:
gdalwarp -t_srs EPSG:900913 -te -12474522 3757031 -12464738 3766815 -tr 19.1 19.1 grdn32w112_13.tif one-tile.tif

Is it black?

-mike.

On Nov 30, 2011, at 5:19 PM, Greg Allensworth wrote:

----------------------------------------------------------------
michal migurski- mi...@stamen.com
415.558.1610

[Greg Allensworth]

> Okay, how is mapnik built? (and what version)?

> linking problem whereby duplicate libtiff versions

mapnik.mapnik_version() returns "601" I see /usr/lib/libmapnik.so.0.6.1
And the Ubuntu (karmic 9.10) package is named libmapnik0.6

Clearly it has GDAL support, as type=gdal isn't generating an error.

I ran ldd on gdalinfo, libmapnik.so.0.6.1, and libgdal.so and they all
report the same libtiff: /usr/lib/libtiff.so.4

I don't know the compile options; I inherited this project which has
lain dormant for a few years. There's no mapnik-config; how else would I
check compile-time settings?

> Ideally post a reduced test case to a new issue at
> https://github.com/mapnik/mapnik/issues

Can do:
https://github.com/mapnik/mapnik/issues/972

[Michal Migurski]

On Dec 1, 2011, at 8:34 AM, Greg Allensworth wrote:

>> Ideally post a reduced test case to a new issue at
>> https://github.com/mapnik/mapnik/issues
>
> Can do:
> https://github.com/mapnik/mapnik/issues/972

I see that you're getting help from Dane and Artem over in the Mapnik forum, so yay!

Do let us know if at any point this loops back to TileStache, and I'll keep an eye on the Mapnik thread.

-mike.

[Gregor at GreenInfo]

The folks at Mapnik helped me out. Turns out that this was a known
issue of Mapnik 0.6 -- it has problems with handling single-band
grayscale TIFFs. I have upgraded to Mapnik 2 and it does just fine.