NetCDF tools 0.1-rc1 released
Ivan Shmakov
a few tools to manipulate NetCDF and, by utilizing the
compatibility API, HDF4 files, and intended to be usable, in
particular, from the Shell scripts.
The present version is a release candidate and contains some
known bugs and misfeatures, but it's proved to be quite usable.
* Examples
In particular, this package could be used to visualize MODIS L3
(HDF-EOS ``Grid'' structure) data from the Shell (thanks to the
NetPBM and ImageMagick packages), like (Bash):
$ rgb3toppm <(ncget-hdf --raw \
MCD43B4.A2009153.h22v03.005.2009181051526.hdf \
Nadir_Reflectance_Band{1,4,3} \
| rawtopgm -bpp 2 -littleendian \
-maxval 15999 1200 1200) \
| convert -gamma 1.6 ppm:- ppm:- \
| display ppm:-
In this example, the Nadir_Reflectance_Band1, ...4 and ...3
``variables'' (i. e., arrays) of the HDF4 file were fetched and
then converted to the PGM image format. Then, the gamma
correction was applied and the resulting image is shown on an X
display.
(To achieve better image contrast and brightness one may
consider using the piecewise-linear transformation suggested in
[1], which could be done using the `rawxform' tool from the yet
to be released Rawtools package.)
The data is the atmosphere-, cloud- and angle-effects free
spectral reflectances for the MODIS bands 1 (red), 4 (green) and
3 (blue) in a sinusoidal projection on a grid at 1 km (roughly)
resolution. The spatial region covers West Siberia (the
Novosibirsk_Reservoir may be seen at the right side of the
resulting image.) The time window is as follows:
$ cat common-hdfeos.sh
hdfeos_metadata_get () {
local file="$1" name="$2"
local n
## .
for n in Struct Core Archive ; do
ncget-hdf -T "$file" ":${n}Metadata.0"
done \
| sed -e "/\\sOBJECT\\s*=\\s*${name}\$/,/END_OBJECT/!d" \
-e '/^["[:blank:]]*VALUE\s*=\s*"\(.*\)"$/!d' \
-e 's//\1/'
}
$ . common-hdfeos.sh
$ hdfeos_metadata_get MCD43B4.A2009153.h22v03.005.2009181051526.hdf \
RANGEBEGINNINGDATE
2009-06-02
$ hdfeos_metadata_get MCD43B4.A2009153.h22v03.005.2009181051526.hdf \
RANGEENDINGDATE
2009-06-17
$
* Download
http://theory.asu.ru/~ivan/src/netcdf-tools-0.1-rc1.tar.gz
http://waterlily.siamics.net/~ivan/src/netcdf-tools-0.1-rc1.tar.gz
http://waterlily.siamics.net/~ivan/archives/git/nctools.git/
http://waterlily.siamics.net/~ivan/archives/git/gitweb.cgi?p=nctools.git
The file used in the example above is available from NASA:
* References
[1] Creating Reprojected True Color MODIS Images: A Tutorial
http://xweb.geos.ed.ac.uk/~s0785690/methodology/process.pdf
--
FSF associate member #7257
Ivan Shmakov
Hash: SHA1
Drawn by my quest for bizzare Shell scripts, I've written the
one that produces an image file using the nadir BRDF-adjusted
spectral reflectances derived from the MODIS sensors data (as
featured on the NASA EOS Terra and Aqua satellites.)
The script (below) will produce the image in the sinusoidal
projection, mainly due to the lack of magic, but also since the
MODIS data used is distributed that way. Also note that the
data is collected for ``land'' pixels only.
The script expects the HDF files to be in the current directory,
and sends the resulting image to stdout.
I've tested the script using a whole directory (1.66 GiB) of
MODIS data from [1], but it's designed to behave sensibly when
given a part of the data. With little tweaking, I expect this
script to be able to handle other MODIS L3 (say, NDVI, or
LAI/FPAR, or aerosol) datasets just as well.
(BTW, feel free to tweak the `sparsex' and `sparsey' settings at
the top of the script. The less the value the larger will be
the image. But note that setting both to 1 will result in a
43200x21600 PPM file, with the estimated size of 5.21 GiB.)
The results of the test run (converted to PNG) are at [2].
There's also an image further enhanced by $ convert -gamma 1.6
-modulate 130,120,100 at [3].
[1] ftp://e4ftl01u.ecs.nasa.gov/MODIS_Composites/MOTA/MCD43B4.005/2009.06.02/
[2] http://waterlily.siamics.net/~ivan/graphic/1264450492.png
[3] http://waterlily.siamics.net/~ivan/graphic/1264450492-1.png
[4] http://en.wikipedia.org/wiki/MODIS
[5] http://en.wikipedia.org/wiki/Earth_Observing_System
[6] http://en.wikipedia.org/wiki/Sinusoidal_projection
[7] http://en.wikipedia.org/wiki/Hierarchical_Data_Format
[8] http://en.wikipedia.org/wiki/NetCDF
#!/bin/bash
### mcd43b4-globe-1.sh --- Global Aqua+Terra MODIS (MCD43B4) mosaic -*- Sh -*-
## NB: comment out for big-endian (MSB first) systems
le=-littleendian
varp=Nadir_Reflectance_Band
tilew=1200 ; tileh=1200
sparsex=40 ; sparsey=40
tw=$(($tilew / $sparsex)) ; th=$(($tileh / $sparsey))
iw=$((36 * $tw)) ; ih=$((18 * $th))
rawtopgmopts="-bpp 2 ${le} -maxval 15999 ${tw} ${th}"
ix="$(seq 0 "$sparsex" 1199)" ; iy="$(seq 0 "$sparsey" 1199)"
p=/tmp/$(date +%s).${RANDOM}.
i1="$p"1.ppm ; i2="$p"2.ppm ; i3="$p"3.ppm ; it="$p"t.ppm
: > "$i1"
for v in $(seq -f %02.f 0 17) ; do
: > "$i2"
for h in $(seq -f %02.f 0 35) ; do
for f in MCD43B4.A???????."h${h}v${v}".005.?????????????.hdf ; do
if ! [ -e "$f" ] ; then
ppmmake rgb:0/0/0 "$tw" "$th"
else
rgb3toppm <(ncget-hdf --raw -- "$f" "$varp"{1,4,3} \
"${iy}, ${ix}" \
| rawtopgm $rawtopgmopts)
fi
break
done > "$i3"
if ! [ -s "$i2" ] ; then
mv -- "$i3" "$i2"
else
pnmcat -leftright "$i2" "$i3" > "$it"
mv -- "$it" "$i2"
fi
done
if ! [ -s "$i1" ] ; then
mv -- "$i2" "$i1"
else
pnmcat -topbottom "$i1" "$i2" > "$it"
mv -- "$it" "$i1"
fi
done
rm -- "$i2" "$i3"
## .
exec cat -- "$i1"
### mcd43b4-globe-1.sh ends here
- --
FSF associate member #7257
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkteCh0ACgkQMBO2oCMOM0ryxwCglAsiXbLk55MbUwke6w5qMiSC
XAIAn2ZmGkBGesy0PpreCwqiOQg5RXIp
=L/G9
-----END PGP SIGNATURE-----