Spark crashing when trying to read GeoTiff file with geotrellis-geotools

[Hendrik Esch]

Hello geotrellis experts,

my usecase is pretty much the one that is described in https://groups.google.com/forum/#!topic/geotrellis-user/o1kwEyyKvBU but the links provided there are not working anymore. Since my question is probably quite basic I am creating this new topic here. My main problem is that I have never worked with Scala or sbt before and what I am doing wrong is probably something very simple and hopefully easy to fix for an expert.

To summarize the situation, I have a small single-node cluster (CentOS 7 with Spark 1.5.2 with hadoop 2.6) only for testing. All prerequesits like Java 8.0.50, Scala 2.10.5, Python 2.7.5 and sbt 0.13.8 are installed and working properly.

I want to read a GeoTiff file holding satellite data (LandSat) with 3 bands, create a RasterRDD and work with it in the following way. I want to calculate the NDVI vegetation index based on two of the three bands in each pixel of the rasters and save this information as a new layer to some file (probably again GeoTiff).

I created only a very short project following the tutorials on the geotrellis website. The build.sbt and the ReadTiff.scala are attached. When compiling with 'sbt package' everything looks fine and the compilation is successful. I am submitting the job with the command:

 ./spark/bin/spark-submit --class ReadTiff --master local ReadTiff/target/scala-2.10/read-tif-file-test-applicaton_2.10-1.0.jar GeoTiffs/LandSat/LC81950252015114LGN00_bands_543_warped.TIF

The job is crashing with the error I posted to the end of this thread. As I mentioned I am very much a newbie here. I guess there are some jars missing somewhere but I am pretty much stuck here. I would appreciate it very much if somebody could help me with this problem or at least give some commend if I am on the right track. Also other suggestions how to treat my usecase are very much appreciated.

Thanks a lot for your help in advance,
Hendrik

------------

15/11/27 13:11:49 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/11/27 13:11:50 WARN MetricsSystem: Using default name DAGScheduler for source because spark.app.id is not set.
Exception in thread "main" java.lang.NoClassDefFoundError: geotrellis/data/GeoTiff$
        at ReadTiff$.main(ReadTiff.scala:27)
        at ReadTiff.main(ReadTiff.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:674)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: geotrellis.data.GeoTiff$
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 11 more