Compilation error in UIMAMapReduceBase.java with older version of hadoop

[Samudra Banerjee]

Hi all,

I have a cluster with an older version of hadoop deployed. It is 1.0.3. When I change the hadoop.version parameter in pom.xml to 1.0.3, I get the following compilation error during build

/home/sabanerjee/dkpro-bigdata/dkpro-bigdata/de.tudarmstadt.ukp.dkpro.bigdata/de.tudarmstadt.ukp.dkpro.bigdata.hadoop/src/main/java/de/tudarmstadt/ukp/dkpro/bigdata/hadoop/UIMAMapReduceBase.java:[190,16] error: cannot find symbol
[ERROR] -> [Help 1]

On digging further, I find that in line 190, the listFiles method is accessed. Looks like this is not present in version 1.0.3 of hadoop-client and hence the error.

Is there any workaround for this?

Thanks and Regards,
Samudra

[Hans-Peter Zorn]

Hi Samudra,

sorry for not replying earlier. I have looked at it and it seems that I used an API that was introduced with hadoop2 in this class. I've looked at it

but did not have the time to test and fix it.

I think it should work if you modify the method copyRecursively to use 

if (this.localFS.exists(results_dir) && this.localFS.listStatus(results_dir).length > 0)

instead of listFiles. This should also work in Hadoop2. It would be great if you could give feedback if that worked.

Best,

-hp

[Samudra Banerjee]

Hi Hans,

Thanks a lot for the info! That actually worked! :)

Regards,

Samudra