Hadoop YARN compaitbility

[rax...@gmail.com]

I get this error when I try to run on CDH4 distribution, when will be development for YARN is expected.

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at com.linkedin.camus.etl.kafka.mapred.EtlInputFormat.getKafkaBlacklistTopic(EtlInputFormat.java:403)
at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:194)
at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:551)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at com.linkedin.camus.etl.kafka.CamusJob.main(CamusJob.java:516)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

Thanks
Rakshith Begane.

[aco...@gmail.com]

Ah! I'm getting this as well right now. I'm looking into it, but not having much luck at the moment.

[aco...@gmail.com]

Actually, my full error message is:

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

at com.linkedin.camus.etl.kafka.CamusJob.getLog4jConfigure(CamusJob.java:655)
at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:199)
at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:598)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at com.linkedin.camus.etl.kafka.CamusJob.main(CamusJob.java:562)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

[aco...@gmail.com]

Ah! In my case this was caused by not running mvn clean after changing hadoop dependencies in pom.xml files.

[mjame...@gmail.com]

On Tuesday, January 7, 2014 1:04:38 PM UTC-7, aco...@gmail.com wrote:
> Ah! In my case this was caused by not running mvn clean after changing hadoop dependencies in pom.xml files.

Struggling with the dependency changes for building for yarn, can you post your pom changes please?

[doranj...@gmail.com]

I was getting this same exception when trying to build in Maven due to trying to change Camus to run on Hadoop 2. I replaced the hadoop-client dependency with the hadoop-core and hadoop-common dependencies. Note that hadoop-core uses the mr1 version and hadoop-common doesn't. This allowed me to build with Maven.

<hadoop.version>2.0.0-cdh4.5.0</hadoop.version>
<hadoop.mr1.version>2.0.0-mr1-cdh4.5.0</hadoop.mr1.version>

<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>${hadoop.mr1.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>${hadoop.version}</version>
</dependency>