java.lang.ClassNotFoundException: alluxio.yarn.Client

[Harel Gliksman]

I am trying to start Alluxio under Yarn on an Amazon EMR cluster ( 1 master 1 node ).

When running integration/bin/alluxio-yarn.sh 1 hdfs:///tmp_alluxio masterDNS

I hit:

Exception in thread "main" java.lang.ClassNotFoundException: alluxio.yarn.Client

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:348)

at org.apache.hadoop.util.RunJar.run(RunJar.java:214)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Thanks for any help,

Harel.

[Harel Gliksman]

Skipped important details: 

using emr-5.0.0 ( hadoop 2.7.2 ) and Alluxio 1.2.0

[Andrew Audibert]

Hi Harel,

The missing class, alluxio.yarn.Client, is only compiled into the Alluxio jar when the -Pyarn profile is provided to the maven build. See the "Configure Alluxio integration with Yarn" section of http://www.alluxio.org/docs/master/en/Running-Alluxio-on-EC2-Yarn.html

Best,

Andrew

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[and...@alluxio.com]

Hi Harel, 

Did compiling with -Pyarn solve your problem?

Best,

Andrew

On Thursday, September 8, 2016 at 10:34:24 AM UTC-7, Andrew Audibert wrote:

Hi Harel,

The missing class, alluxio.yarn.Client, is only compiled into the Alluxio jar when the -Pyarn profile is provided to the maven build. See the "Configure Alluxio integration with Yarn" section of http://www.alluxio.org/docs/master/en/Running-Alluxio-on-EC2-Yarn.html

Best,

Andrew

[Harel Gliksman]

Hi,

It did help.

I got confused because I had downloaded alluxio-1.2.0-hadoop2.7-bin.tar.gz from

http://downloads.alluxio.org/downloads/files/1.2.0/ and expected its build to include the Yarn classes.

Thanks for the quick reply!

On Fri, Sep 9, 2016 at 7:37 PM, <and...@alluxio.com> wrote:

Hi Harel, 

Did compiling with -Pyarn solve your problem?

Best,

Andrew

On Thursday, September 8, 2016 at 10:34:24 AM UTC-7, Andrew Audibert wrote:

Hi Harel,

The missing class, alluxio.yarn.Client, is only compiled into the Alluxio jar when the -Pyarn profile is provided to the maven build. See the "Configure Alluxio integration with Yarn" section of http://www.alluxio.org/docs/master/en/Running-Alluxio-on-EC2-Yarn.html

Best,

Andrew

[and...@alluxio.com]

Thanks for confirming.

I see how that's confusing. We'll look into either compiling with -Pyarn by default or releasing yarn-compiled artifacts in the next release. Currently we don't include the Yarn dependency in the main artifact because we want to have fewer dependencies and keep the jar smaller.

Best,

Andrew

On Monday, September 12, 2016 at 3:32:07 AM UTC-7, Harel Gliksman wrote:

Hi,

It did help.

I got confused because I had downloaded alluxio-1.2.0-hadoop2.7-bin.tar.gz from

http://downloads.alluxio.org/downloads/files/1.2.0/ and expected its build to include the Yarn classes.

Thanks for the quick reply!

On Fri, Sep 9, 2016 at 7:37 PM, <and...@alluxio.com> wrote:

Hi Harel, 

Did compiling with -Pyarn solve your problem?

Best,

Andrew

On Thursday, September 8, 2016 at 10:34:24 AM UTC-7, Andrew Audibert wrote:

Hi Harel,

The missing class, alluxio.yarn.Client, is only compiled into the Alluxio jar when the -Pyarn profile is provided to the maven build. See the "Configure Alluxio integration with Yarn" section of http://www.alluxio.org/docs/master/en/Running-Alluxio-on-EC2-Yarn.html

Best,

Andrew