run hadoop job error: Cannot initialize Cluster

[Filix Suo]

Hi All,

I'm new to hadoop and azkaban.

I wrote a test mapreduce job, and it can success run by using : hadoop jar test.jar. It ran success and got the correct output.

But, when I ran it on azkaban with jobtypes-plugin installed, it failed! 

This is the log:

30-07-2014 23:37:27 PDT 1 INFO - Starting job 1 at 1406788647588
30-07-2014 23:37:27 PDT 1 INFO - Building hadoopJava job executor. 
30-07-2014 23:37:27 PDT 1 INFO - No classpath specified. Trying to load classes from /usr/local/azkaban-solo-2.5.0/executions/29/hadoop
30-07-2014 23:37:27 PDT 1 INFO - 1 commands to execute.
30-07-2014 23:37:27 PDT 1 INFO - Command: java  -Xms64M -Xmx256M -cp mapred.jar:/usr/local/azkaban-solo-2.5.0/plugins/jobtypes/hadoopJava/azkaban-jobtype-2.5.0.jar:/usr/local/azkaban-solo-2.5.0/lib/azkaban-2.5.0.jar:/usr/local/azkaban-solo-2.5.0/plugins/jobtypes/hadoopJava/azkaban-hadoopsecuritymanager-2.5.0.jar azkaban.jobtype.HadoopJavaJobRunnerMain 
30-07-2014 23:37:27 PDT 1 INFO - Environment variables: {JOB_NAME=1, JOB_PROP_FILE=/usr/local/azkaban-solo-2.5.0/executions/29/hadoop/1_props_7333591551871568293_tmp, JOB_OUTPUT_PROP_FILE=/usr/local/azkaban-solo-2.5.0/executions/29/hadoop/1_output_7859828782888754080_tmp}
30-07-2014 23:37:27 PDT 1 INFO - Working directory: /usr/local/azkaban-solo-2.5.0/executions/29/hadoop
30-07-2014 23:37:28 PDT 1 ERROR - 2014-07-31 14:37:28.170 java[16428:1903] Unable to load realm info from SCDynamicStore
30-07-2014 23:37:28 PDT 1 INFO - WARN Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
30-07-2014 23:37:28 PDT 1 INFO - INFO Running job 1
30-07-2014 23:37:28 PDT 1 INFO - INFO Class name com.test.Test
30-07-2014 23:37:28 PDT 1 INFO - INFO Constructor found public com.test.Test()
30-07-2014 23:37:28 PDT 1 INFO - INFO Got object com.test.Test@72f2a1bd
30-07-2014 23:37:28 PDT 1 INFO - INFO Constructor found public com.test.Test()
30-07-2014 23:37:28 PDT 1 INFO - INFO Invoking method run
30-07-2014 23:37:28 PDT 1 INFO - INFO Proxy check failed, not proxying run.
30-07-2014 23:37:29 PDT 1 INFO - ERROR PriviledgedActionException as:filix (auth:SIMPLE) cause:java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
30-07-2014 23:37:29 PDT 1 ERROR - Exception in thread "main" java.lang.reflect.InvocationTargetException
30-07-2014 23:37:29 PDT 1 ERROR - 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
30-07-2014 23:37:29 PDT 1 ERROR - 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
30-07-2014 23:37:29 PDT 1 ERROR - 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
30-07-2014 23:37:29 PDT 1 ERROR - 	at java.lang.reflect.Method.invoke(Method.java:606)
30-07-2014 23:37:29 PDT 1 ERROR - 	at azkaban.jobtype.HadoopJavaJobRunnerMain.runMethod(HadoopJavaJobRunnerMain.java:214)
30-07-2014 23:37:29 PDT 1 ERROR - 	at azkaban.jobtype.HadoopJavaJobRunnerMain.(HadoopJavaJobRunnerMain.java:167)
30-07-2014 23:37:29 PDT 1 ERROR - 	at azkaban.jobtype.HadoopJavaJobRunnerMain.main(HadoopJavaJobRunnerMain.java:76)
30-07-2014 23:37:29 PDT 1 ERROR - Caused by: java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:82)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:75)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1238)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1234)
30-07-2014 23:37:29 PDT 1 ERROR - 	at java.security.AccessController.doPrivileged(Native Method)
30-07-2014 23:37:29 PDT 1 ERROR - 	at javax.security.auth.Subject.doAs(Subject.java:415)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Job.connect(Job.java:1233)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1262)
30-07-2014 23:37:29 PDT 1 ERROR - 	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
30-07-2014 23:37:29 PDT 1 ERROR - 	at com.test.Test.main(Test.java:43)
30-07-2014 23:37:29 PDT 1 ERROR - 	at com.test.Test.run(Test.java:47)
30-07-2014 23:37:29 PDT 1 ERROR - 	... 7 more
30-07-2014 23:37:29 PDT 1 INFO - Process completed unsuccessfully in 1 seconds.
30-07-2014 23:37:29 PDT 1 ERROR - Job run failed!
30-07-2014 23:37:29 PDT 1 ERROR - java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureExceptionjava.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException
30-07-2014 23:37:29 PDT 1 INFO - Finishing job 1 at 1406788649507 with status FAILED

it's on hadoop 2.2 and azkaban2.5

 

Anyone can help? 

Best regards