java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration

[ig k]

I use CDH 6.3.2

I want to run hive script

but I get a error

23-06-2021 10:54:27 CST hive-jdbc ERROR - Job run failed! java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at azkaban.jobtype.AbstractHadoopJavaProcessJob.setupHadoopJobProperties(AbstractHadoopJavaProcessJob.java:68) at azkaban.jobtype.HadoopHiveJob.run(HadoopHiveJob.java:49) at azkaban.execapp.JobRunner.runJob(JobRunner.java:830) at azkaban.execapp.JobRunner.doRun(JobRunner.java:607) at azkaban.execapp.JobRunner.run(JobRunner.java:568) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 10 more 23-06-2021 10:54:27 CST hive-jdbc ERROR - org/apache/hadoop/conf/Configuration cause: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration 23-06-2021 10:54:27 CST hive-jdbc INFO - Finishing job hive-jdbc at 1624416867032 with status FAILED

in commonprivate.properties

hadoop.classpath=`hadoop classpath`

hadoop.home=/opt/cloudera/parcels/CDH/lib/hadoop

hive.home=/opt/cloudera/parcels/CDH/lib/hive

spark.home=/opt/cloudera/parcels/CDH/lib/spark

hadoop.conf=/etc/hadoop/conf

jobtype.global.classpath=${hadoop.conf},/opt/cloudera/parcels/CDH/jars/*,${hive.home}/conf,${spark.home}/conf

in hive/private.properties

jobtype.classpath=${hadoop.home}/conf,/opt/cloudera/parcels/CDH/jars/*,${hive.home}/conf,${hive.aux.jar.path}

jobtype.class=azkaban.jobtype.HadoopHiveJob

hive.aux.jar.path=${hive.home}/auxlib

[ig k]

The cause of this error might be in bin/internal/internal-start-executor.sh. I was able to make it work with following tweaks.

if [ "$HADOOP_HOME" != "" ]; then

echo "Using Hadoop from $HADOOP_HOME"

#CLASSPATH=$CLASSPATH:$HADOOP_HOME/conf:$HADOOP_HOME/*

CLASSPATH=$CLASSPATH:/etc/hadoop/conf:/opt/cloudera/parcels/CDH/jars/*

JAVA_LIB_PATH="-Djava.library.path=$HADOOP_HOME/lib/native/Linux-amd64-64"

else

echo "Error: HADOOP_HOME is not set. Hadoop job types will not run properly."

fi