at azkaban.jobtype.HadoopSecureHiveWrapper.runHive(HadoopSecureHiveWrapper.java:83)
23 views
Skip to first unread message
Senhui Lee
Sep 4, 2017, 4:46:26 AM9/4/17
to azkaban
There so strange when I submit a simple Hive query script. Throw an Provider create exception, the log see below:
what't wrong with my setting?
04-09-2017 16:35:45 CST query_scan INFO - Starting job query_scan at 1504514145469
04-09-2017 16:35:45 CST query_scan INFO - azkaban.webserver.url property was not set
04-09-2017 16:35:45 CST query_scan INFO - job JVM args: -Dazkaban.flowid=query_scan -Dazkaban.execid=239 -Dazkaban.jobid=query_scan
04-09-2017 16:35:45 CST query_scan INFO - Building hive job executor.
04-09-2017 16:35:45 CST query_scan INFO - Memory granted for job query_scan
04-09-2017 16:35:45 CST query_scan INFO - Not setting up secure proxy info for child process
04-09-2017 16:35:45 CST query_scan INFO - No classpath specified. Trying to load classes from /data/azkaban_3.32.1/exec-server/executions/239
04-09-2017 16:35:45 CST query_scan INFO - 1 commands to execute.
04-09-2017 16:35:45 CST query_scan INFO - effective user is: hive
04-09-2017 16:35:45 CST query_scan INFO - Command: java -Dazkaban.flowid=query_scan -Dazkaban.execid=239 -Dazkaban.jobid=query_scan -Dhive.querylog.location=. -Dhive.exec.scratchdir=/tmp/hive-hive -Dhive.aux.jars.path=/usr/hdp/2.5.3.0-37/hive/aux/lib -Xms64M -Xmx256M -cp /data/azkaban_3.32.1/exec-server/lib/azkaban-common-0.1.0-SNAPSHOT.jar:/data/azkaban_3.32.1/exec-server/plugins/jobtypes/hive/azkaban-jobtype-3.0.0.jar:/data/azkaban_3.32.1/exec-server/lib/azkaban-hadoop-security-plugin-0.1.0-SNAPSHOT.jar:/data/azkaban_3.32.1/exec-server/executions/239/_resources_query_scan:/usr/hdp/2.5.3.0-37/hadoop/conf:/usr/hdp/2.5.3.0-37/hadoop/lib/*:/usr/hdp/2.5.3.0-37/hive/lib/*:/usr/hdp/2.5.3.0-37/hive/conf:/usr/hdp/2.5.3.0-37/hive/aux/lib:/usr/hdp/2.5.3.0-37/hadoop/*:/usr/hdp/2.5.3.0-37/hadoop/lib:/usr/hdp/2.5.3.0-37/hadoop-mapreduce/*:/usr/hdp/2.5.3.0-37/hadoop-mapreduce/lib azkaban.jobtype.HadoopSecureHiveWrapper -f scripts/query.hql
04-09-2017 16:35:45 CST query_scan INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/data/azkaban_3.32.1/exec-server/executions/239/query_scan_output_917675092567524782_tmp, JOB_PROP_FILE=/data/azkaban_3.32.1/exec-server/executions/239/query_scan_props_4528675676795052964_tmp, KRB5CCNAME=/tmp/krb5cc__Hive_Job__query_scan__query_scan__239__hive, JOB_NAME=query_scan}
04-09-2017 16:35:45 CST query_scan INFO - Working directory: /data/azkaban_3.32.1/exec-server/executions/239
04-09-2017 16:35:51 CST query_scan INFO - Exception in thread "main" javax.xml.parsers.FactoryConfigurationError: Provider for class javax.xml.parsers.DocumentBuilderFactory cannot be created
04-09-2017 16:35:51 CST query_scan INFO - at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:311)
04-09-2017 16:35:51 CST query_scan INFO - at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:267)
04-09-2017 16:35:51 CST query_scan INFO - at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:120)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2555)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2532)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2424)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.conf.Configuration.set(Configuration.java:1149)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.conf.Configuration.set(Configuration.java:1121)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:520)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:538)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:432)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2921)
04-09-2017 16:35:51 CST query_scan INFO - at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:2885)
04-09-2017 16:35:51 CST query_scan INFO - at azkaban.jobtype.HadoopSecureHiveWrapper.runHive(HadoopSecureHiveWrapper.java:83)
04-09-2017 16:35:51 CST query_scan INFO - at azkaban.jobtype.HadoopSecureHiveWrapper.main(HadoopSecureHiveWrapper.java:77)
04-09-2017 16:35:51 CST query_scan INFO - Caused by: java.lang.RuntimeException: Provider for class javax.xml.parsers.DocumentBuilderFactory cannot be created
04-09-2017 16:35:51 CST query_scan INFO - at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:308)
04-09-2017 16:35:51 CST query_scan INFO - ... 14 more
04-09-2017 16:35:51 CST query_scan INFO - Caused by: java.util.ServiceConfigurationError: javax.xml.parsers.DocumentBuilderFactory: Provider org.apache.xerces.jaxp.DocumentBuilderFactoryImpl not found
04-09-2017 16:35:51 CST query_scan INFO - at java.util.ServiceLoader.fail(ServiceLoader.java:239)
04-09-2017 16:35:51 CST query_scan INFO - at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
04-09-2017 16:35:51 CST query_scan INFO - at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)
04-09-2017 16:35:51 CST query_scan INFO - at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
04-09-2017 16:35:51 CST query_scan INFO - at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
04-09-2017 16:35:51 CST query_scan INFO - at javax.xml.parsers.FactoryFinder$1.run(FactoryFinder.java:294)
04-09-2017 16:35:51 CST query_scan INFO - at java.security.AccessController.doPrivileged(Native Method)
04-09-2017 16:35:51 CST query_scan INFO - at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:289)
04-09-2017 16:35:51 CST query_scan INFO - ... 14 more
04-09-2017 16:35:53 CST query_scan INFO - Process completed unsuccessfully in 7 seconds.
04-09-2017 16:35:53 CST query_scan ERROR - caught error running the job
04-09-2017 16:35:53 CST query_scan ERROR - Job run failed!
java.lang.Exception: java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException
at azkaban.jobtype.HadoopHiveJob.run(HadoopHiveJob.java:105)
at azkaban.execapp.JobRunner.runJob(JobRunner.java:745)
at azkaban.execapp.JobRunner.doRun(JobRunner.java:587)
at azkaban.execapp.JobRunner.run(JobRunner.java:548)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:270)
at azkaban.jobtype.HadoopHiveJob.run(HadoopHiveJob.java:101)
... 8 more
Caused by: azkaban.jobExecutor.utils.process.ProcessFailureException
at azkaban.jobExecutor.utils.process.AzkabanProcess.run(AzkabanProcess.java:131)
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:261)
... 9 more
04-09-2017 16:35:53 CST query_scan ERROR - java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException cause: java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException
04-09-2017 16:35:53 CST query_scan INFO - Finishing job query_scan attempt: 0 at 1504514153052 with status FAILED
what't wrong with my setting?
Reply all
Reply to author
Forward
0 new messages