Running gobblin job as Oozie Mapreduce Job

136 views
Skip to first unread message

Revanth Reddy

unread,
May 30, 2017, 12:52:56 AM5/30/17
to gobblin-users
Hi Guys ,

I am trying to run a gobblin job via oozie in MR mode .Below is my workflow.xml file .

and i had set "export HADOOP_USER_CLASSPATH_FIRST=true "in my .bash_profile and also i had included 

job.hdfs.jars=hdfs://xxxxxxxxx:9000/user/centos/oozie-gobblin/lib/ in my gobblin-mapreduce.properties file .


<workflow-app name="gobblin-oozie-example-workflow" xmlns="uri:oozie:workflow:0.1">

<start to="gobblin-local-job"/>

<action name="gobblin-local-job">

<java>

<job-tracker>${jobTracker}</job-tracker>

<name-node>${nameNode}</name-node>

            <configuration>

                <property>

<name>oozie.launcher.mapreduce.user.classpath.first</name>

<value>false</value>

                </property>

</configuration>

<main-class>gobblin.runtime.mapreduce.CliMRJobLauncher</main-class>

<arg>--sysconfig</arg>

            <arg>${nameNode}/user/centos/oozie-gobblin/gobblin-mapreduce.properties</arg>

<arg>--jobconfig</arg>

            <arg>${nameNode}/user/centos/oozie-gobblin/job-pull.properties</arg>

        </java>

<ok to="end"/>

<error to="fail"/>

</action>

<kill name="fail">

<message>Error Message: ${wf:errorMessage(wf:lastErrorNode())}</message>

</kill>

<end name="end"/>

</workflow-app>



But even though i am getting the below error , could any one figure out the issue i am facing .





=================================================================


>>> Invoking Main class now >>>


Fetching child yarn jobs

tag id : oozie-8773ef5fb42cdcd336049ff0b082fe63

Child yarn jobs are found - 

Main class        : gobblin.runtime.mapreduce.CliMRJobLauncher

Arguments         :

                    --sysconfig

                    hdfs://xxxxxxx:9000/user/centos/oozie-gobblin/gobblin-mapreduce.properties

                    --jobconfig

                    hdfs://xxxxxxxx:9000/user/centos/oozie-gobblin/job-pull.properties



<<< Invocation of Main class completed <<<


Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw exception, java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder()Lorg/apache/commons/cli/Option$Builder;

org.apache.oozie.action.hadoop.JavaMainException: java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder()Lorg/apache/commons/cli/Option$Builder;

at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:59)

at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)

at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:35)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)

at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)

at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)

at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder()Lorg/apache/commons/cli/Option$Builder;

at gobblin.runtime.cli.CliOptions.<clinit>(CliOptions.java:34)

at gobblin.runtime.mapreduce.CliMRJobLauncher.main(CliMRJobLauncher.java:103)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:56)

... 19 more




Classpath  jars as below       :

------------------------

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001

  /etc/hadoop

  /opt/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar

  /opt/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar

  /opt/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar

  /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar

  /opt/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar

  /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar

  job.jar/job.jar

  job.jar/classes/

  job.jar/lib/*

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/metrics-core-2.2.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/stax-api-1.0-2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-hive-registration-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/httpclient-4.5.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/groovy-all-2.1.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jopt-simple-3.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-email-1.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/libfb303-0.9.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/influxdb-java-2.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-compaction-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hamcrest-core-1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/org.apache.oltu.oauth2.common-1.0.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jersey-json-1.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jasper-compiler-5.5.23.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-demo-0.1.0-SNAPSHOT-jar-with-dependencies.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/restli-docgen-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-all-7.6.0.v20120127.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/kafka-avro-serializer-2.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-servlet-9.2.14.v20151106.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/data-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/javax.servlet-api-3.1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-dataformat-cbor-2.6.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-collections-3.2.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/mockito-core-1.10.19.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/plexus-utils-1.5.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jersey-server-1.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-admin-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/mina-core-1.1.7.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/regexp-1.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-vfs2-2.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-cli-1.3.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/apache-log4j-extras-1.2.17.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/httpcore-4.4.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/zookeeper-3.4.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/libthrift-0.9.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jaxb-impl-2.2.3-1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-util-9.2.14.v20151106.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/paranamer-2.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-math3-3.5.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/calcite-linq4j-1.2.0-incubating.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/avro-ipc-1.7.7-tests.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/pegasus-common-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/java-xmlbuilder-0.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jersey-guice-1.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/lz4-1.2.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/kafka-json-serializer-2.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/joda-time-2.9.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-pool-1.5.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/bcpg-jdk15on-1.52.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/lombok-1.16.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/activation-1.1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-pool2-2.4.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aopalliance-1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/servlet-api-2.5-20081211.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-sts-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-kms-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/c3p0-0.9.1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/helix-core-0.6.6-SNAPSHOT.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-kafka-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/geronimo-jaspic_1.0_spec-1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/calcite-core-1.2.0-incubating.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/metrics-graphite-3.1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/stax-api-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/junit-3.8.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gson-2.6.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-applicationautoscaling-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/maven-scm-provider-svnexe-1.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jansi-1.11.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-io-9.2.14.v20151106.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-cluster-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-lang-2.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-rest-client-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/bcprov-jdk15on-1.52.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/asm-3.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/datanucleus-api-jdo-3.2.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-logging-1.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/maven-scm-provider-svn-commons-1.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-mapper-asl-1.9.13.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-data-management-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/scala-parser-combinators_2.11-1.0.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-metastore-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-core-2.6.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jasper-runtime-5.5.23.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/zkclient-0.7.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-io-2.5.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/metrics-jvm-3.1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/li-jersey-uri-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/retrofit-1.9.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-rest-api-rest-client-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/javax.inject-1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-compiler-2.7.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/oozie-sharelib-oozie-4.2.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/common-utils-2.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-databind-2.6.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/d2-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/mysql-connector-java-5.1.38.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/r2-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-http-9.2.14.v20151106.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/kafka-clients-0.8.2.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/scala-library-2.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-core-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/netty-3.2.3.Final.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-lang3-3.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-util-6.1.26.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/kafka_2.11-0.8.2.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-rest-server-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jsch-0.1.53.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-shims-common-secure-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/protobuf-java-2.5.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/curator-framework-2.10.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/metrics-core-3.1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/maven-scm-api-1.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/guava-retrying-2.0.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-shims-0.23-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/guice-servlet-3.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-azkaban-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/common-config-2.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jdo-api-3.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-core-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jsp-api-2.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-service-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/asm-commons-3.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jaxb-api-2.2.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/ant-launcher-1.9.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/opencsv-2.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/ant-1.9.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/snappy-0.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-config-core-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jets3t-0.9.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/antlr-runtime-3.5.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jasypt-1.9.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-6.1.26.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-daemon-1.0.13.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-ant-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/guava-15.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/eigenbase-properties-1.1.5.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/flyway-core-3.2.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/restli-netty-standalone-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/transaction-api-1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-shims-common-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-jaxrs-1.8.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/avro-mapred-1.7.7-hadoop2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/org.apache.oltu.oauth2.client-1.0.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/annotations-2.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/xmlenc-0.52.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/codemodel-2.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/testng-6.9.10.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-compress-1.10.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/okio-1.4.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/config-1.2.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/geronimo-jpa_3.0_spec-1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-yarn-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/slf4j-api-1.7.21.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-common-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/objenesis-2.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-iam-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/geronimo-annotation_1.0_spec-1.1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-shims-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-security-9.2.14.v20151106.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/avro-1.7.7.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jcommander-1.48.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/json-simple-1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/javax.mail-1.5.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jetty-server-9.2.14.v20151106.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/restli-client-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/scala-xml_2.11-1.0.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/kafka-schema-registry-client-2.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/javassist-3.18.2-GA.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/json-20140107.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/oozie-hadoop-utils-hadoop-2-4.2.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-el-1.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/restli-common-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/datanucleus-core-3.2.10.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-api-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/asm-tree-3.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/netty-3.7.0.Final.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/azkaban-2.5.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/mail-1.4.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/quartz-2.2.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-tunnel-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/calcite-avatica-1.2.0-incubating.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-core-asl-1.9.13.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-dbcp-1.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-xc-1.8.3.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/restli-server-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-metrics-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/reflections-0.9.10.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-config-client-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/degrader-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jettison-1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/datanucleus-rdbms-3.2.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/curator-client-2.10.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/cglib-nodep-2.2.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/avro-ipc-1.7.7.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-shims-0.20S-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-configuration-1.10.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-exec-1.0.1-core.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/snappy-java-1.1.1.7.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/ST4-4.0.4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-test-harness-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/derby-10.12.1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-aws-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jta-1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-autoscaling-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-httpclient-3.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-utility-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-runtime-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-example-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-serde-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/geronimo-jta_1.1_spec-1.1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-rest-api-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/slf4j-log4j12-1.7.21.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/velocity-1.7.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/parseq-1.3.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/janino-2.7.6.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/restli-tools-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-net-3.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jline-0.9.94.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jdo2-api-2.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/generator-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-ec2-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/gobblin-rest-api-data-template-0.8.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/okhttp-2.4.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/commons-codec-1.10.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jersey-core-1.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/bonecp-0.8.0.RELEASE.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jsr305-3.0.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/bsh-2.0b4.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-shims-0.20-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jpam-1.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/log4j-1.2.17.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-metastore-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/hive-jdbc-1.0.1.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/data-transform-1.15.9.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/aws-java-sdk-s3-1.11.8.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/servlet-api-2.5.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/curator-recipes-2.10.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/guice-4.0.jar

  /data1/yarn/usercache/centos/appcache/application_1492669939336_40900/container_1492669939336_40900_01_000001/jackson-annotations-2.6.0.jar

------------------------

Issac Buenrostro

unread,
May 30, 2017, 11:18:01 AM5/30/17
to Revanth Reddy, gobblin-users
Hi Revanth,

That error means that Gobblin is using the commons-cli artifact in the Hadoop classpath, instead of the one in the Gobblin lib directory. Unfortunately I don't know the details of Oozie, but make sure the gobblin lib/ directory is the first entry in the classpath.

Best,
Issac

--
You received this message because you are subscribed to the Google Groups "gobblin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gobblin-users+unsubscribe@googlegroups.com.
To post to this group, send email to gobbli...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gobblin-users/2969481d-1557-4295-84ce-031a8609f4df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Revanth Reddy

unread,
May 31, 2017, 9:28:10 AM5/31/17
to gobblin-users, revanth...@gmail.com
I had fixed the issue , basically oozie is not taking care of gobblin jars as precedence when it launches map tasks in multiple containers .

 So i had twicked the gooblin code , I am using gobblin 0.8.0 version .

In  gobblin.runtime.mapreduce.MRJobLauncher.java class we have method called 

private void prepareHadoopJob(List<WorkUnit> workUnits){


In this method i had added the below line

 

 this.job.getConfiguration().set("mapreduce.job.user.classpath.first", "true");


then i had build the gobblin-runtime jar and i had used this jar in my class path , then it started working fine .



If  you look at the gobblin 0.10.0 version they basically did the same thing , but this changes are not there in 0.8.0 version .

https://github.com/linkedin/gobblin/blob/gobblin_0.10.0/gobblin-runtime/src/main/java/gobblin/runtime/mapreduce/MRJobLauncher.java

line no 342 .


Thanks,

Revanth Reddy .

...
Reply all
Reply to author
Forward
0 new messages