Unable to execute spark program through workflow
amit...@gmail.com
Terence Yim
--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/8b5520a5-85c8-4de2-aba3-692c9c6ebcdc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nitin Motgi
Thanks,
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/FECDA72B-9F3D-490A-8222-22629FE9D89A%40cask.co.
Sagar Kapare
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/C3DA4FC1-E4D3-457A-8C0B-BEB86F16D463%40cask.co.
amit...@gmail.com
Yes, Correct.
chris
amit...@gmail.com
We tried to run simple workflow example avalilable in SparkPageRank example. Got same exception related to SparkConf.
Observations:
1. Full container logs: stdout, stderr, and syslog (logs from cdap-master logs)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2015-08-12T09:29:13,382Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371212656
2015-08-12T09:30:13,406Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439371813405
2015-08-12T09:30:13,407Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439371813405, readPointer=1439371813286000000, writePointer=1439371813286000000, invalidSize=52, inProgressSize=0, committingSize=0, committedSize=0}
2015-08-12T09:30:13,419Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439371813405
2015-08-12T09:30:13,424Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3082] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439371813405
2015-08-12T09:30:13,448Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439371813405
2015-08-12T09:30:13,458Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371272724
2015-08-12T09:30:13,467Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371272724
2015-08-12 15:00:19,445 - INFO [executor-137:c.c.c.i.a.r.d.DistributedWorkflowProgramRunner@121] - Launching distributed workflow: PageRankWorkflow:PageRankWorkflow
2015-08-12 15:00:20,001 INFO [ProducerSendThread-] client.ClientUtils$: Fetching metadata from broker id:0,host:hdp-4.xyz.co.in,port:9092 with correlation id 714 for 1 topic(s) Set(logs.user-v2)
2015-08-12 15:00:20,001 INFO [ProducerSendThread-] producer.SyncProducer: Connected to hdp-4.xyz.co.in:9092 for producing
2015-08-12 15:00:20,003 INFO [ProducerSendThread-] producer.SyncProducer: Disconnecting from hdp-4.xyz.co.in:9092
2015-08-12 15:00:20,003 INFO [ProducerSendThread-] producer.SyncProducer: Disconnecting from hdp-4.xyz.co.in:9092
2015-08-12 15:00:20,004 INFO [ProducerSendThread-] producer.SyncProducer: Connected to hdp-4.xyz.co.in:9092 for producing
2015-08-12 15:00:46,674 - INFO [ STARTING:c.c.c.i.a.r.d.AbstractTwillProgramController$1@62] - Twill program running: PageRankWorkflow 2c468aae-86a1-4a91-8732-32f595f962f1
2015-08-12T09:30:44,416Z INFO o.a.t.i.ServiceMain [localhost] [main] ServiceMain:doMain(ServiceMain.java:89) - Starting service ApplicationMasterService [NEW].
2015-08-12T09:30:44,552Z WARN o.a.t.i.k.c.SimpleKafkaPublisher [localhost] [kafka-publisher] SimpleKafkaPublisher$BrokerListChangeListener:changed(SimpleKafkaPublisher.java:163) - Broker list is empty. No Kafka producer is created.
2015-08-12T09:30:44,599Z INFO o.a.t.i.a.TrackerService [localhost] [TrackerService STARTING] TrackerService:startUp(TrackerService.java:159) - Tracker service started at http://localhost:38847
2015-08-12T09:30:44,998Z INFO o.a.t.i.ServiceMain$TwillZKPathService [localhost] [TwillZKPathService STARTING] ServiceMain$TwillZKPathService:startUp(ServiceMain.java:277) - Creating container ZK path: localhost:2181/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1
2015-08-12T09:30:45,313Z INFO o.a.t.i.k.c.SimpleKafkaPublisher [localhost] [kafka-publisher] SimpleKafkaPublisher$BrokerListChangeListener:changed(SimpleKafkaPublisher.java:185) - Update Kafka producer broker list: localhost:55999
2015-08-12T09:30:45,810Z INFO o.a.t.i.AbstractTwillService [localhost] [ApplicationMasterService] AbstractTwillService:createLiveNode(AbstractTwillService.java:196) - Create live node localhost:2181/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/instances/2c468aae-86a1-4a91-8732-32f595f962f1
2015-08-12T09:30:45,847Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:doStart(ApplicationMasterService.java:220) - Start application master with spec: {"name":"workflow.default.SparkPageRank.PageRankWorkflow","runnables":{"PageRankWorkflow":{"name":"PageRankWorkflow","runnable":{"classname":"co.cask.cdap.internal.app.runtime.distributed.WorkflowTwillRunnable","name":"PageRankWorkflow","arguments":{"hConf":"hConf.xml","cConf":"cConf.xml"}},"resources":{"cores":1,"memorySize":1024,"instances":1,"uplink":-1,"downlink":-1},"files":[{"name":"mr-framework","uri":"hdfs://localhost:8020/hdp/apps/2.2.4.2-2/mapreduce/mapreduce.tar.gz","lastModified":1438671719280,"size":192697367,"archive":true,"pattern":null},{"name":"cConf.xml","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/cConf.xml.9de269d5-520b-4874-a264-a115c34ac12a.xml","lastModified":1439371833272,"size":21745,"archive":false,"pattern":null},{"name":"spark-assembly-1.2.1.2.2.4.2-2-hadoop2.6.0.2.2.4.2-2.jar","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/spark-assembly-1.2.1.2.2.4.2-2-hadoop2.6.0.2.2.4.2-2.jar.311a8860-68b9-40fa-9d16-a8927794660d.jar","lastModified":1439371837244,"size":156197440,"archive":false,"pattern":null},{"name":"PageRankWorkflow656786471940694075.jar","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/PageRankWorkflow656786471940694075.jar.2d3a72bc-daf9-41d0-a27a-3ed0fdd74012.jar","lastModified":1439371833182,"size":24126,"archive":false,"pattern":null},{"name":"hConf.xml","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/hConf.xml.f00f0f1d-0bb9-4ac8-becb-76ecf2aaa7c8.xml","lastModified":1439371833211,"size":111268,"archive":false,"pattern":null},{"name":"launcher.jar","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/launcher.jar.40f2a3db-186d-46ab-be4e-b63d64f85412.jar","lastModified":1439371833236,"size":8617,"archive":false,"pattern":null}]}},"orders":[{"names":["PageRankWorkflow"],"type":"STARTED"}],"placementPolicies":[],"handler":{"classname":"co.cask.cdap.common.twill.AbortOnTimeoutEventHandler","configs":{"abortIfNotFull":"false","abortTime":"120000"}}}
2015-08-12T09:30:45,884Z INFO o.a.t.i.ServiceMain [localhost] [main] ServiceMain:doMain(ServiceMain.java:91) - Service ApplicationMasterService [RUNNING] started.
2015-08-12T09:30:45,956Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:addContainerRequests(ApplicationMasterService.java:591) - Request 1 container with capability <memory:1024, vCores:1> for runnable PageRankWorkflow
2015-08-12T09:30:48,018Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:launchRunnable(ApplicationMasterService.java:607) - Got container container_e04_1438850569323_0121_01_000002
2015-08-12T09:30:48,020Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:launchRunnable(ApplicationMasterService.java:614) - Starting runnable PageRankWorkflow with RunnableProcessLauncher{container=org.apache.twill.internal.yarn.Hadoop21YarnContainerInfo@e3189b0}
2015-08-12T09:30:48,104Z INFO o.a.t.i.a.RunnableProcessLauncher [localhost] [ApplicationMasterService] RunnableProcessLauncher:doLaunch(RunnableProcessLauncher.java:71) - Launching in container container_e04_1438850569323_0121_01_000002 at localhost:45454, [$JAVA_HOME/bin/java -Djava.io.tmpdir=tmp -Dyarn.container=$YARN_CONTAINER_ID -Dtwill.runnable=$TWILL_APP_NAME.$TWILL_RUNNABLE_NAME -cp launcher.jar:$HADOOP_CONF_DIR -Xmx774m -XX:MaxPermSize=128M -verbose:gc -Xloggc:<LOG_DIR>/gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -Dhdp.version=2.2.4.2-2 org.apache.twill.launcher.TwillLauncher container.jar org.apache.twill.internal.container.TwillContainerMain true 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr]
2015-08-12T09:30:48,269Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:launchRunnable(ApplicationMasterService.java:650) - Runnable PageRankWorkflow fully provisioned with 1 instances.
2015-08-12 15:00:53,061 - WARN [pool-5-thread-1:c.c.c.i.a.s.ProgramLifecycleService@273] - Found 1 RunRecords with RUNNING status but the program not actually running
2015-08-12 15:00:53,387 - WARN [pool-5-thread-1:c.c.c.i.a.s.ProgramLifecycleService@273] - Found 1 RunRecords with RUNNING status but the program not actually running
2015-08-12 15:00:53,880 - WARN [pool-5-thread-1:c.c.c.i.a.s.ProgramLifecycleService@273] - Found 1 RunRecords with RUNNING status but the program not actually running
2015-08-12 15:00:54,065 - WARN [pool-5-thread-1:c.c.c.i.a.s.ProgramLifecycleService@273] - Found 1 RunRecords with RUNNING status but the program not actually running
2015-08-12 15:00:54,414 - WARN [pool-5-thread-1:c.c.c.i.a.s.ProgramLifecycleService@273] - Found 1 RunRecords with RUNNING status but the program not actually running
2015-08-12 15:00:54,715 - WARN [pool-5-thread-1:c.c.c.i.a.s.ProgramLifecycleService@273] - Found 1 RunRecords with RUNNING status but the program not actually running
2015-08-12 15:00:54,914 - WARN [pool-5-thread-1:c.c.c.i.a.s.ProgramLifecycleService@273] - Found 1 RunRecords with RUNNING status but the program not actually running
2015-08-12T09:31:01,901Z INFO o.a.t.i.TwillContainerLauncher [localhost] [zk-client-EventThread] TwillContainerLauncher$TwillContainerControllerImpl:instanceNodeUpdated(TwillContainerLauncher.java:196) - Container LiveNodeData updated: {"data":{"containerId":"container_e04_1438850569323_0121_01_000002","host":"localhost"}}
2015-08-12T09:31:01,748Z INFO o.a.t.i.ServiceMain [localhost] [main] ServiceMain:doMain(ServiceMain.java:89) - Starting service TwillContainerService [NEW].
2015-08-12T09:31:01,772Z INFO o.a.t.i.ServiceMain$TwillZKPathService [localhost] [TwillZKPathService STARTING] ServiceMain$TwillZKPathService:startUp(ServiceMain.java:277) - Creating container ZK path: localhost:2181/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/runnables/PageRankWorkflow/3857c51e-9ba3-4a3c-af2b-5950522e1a50-0
2015-08-12T09:31:01,875Z INFO o.a.t.i.AbstractTwillService [localhost] [TwillContainerService] AbstractTwillService:createLiveNode(AbstractTwillService.java:196) - Create live node localhost:2181/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/runnables/PageRankWorkflow/instances/3857c51e-9ba3-4a3c-af2b-5950522e1a50-0
2015-08-12T09:31:02,024Z INFO c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [TwillContainerService] AbstractProgramTwillRunnable:initialize(AbstractProgramTwillRunnable.java:181) - Initialize runnable: PageRankWorkflow
2015-08-12T09:31:02,421Z WARN c.c.c.c.c.Configuration [localhost] [TwillContainerService] Configuration:loadProperty(Configuration.java:1748) - file:/hadoop/yarn/local/usercache/cdap/appcache/application_1438850569323_0121/container_e04_1438850569323_0121_01_000002/cConf.xml:an attempt to override final parameter: app.template.plugin.dir; Ignoring.
2015-08-12T09:31:02,424Z WARN c.c.c.c.c.Configuration [localhost] [TwillContainerService] Configuration:loadProperty(Configuration.java:1748) - file:/hadoop/yarn/local/usercache/cdap/appcache/application_1438850569323_0121/container_e04_1438850569323_0121_01_000002/cConf.xml:an attempt to override final parameter: stream.instance.file.prefix; Ignoring.
2015-08-12T09:31:02,754Z INFO o.a.t.i.k.c.SimpleKafkaPublisher [localhost] [kafka-publisher] SimpleKafkaPublisher$BrokerListChangeListener:changed(SimpleKafkaPublisher.java:185) - Update Kafka producer broker list: localhost:55999
2015-08-12T09:31:04,329Z INFO c.c.c.l.a.LogAppenderInitializer [localhost] [TwillContainerService] LogAppenderInitializer:initialize(LogAppenderInitializer.java:68) - Initializing log appender KafkaLogAppender
2015-08-12T09:31:04,364Z INFO c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [TwillContainerService] AbstractProgramTwillRunnable:initialize(AbstractProgramTwillRunnable.java:223) - Runnable initialized: PageRankWorkflow
2015-08-12T09:31:04,370Z INFO c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [TwillContainerService] AbstractProgramTwillRunnable:run(AbstractProgramTwillRunnable.java:273) - Starting metrics service
2015-08-12T09:31:04,377Z INFO o.a.t.i.ServiceMain [localhost] [main] ServiceMain:doMain(ServiceMain.java:91) - Service TwillContainerService [RUNNING] started.
2015-08-12T09:31:04,401Z INFO o.a.t.i.k.c.SimpleKafkaPublisher [localhost] [kafka-publisher] SimpleKafkaPublisher$BrokerListChangeListener:changed(SimpleKafkaPublisher.java:185) - Update Kafka producer broker list: hdp-4.xyz.co.in:9092
2015-08-12T09:31:04,416Z INFO c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [TwillContainerService] AbstractProgramTwillRunnable:run(AbstractProgramTwillRunnable.java:278) - Starting runnable: PageRankWorkflow
2015-08-12T09:31:04,439Z INFO c.c.c.c.g.LocationRuntimeModule [localhost] [TwillContainerService] LocationRuntimeModule$HDFSLocationModule:providesHDFSLocationFactory(LocationRuntimeModule.java:105) - HDFS namespace is /cdap
2015-08-12T09:31:04,538Z INFO c.c.c.i.a.r.w.WorkflowDriver [localhost] [WorkflowDriver] WorkflowDriver:startUp(WorkflowDriver.java:168) - Starting Workflow WorkflowSpecification{className='co.cask.cdap.examples.sparkpagerank.SparkPageRankApp$PageRankWorkflow', name='PageRankWorkflow', description='Runs SparkPageRankProgram followed by RanksCounter MapReduce', properties={}, nodes=[WorkflowActionNode{nodeId=SparkPageRankProgram, program=ScheduleProgramInfo{programName='SparkPageRankProgram', programType=SPARK}, actionSpecification=null}, WorkflowActionNode{nodeId=RanksCounter, program=ScheduleProgramInfo{programName='RanksCounter', programType=MAPREDUCE}, actionSpecification=null}]}
2015-08-12T09:31:04,631Z INFO c.c.h.NettyHttpService [localhost] [NettyHttpService STARTING] NettyHttpService:startUp(NettyHttpService.java:259) - Starting service on address localhost/10.77.36.73:0...
2015-08-12T09:31:04,748Z INFO c.c.h.NettyHttpService [localhost] [NettyHttpService STARTING] NettyHttpService:startUp(NettyHttpService.java:264) - Started service on address /10.77.36.73:54380
2015-08-12T09:31:04,782Z INFO c.c.c.i.a.r.w.WorkflowProgramController [localhost] [WorkflowDriver] WorkflowProgramController$1:running(WorkflowProgramController.java:78) - Workflow service workflow.default.SparkPageRank.PageRankWorkflow.bf7071fb-40d4-11e5-a1b6-08002714ea67 announced at /10.77.36.73:54380
2015-08-12T09:31:04,786Z INFO c.c.c.i.a.r.w.WorkflowDriver [localhost] [WorkflowDriver] WorkflowDriver:run(WorkflowDriver.java:454) - Start workflow execution for WorkflowSpecification{className='co.cask.cdap.examples.sparkpagerank.SparkPageRankApp$PageRankWorkflow', name='PageRankWorkflow', description='Runs SparkPageRankProgram followed by RanksCounter MapReduce', properties={}, nodes=[WorkflowActionNode{nodeId=SparkPageRankProgram, program=ScheduleProgramInfo{programName='SparkPageRankProgram', programType=SPARK}, actionSpecification=null}, WorkflowActionNode{nodeId=RanksCounter, program=ScheduleProgramInfo{programName='RanksCounter', programType=MAPREDUCE}, actionSpecification=null}]}
2015-08-12T09:31:08,567Z INFO c.c.t.d.AbstractClientProvider [localhost] [WorkflowDriver] AbstractClientProvider:newClient(AbstractClientProvider.java:109) - Service discovered at hdp-3.xyz.co.in:15165
2015-08-12T09:31:08,568Z INFO c.c.t.d.AbstractClientProvider [localhost] [WorkflowDriver] AbstractClientProvider:newClient(AbstractClientProvider.java:117) - Attempting to connect to tx service at hdp-3.xyz.co.in:15165 with timeout 30000 ms.
2015-08-12T09:31:08,594Z INFO c.c.t.d.AbstractClientProvider [localhost] [WorkflowDriver] AbstractClientProvider:newClient(AbstractClientProvider.java:131) - Connected to tx service at hdp-3.xyz.co.in:15165
2015-08-12T09:31:08,647Z INFO c.c.c.i.w.ProgramWorkflowAction [localhost] [WorkflowDriver] ProgramWorkflowAction:initialize(ProgramWorkflowAction.java:73) - Initialized for SPARK Program SparkPageRankProgram in workflow action
2015-08-12T09:31:08,676Z INFO c.c.c.i.w.ProgramWorkflowAction [localhost] [workflow-executor-0] ProgramWorkflowAction:run(ProgramWorkflowAction.java:80) - Starting Program for workflow action: SparkPageRankProgram
2015-08-12T09:31:08,725Z INFO c.c.t.d.AbstractClientProvider [localhost] [workflow-executor-0] AbstractClientProvider:newClient(AbstractClientProvider.java:109) - Service discovered at hdp-3.xyz.co.in:15165
2015-08-12T09:31:08,727Z INFO c.c.t.d.AbstractClientProvider [localhost] [workflow-executor-0] AbstractClientProvider:newClient(AbstractClientProvider.java:117) - Attempting to connect to tx service at hdp-3.xyz.co.in:15165 with timeout 30000 ms.
2015-08-12T09:31:08,729Z INFO c.c.t.d.AbstractClientProvider [localhost] [workflow-executor-0] AbstractClientProvider:newClient(AbstractClientProvider.java:131) - Connected to tx service at hdp-3.xyz.co.in:15165
2015-08-12T09:31:08,742Z ERROR c.c.c.i.a.r.w.WorkflowDriver [localhost] [WorkflowDriver] WorkflowDriver:executeAction(WorkflowDriver.java:271) - Exception on WorkflowAction.run(), aborting Workflow. WorkflowActionSpecification{name=SparkPageRankProgram, class=co.cask.cdap.internal.workflow.ProgramWorkflowAction, options={ProgramType=SPARK, ProgramName=SparkPageRankProgram}, datasets=[]}
2015-08-12T09:31:08,748Z INFO c.c.h.NettyHttpService [localhost] [NettyHttpService STOPPING] NettyHttpService:shutDown(NettyHttpService.java:276) - Stopping service on address /10.77.36.73:54380...
2015-08-12T09:31:08,762Z INFO c.c.h.NettyHttpService [localhost] [NettyHttpService STOPPING] NettyHttpService:shutDown(NettyHttpService.java:286) - Done stopping service on address /10.77.36.73:54380
2015-08-12T09:31:08,779Z INFO c.c.c.i.a.r.w.WorkflowProgramController [localhost] [WorkflowDriver] WorkflowProgramController$1:failed(WorkflowProgramController.java:98) - Workflow service failed from RUNNING. Un-registering service workflow.default.SparkPageRank.PageRankWorkflow.bf7071fb-40d4-11e5-a1b6-08002714ea67.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:475)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:456)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAction(WorkflowDriver.java:269)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeNode(WorkflowDriver.java:413)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:468)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:456)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkContext.<init>(AbstractSparkContext.java:90)
at co.cask.cdap.internal.app.runtime.spark.ClientSparkContext.<init>(ClientSparkContext.java:55)
at co.cask.cdap.internal.app.runtime.spark.SparkProgramRunner.run(SparkProgramRunner.java:118)
at co.cask.cdap.internal.app.runtime.workflow.SparkProgramWorkflowRunner.runAndWait(SparkProgramWorkflowRunner.java:73)
at co.cask.cdap.internal.app.runtime.workflow.AbstractProgramWorkflowRunner$1.run(AbstractProgramWorkflowRunner.java:117)
at co.cask.cdap.internal.workflow.ProgramWorkflowAction.run(ProgramWorkflowAction.java:81)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver$1.run(WorkflowDriver.java:258)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkContext.<init>(AbstractSparkContext.java:90)
at co.cask.cdap.internal.app.runtime.spark.ClientSparkContext.<init>(ClientSparkContext.java:55)
at co.cask.cdap.internal.app.runtime.spark.SparkProgramRunner.run(SparkProgramRunner.java:118)
at co.cask.cdap.internal.app.runtime.workflow.SparkProgramWorkflowRunner.runAndWait(SparkProgramWorkflowRunner.java:73)
at co.cask.cdap.internal.app.runtime.workflow.AbstractProgramWorkflowRunner$1.run(AbstractProgramWorkflowRunner.java:117)
at co.cask.cdap.internal.workflow.ProgramWorkflowAction.run(ProgramWorkflowAction.java:81)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver$1.run(WorkflowDriver.java:258)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-08-12T09:31:08,806Z INFO c.c.c.i.a.r.w.WorkflowProgramController [localhost] [WorkflowDriver] WorkflowProgramController$1:failed(WorkflowProgramController.java:100) - Service workflow.default.SparkPageRank.PageRankWorkflow.bf7071fb-40d4-11e5-a1b6-08002714ea67 unregistered.
2015-08-12T09:31:08,808Z ERROR c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [WorkflowDriver] AbstractProgramTwillRunnable$1:error(AbstractProgramTwillRunnable.java:309) - Program runner error out.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:475)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:456)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAction(WorkflowDriver.java:269)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeNode(WorkflowDriver.java:413)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:468)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:456)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkContext.<init>(AbstractSparkContext.java:90)
at co.cask.cdap.internal.app.runtime.spark.ClientSparkContext.<init>(ClientSparkContext.java:55)
at co.cask.cdap.internal.app.runtime.spark.SparkProgramRunner.run(SparkProgramRunner.java:118)
at co.cask.cdap.internal.app.runtime.workflow.SparkProgramWorkflowRunner.runAndWait(SparkProgramWorkflowRunner.java:73)
at co.cask.cdap.internal.app.runtime.workflow.AbstractProgramWorkflowRunner$1.run(AbstractProgramWorkflowRunner.java:117)
at co.cask.cdap.internal.workflow.ProgramWorkflowAction.run(ProgramWorkflowAction.java:81)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver$1.run(WorkflowDriver.java:258)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkContext.<init>(AbstractSparkContext.java:90)
at co.cask.cdap.internal.app.runtime.spark.ClientSparkContext.<init>(ClientSparkContext.java:55)
at co.cask.cdap.internal.app.runtime.spark.SparkProgramRunner.run(SparkProgramRunner.java:118)
at co.cask.cdap.internal.app.runtime.workflow.SparkProgramWorkflowRunner.runAndWait(SparkProgramWorkflowRunner.java:73)
at co.cask.cdap.internal.app.runtime.workflow.AbstractProgramWorkflowRunner$1.run(AbstractProgramWorkflowRunner.java:117)
at co.cask.cdap.internal.workflow.ProgramWorkflowAction.run(ProgramWorkflowAction.java:81)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver$1.run(WorkflowDriver.java:258)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-08-12T09:31:08,830Z ERROR c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [TwillContainerService] AbstractProgramTwillRunnable:run(AbstractProgramTwillRunnable.java:320) - Program execution failed.
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294)
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at co.cask.cdap.internal.app.runtime.distributed.AbstractProgramTwillRunnable.run(AbstractProgramTwillRunnable.java:315)
at org.apache.twill.internal.container.TwillContainerService.doRun(TwillContainerService.java:130)
at org.apache.twill.internal.AbstractTwillService.run(AbstractTwillService.java:179)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:475)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:456)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAction(WorkflowDriver.java:269)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeNode(WorkflowDriver.java:413)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:468)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:456)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkContext.<init>(AbstractSparkContext.java:90)
at co.cask.cdap.internal.app.runtime.spark.ClientSparkContext.<init>(ClientSparkContext.java:55)
at co.cask.cdap.internal.app.runtime.spark.SparkProgramRunner.run(SparkProgramRunner.java:118)
at co.cask.cdap.internal.app.runtime.workflow.SparkProgramWorkflowRunner.runAndWait(SparkProgramWorkflowRunner.java:73)
at co.cask.cdap.internal.app.runtime.workflow.AbstractProgramWorkflowRunner$1.run(AbstractProgramWorkflowRunner.java:117)
at co.cask.cdap.internal.workflow.ProgramWorkflowAction.run(ProgramWorkflowAction.java:81)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver$1.run(WorkflowDriver.java:258)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkContext.<init>(AbstractSparkContext.java:90)
at co.cask.cdap.internal.app.runtime.spark.ClientSparkContext.<init>(ClientSparkContext.java:55)
at co.cask.cdap.internal.app.runtime.spark.SparkProgramRunner.run(SparkProgramRunner.java:118)
at co.cask.cdap.internal.app.runtime.workflow.SparkProgramWorkflowRunner.runAndWait(SparkProgramWorkflowRunner.java:73)
at co.cask.cdap.internal.app.runtime.workflow.AbstractProgramWorkflowRunner$1.run(AbstractProgramWorkflowRunner.java:117)
at co.cask.cdap.internal.workflow.ProgramWorkflowAction.run(ProgramWorkflowAction.java:81)
at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver$1.run(WorkflowDriver.java:258)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-08-12T09:31:08,843Z INFO c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [TwillContainerService] AbstractProgramTwillRunnable:destroy(AbstractProgramTwillRunnable.java:333) - Releasing resources: PageRankWorkflow
2015-08-12T09:31:08,852Z INFO o.a.t.i.k.c.ZKKafkaClientService [localhost] [ZKKafkaClientService STOPPING] ZKKafkaClientService:shutDown(ZKKafkaClientService.java:106) - Stopping KafkaClientService
2015-08-12T09:31:08,853Z INFO o.a.t.i.k.c.SimpleKafkaConsumer [localhost] [ZKKafkaClientService STOPPING] SimpleKafkaConsumer:stop(SimpleKafkaConsumer.java:101) - Stopping Kafka consumer
2015-08-12T09:31:08,854Z INFO o.a.t.i.k.c.SimpleKafkaConsumer [localhost] [ZKKafkaClientService STOPPING] SimpleKafkaConsumer:stop(SimpleKafkaConsumer.java:108) - Kafka Consumer stopped
2015-08-12T09:31:08,856Z INFO o.a.t.i.k.c.ZKKafkaClientService [localhost] [ZKKafkaClientService STOPPING] ZKKafkaClientService:shutDown(ZKKafkaClientService.java:114) - KafkaClientService stopped
2015-08-12T09:31:08,860Z INFO c.c.c.i.a.r.d.AbstractProgramTwillRunnable [localhost] [TwillContainerService] AbstractProgramTwillRunnable:destroy(AbstractProgramTwillRunnable.java:340) - Runnable stopped: PageRankWorkflow
2015-08-12T09:31:08,861Z INFO o.a.t.i.AbstractTwillService [localhost] [TwillContainerService] AbstractTwillService:removeLiveNode(AbstractTwillService.java:209) - Remove live node localhost:2181/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/runnables/PageRankWorkflow/instances/3857c51e-9ba3-4a3c-af2b-5950522e1a50-0
2015-08-12T09:31:08,864Z INFO o.a.t.i.AbstractTwillService [localhost] [TwillContainerService] AbstractTwillService:shutDown(AbstractTwillService.java:190) - Service TwillContainerService with runId 3857c51e-9ba3-4a3c-af2b-5950522e1a50-0 shutdown completed
2015-08-12T09:31:08,864Z INFO o.a.t.i.ServiceMain [localhost] [main] ServiceMain:doMain(ServiceMain.java:100) - Service TwillContainerService [TERMINATED] completed.
2015-08-12T09:31:08,866Z INFO o.a.t.i.ServiceMain$TwillZKPathService [localhost] [TwillZKPathService STOPPING] ServiceMain$TwillZKPathService:shutDown(ServiceMain.java:284) - Removing container ZK path: localhost:2181/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/runnables/PageRankWorkflow/3857c51e-9ba3-4a3c-af2b-5950522e1a50-0
2015-08-12T09:31:10,517Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:handleCompleted(ApplicationMasterService.java:440) - Container container_e04_1438850569323_0121_01_000002 completed with COMPLETE:.
2015-08-12T09:31:10,524Z INFO o.a.t.i.a.RunningContainers [localhost] [ApplicationMasterService] RunningContainers:handleCompleted(RunningContainers.java:393) - Container container_e04_1438850569323_0121_01_000002 exited normally with state COMPLETE
2015-08-12T09:31:10,537Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:doRun(ApplicationMasterService.java:361) - All containers completed. Shutting down application master.
2015-08-12T09:31:10,542Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:doStop(ApplicationMasterService.java:237) - Stop application master with spec: {"name":"workflow.default.SparkPageRank.PageRankWorkflow","runnables":{"PageRankWorkflow":{"name":"PageRankWorkflow","runnable":{"classname":"co.cask.cdap.internal.app.runtime.distributed.WorkflowTwillRunnable","name":"PageRankWorkflow","arguments":{"hConf":"hConf.xml","cConf":"cConf.xml"}},"resources":{"cores":1,"memorySize":1024,"instances":1,"uplink":-1,"downlink":-1},"files":[{"name":"mr-framework","uri":"hdfs://localhost:8020/hdp/apps/2.2.4.2-2/mapreduce/mapreduce.tar.gz","lastModified":1438671719280,"size":192697367,"archive":true,"pattern":null},{"name":"cConf.xml","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/cConf.xml.9de269d5-520b-4874-a264-a115c34ac12a.xml","lastModified":1439371833272,"size":21745,"archive":false,"pattern":null},{"name":"spark-assembly-1.2.1.2.2.4.2-2-hadoop2.6.0.2.2.4.2-2.jar","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/spark-assembly-1.2.1.2.2.4.2-2-hadoop2.6.0.2.2.4.2-2.jar.311a8860-68b9-40fa-9d16-a8927794660d.jar","lastModified":1439371837244,"size":156197440,"archive":false,"pattern":null},{"name":"PageRankWorkflow656786471940694075.jar","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/PageRankWorkflow656786471940694075.jar.2d3a72bc-daf9-41d0-a27a-3ed0fdd74012.jar","lastModified":1439371833182,"size":24126,"archive":false,"pattern":null},{"name":"hConf.xml","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/hConf.xml.f00f0f1d-0bb9-4ac8-becb-76ecf2aaa7c8.xml","lastModified":1439371833211,"size":111268,"archive":false,"pattern":null},{"name":"launcher.jar","uri":"hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1/launcher.jar.40f2a3db-186d-46ab-be4e-b63d64f85412.jar","lastModified":1439371833236,"size":8617,"archive":false,"pattern":null}]}},"orders":[{"names":["PageRankWorkflow"],"type":"STARTED"}],"placementPolicies":[],"handler":{"classname":"co.cask.cdap.common.twill.AbortOnTimeoutEventHandler","configs":{"abortIfNotFull":"false","abortTime":"120000"}}}
2015-08-12T09:31:10,545Z INFO o.a.t.i.a.RunningContainers [localhost] [ApplicationMasterService] RunningContainers:stopAll(RunningContainers.java:332) - Stopping all instances of PageRankWorkflow
2015-08-12T09:31:10,546Z INFO o.a.t.i.a.RunningContainers [localhost] [ApplicationMasterService] RunningContainers:stopAll(RunningContainers.java:342) - Terminated all instances of PageRankWorkflow
2015-08-12T09:31:10,566Z INFO o.a.t.i.a.ApplicationMasterService [localhost] [ApplicationMasterService] ApplicationMasterService:cleanupDir(ApplicationMasterService.java:325) - Application directory deleted: hdfs://localhost:8020/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/2c468aae-86a1-4a91-8732-32f595f962f1
2015-08-12T09:31:10,567Z INFO o.a.t.i.AbstractTwillService [localhost] [ApplicationMasterService] AbstractTwillService:removeLiveNode(AbstractTwillService.java:209) - Remove live node localhost:2181/cdap/twill/workflow.default.SparkPageRank.PageRankWorkflow/instances/2c468aae-86a1-4a91-8732-32f595f962f1
2015-08-12T09:31:10,570Z INFO o.a.t.i.AbstractTwillService [localhost] [ApplicationMasterService] AbstractTwillService:shutDown(AbstractTwillService.java:190) - Service ApplicationMasterService with runId 2c468aae-86a1-4a91-8732-32f595f962f1 shutdown completed
2015-08-12T09:31:10,571Z INFO o.a.t.i.ServiceMain [localhost] [main] ServiceMain:doMain(ServiceMain.java:100) - Service ApplicationMasterService [TERMINATED] completed.
2015-08-12 15:01:13,412 INFO [Kafka-Consumer-log-0] consumer.SimpleConsumer: Reconnect due to socket error: Connection reset by peer
2015-08-12 15:01:13,598 - INFO [ STOPPING:c.c.c.i.a.r.d.AbstractTwillProgramController$2@70] - Twill program terminated: PageRankWorkflow 2c468aae-86a1-4a91-8732-32f595f962f1
2015-08-12T09:31:13,483Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439371873483
2015-08-12T09:31:13,485Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439371873483, readPointer=1439371873470000000, writePointer=1439371873470000000, invalidSize=52, inProgressSize=0, committingSize=0, committedSize=0}
2015-08-12T09:31:13,496Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439371873483
2015-08-12T09:31:13,501Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3085] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439371873483
2015-08-12T09:31:13,522Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439371873483
2015-08-12T09:31:13,530Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371332803
2015-08-12T09:31:13,537Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371332803
2015-08-12T09:32:13,557Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439371933557
2015-08-12T09:32:13,558Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439371933557, readPointer=1439371933541000000, writePointer=1439371933550000000, invalidSize=52, inProgressSize=1, committingSize=1, committedSize=0}
2015-08-12T09:32:13,572Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439371933557
2015-08-12T09:32:13,576Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3083] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439371933557
2015-08-12T09:32:13,602Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439371933557
2015-08-12T09:32:13,609Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371392875
2015-08-12T09:32:13,616Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371392875
2015-08-12T09:32:36,571Z INFO c.c.c.m.p.MetricmyssageCallback [hdp-3.xyz.co.in] [message-callback-0] MetricmyssageCallback:onReceived(MetricmyssageCallback.java:100) - 3366000 metrics records processed
2015-08-12T09:32:36,573Z INFO c.c.c.m.p.MetricmyssageCallback [hdp-3.xyz.co.in] [message-callback-0] MetricmyssageCallback:onReceived(MetricmyssageCallback.java:101) - Last record time: 1439371956
2015-08-12T09:33:13,640Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439371993640
2015-08-12T09:33:13,641Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439371993640, readPointer=1439371993625000000, writePointer=1439371993637000000, invalidSize=52, inProgressSize=1, committingSize=0, committedSize=0}
2015-08-12T09:33:13,653Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439371993640
2015-08-12T09:33:13,658Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3083] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439371993640
2015-08-12T09:33:13,684Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439371993640
2015-08-12T09:33:13,691Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371452956
2015-08-12T09:33:13,698Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371452956
2015-08-12T09:34:13,712Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439372053712
2015-08-12T09:34:13,713Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439372053712, readPointer=1439372053697000000, writePointer=1439372053710000000, invalidSize=52, inProgressSize=1, committingSize=0, committedSize=0}
2015-08-12T09:34:13,727Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439372053712
2015-08-12T09:34:13,731Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3083] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439372053712
2015-08-12T09:34:13,752Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439372053712
2015-08-12T09:34:13,761Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371513038
2015-08-12T09:34:13,767Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371513038
2015-08-12 15:04:41,280 INFO [metrics-collection] client.ClientUtils$: Fetching metadata from broker id:0,host:hdp-4.xyz.co.in,port:9092 with correlation id 1102497 for 1 topic(s) Set(metrics)
2015-08-12 15:04:41,281 INFO [metrics-collection] producer.SyncProducer: Connected to hdp-4.xyz.co.in:9092 for producing
2015-08-12 15:04:41,284 INFO [metrics-collection] producer.SyncProducer: Disconnecting from hdp-4.xyz.co.in:9092
2015-08-12 15:04:41,284 INFO [metrics-collection] producer.SyncProducer: Disconnecting from hdp-4.xyz.co.in:9092
2015-08-12 15:04:41,285 INFO [metrics-collection] producer.SyncProducer: Connected to hdp-4.xyz.co.in:9092 for producing
2015-08-12T09:35:13,789Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439372113789
2015-08-12T09:35:13,790Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439372113789, readPointer=1439372113641000000, writePointer=1439372113641000000, invalidSize=52, inProgressSize=0, committingSize=0, committedSize=0}
2015-08-12T09:35:13,804Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439372113789
2015-08-12T09:35:13,830Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439372113789
2015-08-12T09:35:13,838Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371573112
2015-08-12T09:35:13,845Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371573112
2015-08-12T09:35:13,976Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3083] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439372113789
2015-08-12T09:36:13,864Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439372173863
2015-08-12T09:36:13,865Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439372173863, readPointer=1439372173779000000, writePointer=1439372173779000000, invalidSize=52, inProgressSize=0, committingSize=0, committedSize=0}
2015-08-12T09:36:13,878Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439372173863
2015-08-12T09:36:13,906Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439372173863
2015-08-12T09:36:13,909Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3085] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439372173863
2015-08-12T09:36:13,914Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371633184
2015-08-12T09:36:13,921Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371633184
2015-08-12T09:37:13,942Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:393) - Starting snapshot of transaction state with timestamp 1439372233941
2015-08-12T09:37:13,943Z INFO c.c.t.TransactionManager [hdp-3.xyz.co.in] [tx-snapshot] TransactionManager:getSnapshot(TransactionManager.java:394) - Returning snapshot of state: TransactionSnapshot{timestamp=1439372233941, readPointer=1439372233923000000, writePointer=1439372233938000000, invalidSize=52, inProgressSize=1, committingSize=0, committedSize=0}
2015-08-12T09:37:13,957Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:123) - Writing snapshot to temporary file /cdap/tx.snapshot/.in-progress.snapshot.1439372233941
2015-08-12T09:37:13,962Z INFO c.c.t.p.HDFSTransactionLog [hdp-3.xyz.co.in] [TTransactionServer-rpc-3085] HDFSTransactionLog$LogWriter:<init>(HDFSTransactionLog.java:117) - Created a new TransactionLog writer for /cdap/tx.snapshot/txlog.1439372233941
2015-08-12T09:37:13,983Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:writeSnapshot(HDFSTransactionStateStorage.java:136) - Completed snapshot to file /cdap/tx.snapshot/snapshot.1439372233941
2015-08-12T09:37:13,990Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteOldSnapshots(HDFSTransactionStateStorage.java:209) - Removed 1 old snapshot files prior to 1439371693249
2015-08-12T09:37:13,997Z INFO c.c.t.p.HDFSTransactionStateStorage [hdp-3.xyz.co.in] [tx-snapshot] HDFSTransactionStateStorage:deleteLogsOlderThan(HDFSTransactionStateStorage.java:263) - Removed 1 transaction logs older than 1439371693249
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2. Output of the find /data/yarn/local/usercache/cdap/appcache/<application_id> -name "*.jar" command.
find /data/yarn/local/usercache/cdap/appcache/application_1438850569323_0127 -name "*.jar"
find: `/data/yarn/local/usercache/cdap/appcache/application_1438850569323_0127': No such file or directory
3. pom.xml of the application
Same as available in cdap distribution @ cdap-sdk-3.1.0/examples/SparkPageRank/
4. What is the hadoop version you are using?
[amit@hdp-4 SparkPageRank]$ hadoop version
Hadoop 2.6.0.2.2.4.2-2
Subversion hortonworks/hadoop.git -r 22a563ebe448969d07902aed869ac13c652b2872
Compiled by jenkins on 2015-03-31T19:49Z
Compiled with protoc 2.5.0
From source with checksum b3481c2cdbe2d181f2621331926e267
This command was run using /usr/hdp/2.2.4.2-2/hadoop/hadoop-common-2.6.0.2.2.4.2-2.jar
chris
Bhooshan Mogal
--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/477e3482-0ee9-4f98-902f-88bc379e3dc4%40googlegroups.com.
Bhooshan Mogal
Bhooshan Mogal
Bhooshan
Bhooshan Mogal
On Wed, Aug 19, 2015 at 12:06 AM, Bhooshan Mogal <bhoo...@cask.co> wrote:
Hi Amit,I just sent out a release announcement for CDAP 3.1.1 in a separate email thread. It contains the fix for this issue. Please try it out and let us know your feedback.Thanks,
BhooshanOn Mon, Aug 17, 2015 at 10:02 PM, Amit Kulkarni <amit...@gmail.com> wrote:Hi Bhooshan,Can we get any idea about when the next version will be available? Is it possible to get patch for the fix meanwhile?Thanks,Amit