SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/spark16-516/lib/spark-assembly-1.6.2-SNAPSHOT-hadoop2.6.0-cdh5.5.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/avro-tools-1.7.6-cdh5.5.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/05/24 12:06:41 WARN SparkConf: 
SPARK_CLASSPATH was detected (set to '/opt/alluxio/alluxio-keith.jar:').
This is deprecated in Spark 1.0+.

Please instead use:
 - ./spark-submit with --driver-class-path to augment the driver classpath
 - spark.executor.extraClassPath to augment the executor classpath
        
16/05/24 12:06:41 WARN SparkConf: Setting 'spark.executor.extraClassPath' to '/opt/alluxio/alluxio-keith.jar:' as a work-around.
16/05/24 12:06:41 WARN SparkConf: Setting 'spark.driver.extraClassPath' to '/opt/alluxio/alluxio-keith.jar:' as a work-around.
16/05/24 12:06:42 WARN Utils: Service 'SparkUI' could not bind on port 56040. Attempting port 56041.
16/05/24 12:06:42 WARN Utils: Service 'SparkUI' could not bind on port 56041. Attempting port 56042.
Spark context available as sc.
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-core-3.2.10.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-api-jdo-3.2.6.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-rdbms-3.2.9.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16-516/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-rdbms-3.2.9.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16-516/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-api-jdo-3.2.6.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-rdbms-3.2.9.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-core-3.2.10.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-api-jdo-3.2.6.jar."
16/05/24 12:06:51 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16-516/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-core-3.2.10.jar."
16/05/24 12:07:07 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0
16/05/24 12:07:07 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-core-3.2.10.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-api-jdo-3.2.6.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-rdbms-3.2.9.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16-516/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-rdbms-3.2.9.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16-516/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-api-jdo-3.2.6.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH/lib/hive/lib/datanucleus-rdbms-3.2.9.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-core-3.2.10.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.5.2-1.cdh5.5.2.p0.4/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-api-jdo-3.2.6.jar."
16/05/24 12:07:10 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark16-516/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark16/lib/datanucleus-core-3.2.10.jar."
SQL context available as sqlContext.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.6.1
      /_/
         
Using Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_91)
Type in expressions to have them evaluated.
Type :help for more information.

scala> sc.setLogLevel("DEBUG")

scala> 16/05/24 12:07:32 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #93
16/05/24 12:07:32 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #93
16/05/24 12:07:32 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
.parquet")lContext.read.parquet("alluxio://dcsac035:19998/user/keith/test 
16/05/24 12:07:33 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #94
16/05/24 12:07:33 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #94
16/05/24 12:07:33 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:33 INFO type: initialize(alluxio://dcsac035:19998/user/keith/test.parquet, Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml). Connecting to Alluxio: alluxio://dcsac035:19998/user/keith/test.parquet
16/05/24 12:07:33 INFO type: alluxio://dcsac035:19998 alluxio://dcsac035:19998
16/05/24 12:07:33 INFO type: Loading Alluxio properties from Hadoop configuration: {}
16/05/24 12:07:33 DEBUG type: address: dcsac021/10.5.151.121 isLoopbackAddress: false, with host 10.5.151.121 dcsac021
16/05/24 12:07:33 INFO type: getWorkingDirectory: /
16/05/24 12:07:33 INFO type: getWorkingDirectory: /
16/05/24 12:07:33 INFO ParquetRelation: Listing alluxio://dcsac035:19998/user/keith/test.parquet on driver
16/05/24 12:07:33 INFO type: listStatus(alluxio://dcsac035:19998/user/keith/test.parquet)
16/05/24 12:07:33 INFO type: Alluxio client (version 1.1.0-RC2) is trying to connect with FileSystemMasterClient master @ dcsac035/10.5.151.135:19998
16/05/24 12:07:33 INFO type: Client registered with FileSystemMasterClient master @ dcsac035/10.5.151.135:19998
16/05/24 12:07:33 DEBUG ClosureCleaner: +++ Cleaning closure <function1> (org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27) +++
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared fields: 5
16/05/24 12:07:34 DEBUG ClosureCleaner:      public static final long org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27.serialVersionUID
16/05/24 12:07:34 DEBUG ClosureCleaner:      private final boolean org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27.assumeBinaryIsString$2
16/05/24 12:07:34 DEBUG ClosureCleaner:      private final boolean org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27.assumeInt96IsTimestamp$2
16/05/24 12:07:34 DEBUG ClosureCleaner:      private final boolean org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27.writeLegacyParquetFormat$1
16/05/24 12:07:34 DEBUG ClosureCleaner:      private final org.apache.spark.util.SerializableConfiguration org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27.serializedConf$1
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared methods: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      public final java.lang.Object org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27.apply(java.lang.Object)
16/05/24 12:07:34 DEBUG ClosureCleaner:      public final scala.collection.Iterator org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27.apply(scala.collection.Iterator)
16/05/24 12:07:34 DEBUG ClosureCleaner:  + inner classes: 3
16/05/24 12:07:34 DEBUG ClosureCleaner:      org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27$$anonfun$28
16/05/24 12:07:34 DEBUG ClosureCleaner:      org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27$$anonfun$apply$8
16/05/24 12:07:34 DEBUG ClosureCleaner:      org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27$$anonfun$apply$9
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer classes: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer objects: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + populating accessed fields because this is the starting closure
16/05/24 12:07:34 DEBUG ClosureCleaner:  + fields accessed by starting closure: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + there are no enclosing objects!
16/05/24 12:07:34 DEBUG ClosureCleaner:  +++ closure <function1> (org.apache.spark.sql.execution.datasources.parquet.ParquetRelation$$anonfun$27) is now cleaned +++
16/05/24 12:07:34 DEBUG ClosureCleaner: +++ Cleaning closure <function1> (org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12) +++
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared fields: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      public static final long org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.serialVersionUID
16/05/24 12:07:34 DEBUG ClosureCleaner:      private final org.apache.spark.rdd.RDD$$anonfun$collect$1 org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.$outer
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared methods: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      public final java.lang.Object org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(java.lang.Object)
16/05/24 12:07:34 DEBUG ClosureCleaner:      public final java.lang.Object org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(scala.collection.Iterator)
16/05/24 12:07:34 DEBUG ClosureCleaner:  + inner classes: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer classes: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      org.apache.spark.rdd.RDD$$anonfun$collect$1
16/05/24 12:07:34 DEBUG ClosureCleaner:      org.apache.spark.rdd.RDD
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer objects: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      <function0>
16/05/24 12:07:34 DEBUG ClosureCleaner:      MapPartitionsRDD[1] at parquet at <console>:22
16/05/24 12:07:34 DEBUG ClosureCleaner:  + populating accessed fields because this is the starting closure
16/05/24 12:07:34 DEBUG ClosureCleaner:  + fields accessed by starting closure: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      (class org.apache.spark.rdd.RDD,Set(org$apache$spark$rdd$RDD$$evidence$1))
16/05/24 12:07:34 DEBUG ClosureCleaner:      (class org.apache.spark.rdd.RDD$$anonfun$collect$1,Set($outer))
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outermost object is not a closure, so do not clone it: (class org.apache.spark.rdd.RDD,MapPartitionsRDD[1] at parquet at <console>:22)
16/05/24 12:07:34 DEBUG ClosureCleaner:  + cloning the object <function0> of class org.apache.spark.rdd.RDD$$anonfun$collect$1
16/05/24 12:07:34 DEBUG ClosureCleaner:  + cleaning cloned closure <function0> recursively (org.apache.spark.rdd.RDD$$anonfun$collect$1)
16/05/24 12:07:34 DEBUG ClosureCleaner: +++ Cleaning closure <function0> (org.apache.spark.rdd.RDD$$anonfun$collect$1) +++
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared fields: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      public static final long org.apache.spark.rdd.RDD$$anonfun$collect$1.serialVersionUID
16/05/24 12:07:34 DEBUG ClosureCleaner:      private final org.apache.spark.rdd.RDD org.apache.spark.rdd.RDD$$anonfun$collect$1.$outer
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared methods: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      public org.apache.spark.rdd.RDD org.apache.spark.rdd.RDD$$anonfun$collect$1.org$apache$spark$rdd$RDD$$anonfun$$$outer()
16/05/24 12:07:34 DEBUG ClosureCleaner:      public final java.lang.Object org.apache.spark.rdd.RDD$$anonfun$collect$1.apply()
16/05/24 12:07:34 DEBUG ClosureCleaner:  + inner classes: 1
16/05/24 12:07:34 DEBUG ClosureCleaner:      org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer classes: 1
16/05/24 12:07:34 DEBUG ClosureCleaner:      org.apache.spark.rdd.RDD
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer objects: 1
16/05/24 12:07:34 DEBUG ClosureCleaner:      MapPartitionsRDD[1] at parquet at <console>:22
16/05/24 12:07:34 DEBUG ClosureCleaner:  + fields accessed by starting closure: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      (class org.apache.spark.rdd.RDD,Set(org$apache$spark$rdd$RDD$$evidence$1))
16/05/24 12:07:34 DEBUG ClosureCleaner:      (class org.apache.spark.rdd.RDD$$anonfun$collect$1,Set($outer))
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outermost object is not a closure, so do not clone it: (class org.apache.spark.rdd.RDD,MapPartitionsRDD[1] at parquet at <console>:22)
16/05/24 12:07:34 DEBUG ClosureCleaner:  +++ closure <function0> (org.apache.spark.rdd.RDD$$anonfun$collect$1) is now cleaned +++
16/05/24 12:07:34 DEBUG ClosureCleaner:  +++ closure <function1> (org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12) is now cleaned +++
16/05/24 12:07:34 DEBUG ClosureCleaner: +++ Cleaning closure <function2> (org.apache.spark.SparkContext$$anonfun$runJob$5) +++
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared fields: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      public static final long org.apache.spark.SparkContext$$anonfun$runJob$5.serialVersionUID
16/05/24 12:07:34 DEBUG ClosureCleaner:      private final scala.Function1 org.apache.spark.SparkContext$$anonfun$runJob$5.cleanedFunc$1
16/05/24 12:07:34 DEBUG ClosureCleaner:  + declared methods: 2
16/05/24 12:07:34 DEBUG ClosureCleaner:      public final java.lang.Object org.apache.spark.SparkContext$$anonfun$runJob$5.apply(java.lang.Object,java.lang.Object)
16/05/24 12:07:34 DEBUG ClosureCleaner:      public final java.lang.Object org.apache.spark.SparkContext$$anonfun$runJob$5.apply(org.apache.spark.TaskContext,scala.collection.Iterator)
16/05/24 12:07:34 DEBUG ClosureCleaner:  + inner classes: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer classes: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + outer objects: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + populating accessed fields because this is the starting closure
16/05/24 12:07:34 DEBUG ClosureCleaner:  + fields accessed by starting closure: 0
16/05/24 12:07:34 DEBUG ClosureCleaner:  + there are no enclosing objects!
16/05/24 12:07:34 DEBUG ClosureCleaner:  +++ closure <function2> (org.apache.spark.SparkContext$$anonfun$runJob$5) is now cleaned +++
16/05/24 12:07:34 INFO SparkContext: Starting job: parquet at <console>:22
16/05/24 12:07:34 INFO DAGScheduler: Got job 0 (parquet at <console>:22) with 8 output partitions
16/05/24 12:07:34 INFO DAGScheduler: Final stage: ResultStage 0 (parquet at <console>:22)
16/05/24 12:07:34 INFO DAGScheduler: Parents of final stage: List()
16/05/24 12:07:34 INFO DAGScheduler: Missing parents: List()
16/05/24 12:07:34 DEBUG DAGScheduler: submitStage(ResultStage 0)
16/05/24 12:07:34 DEBUG DAGScheduler: missing: List()
16/05/24 12:07:34 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[1] at parquet at <console>:22), which has no missing parents
16/05/24 12:07:34 DEBUG DAGScheduler: submitMissingTasks(ResultStage 0)
16/05/24 12:07:34 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=19, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=96768
16/05/24 12:07:34 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 99446 lastFlushOffset 97233
16/05/24 12:07:34 DEBUG DFSClient: Queued packet 19
16/05/24 12:07:34 DEBUG DFSClient: Waiting for ack for: 19
16/05/24 12:07:34 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 19 offsetInBlock: 96768 lastPacketInBlock: false lastByteOffsetInBlock: 99446
16/05/24 12:07:34 DEBUG DFSClient: DFSClient seqno: 19 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 727728
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Starting timer to add executors because pending tasks are building up (to expire in 1 seconds)
16/05/24 12:07:34 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 67.3 KB, free 67.3 KB)
16/05/24 12:07:34 DEBUG BlockManager: Put block broadcast_0 locally took  29 ms
16/05/24 12:07:34 DEBUG BlockManager: Putting block broadcast_0 without replication took  30 ms
[Stage 0:>                                                          (0 + 0) / 8]16/05/24 12:07:34 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 23.7 KB, free 91.1 KB)
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.121:39760 (size: 23.7 KB, free: 5.5 GB)
16/05/24 12:07:34 DEBUG BlockManagerMaster: Updated info of block broadcast_0_piece0
16/05/24 12:07:34 DEBUG BlockManager: Told master about block broadcast_0_piece0
16/05/24 12:07:34 DEBUG BlockManager: Put block broadcast_0_piece0 locally took  3 ms
16/05/24 12:07:34 DEBUG BlockManager: Putting block broadcast_0_piece0 without replication took  3 ms
16/05/24 12:07:34 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1006
16/05/24 12:07:34 INFO DAGScheduler: Submitting 8 missing tasks from ResultStage 0 (MapPartitionsRDD[1] at parquet at <console>:22)
16/05/24 12:07:34 DEBUG DAGScheduler: New pending partitions: Set(0, 1, 5, 2, 6, 3, 7, 4)
16/05/24 12:07:34 INFO YarnScheduler: Adding task set 0.0 with 8 tasks
16/05/24 12:07:34 DEBUG TaskSetManager: Epoch for TaskSet 0.0: 0
16/05/24 12:07:34 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: NO_PREF, ANY
16/05/24 12:07:34 INFO FairSchedulableBuilder: Added task set TaskSet_0 tasks to pool default
16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 0
16/05/24 12:07:34 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, 10.5.151.134, partition 0,PROCESS_LOCAL, 1894 bytes)
16/05/24 12:07:34 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, 10.5.151.123, partition 1,PROCESS_LOCAL, 1894 bytes)
16/05/24 12:07:34 INFO TaskSetManager: Starting task 2.0 in stage 0.0 (TID 2, 10.5.151.129, partition 2,PROCESS_LOCAL, 1894 bytes)
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 8 because it is now running a task
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 4 because it is now running a task
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 3 because it is now running a task
16/05/24 12:07:34 INFO TaskSetManager: Starting task 3.0 in stage 0.0 (TID 3, 10.5.151.125, partition 3,PROCESS_LOCAL, 1894 bytes)
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 7 because it is now running a task
16/05/24 12:07:34 INFO TaskSetManager: Starting task 4.0 in stage 0.0 (TID 4, 10.5.151.127, partition 4,PROCESS_LOCAL, 1894 bytes)
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 6 because it is now running a task
16/05/24 12:07:34 INFO TaskSetManager: Starting task 5.0 in stage 0.0 (TID 5, 10.5.151.130, partition 5,PROCESS_LOCAL, 1894 bytes)
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 5 because it is now running a task
16/05/24 12:07:34 INFO TaskSetManager: Starting task 6.0 in stage 0.0 (TID 6, 10.5.151.133, partition 6,PROCESS_LOCAL, 1894 bytes)
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 2 because it is now running a task
16/05/24 12:07:34 INFO TaskSetManager: Starting task 7.0 in stage 0.0 (TID 7, 10.5.151.126, partition 7,PROCESS_LOCAL, 1968 bytes)
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing timer to add executors because there are no more pending tasks
16/05/24 12:07:34 DEBUG ExecutorAllocationManager: Clearing idle timer for 1 because it is now running a task
16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 8
16/05/24 12:07:34 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #95
16/05/24 12:07:34 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #95
16/05/24 12:07:34 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.134:47769 (size: 23.7 KB, free: 2.7 GB)
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.129:50767 (size: 23.7 KB, free: 2.7 GB)
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.127:43177 (size: 23.7 KB, free: 2.7 GB)
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.130:52204 (size: 23.7 KB, free: 2.7 GB)
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.126:38780 (size: 23.7 KB, free: 2.7 GB)
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.123:39707 (size: 23.7 KB, free: 2.7 GB)
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.125:45433 (size: 23.7 KB, free: 2.7 GB)
16/05/24 12:07:34 DEBUG BlockManager: Level for block broadcast_0_piece0 is StorageLevel(true, true, false, false, 1)
16/05/24 12:07:34 DEBUG BlockManager: Getting block broadcast_0_piece0 from memory
16/05/24 12:07:34 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 10.5.151.133:46700 (size: 23.7 KB, free: 2.7 GB)
[Stage 0:>                                                          (0 + 8) / 8]16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 7
16/05/24 12:07:34 DEBUG TaskSetManager: No tasks for locality level NO_PREF, so moving to locality level ANY
16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 6
16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 5
16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 4
16/05/24 12:07:34 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 619 ms on 10.5.151.134 (1/8)
16/05/24 12:07:34 INFO TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 587 ms on 10.5.151.129 (2/8)
16/05/24 12:07:34 INFO TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 577 ms on 10.5.151.127 (3/8)
16/05/24 12:07:34 INFO TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 572 ms on 10.5.151.130 (4/8)
16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 3
16/05/24 12:07:34 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 604 ms on 10.5.151.123 (5/8)
16/05/24 12:07:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 2
16/05/24 12:07:34 INFO TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 604 ms on 10.5.151.125 (6/8)
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Starting idle timer for 8 because there are no more tasks scheduled to run on the executor (to expire in 60 seconds)
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Starting idle timer for 3 because there are no more tasks scheduled to run on the executor (to expire in 60 seconds)
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Starting idle timer for 6 because there are no more tasks scheduled to run on the executor (to expire in 60 seconds)
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Starting idle timer for 5 because there are no more tasks scheduled to run on the executor (to expire in 60 seconds)
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Starting idle timer for 4 because there are no more tasks scheduled to run on the executor (to expire in 60 seconds)
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Starting idle timer for 7 because there are no more tasks scheduled to run on the executor (to expire in 60 seconds)
16/05/24 12:07:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:35 INFO TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 643 ms on 10.5.151.133 (7/8)
[Stage 0:===================================================>       (7 + 1) / 8]16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Lowering target number of executors to 2 (previously 8) because not all requested executors are actually needed
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Starting idle timer for 2 because there are no more tasks scheduled to run on the executor (to expire in 60 seconds)
16/05/24 12:07:35 DEBUG ExecutorAllocationManager: Lowering target number of executors to 1 (previously 2) because not all requested executors are actually needed
16/05/24 12:07:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:35 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #96
16/05/24 12:07:35 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #96
16/05/24 12:07:35 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:36 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:36 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #97
16/05/24 12:07:36 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #97
16/05/24 12:07:36 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:37 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:37 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #98
16/05/24 12:07:37 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #98
16/05/24 12:07:37 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:38 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:38 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #99
16/05/24 12:07:38 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #99
16/05/24 12:07:38 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:39 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:39 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #100
16/05/24 12:07:39 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #100
16/05/24 12:07:39 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:40 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:40 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #101
16/05/24 12:07:40 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #101
16/05/24 12:07:40 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:41 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:41 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #102
16/05/24 12:07:41 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #102
16/05/24 12:07:41 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:42 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:42 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #103
16/05/24 12:07:42 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #103
16/05/24 12:07:42 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:43 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:43 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #104
16/05/24 12:07:43 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #104
16/05/24 12:07:43 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:44 DEBUG Client: The ping interval is 60000 ms.
16/05/24 12:07:44 DEBUG Client: Connecting to dcsac022/10.5.151.122:8020
16/05/24 12:07:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: starting, having connections 2
16/05/24 12:07:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #105
16/05/24 12:07:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #105
16/05/24 12:07:44 DEBUG ProtobufRpcEngine: Call: renewLease took 5ms
16/05/24 12:07:44 DEBUG LeaseRenewer: Lease renewed for client DFSClient_NONMAPREDUCE_-1596888490_1
16/05/24 12:07:44 DEBUG LeaseRenewer: Lease renewer daemon for [DFSClient_NONMAPREDUCE_-1596888490_1] with renew id 1 executed
16/05/24 12:07:44 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #106
16/05/24 12:07:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #106
16/05/24 12:07:44 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:45 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:45 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #107
16/05/24 12:07:45 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #107
16/05/24 12:07:45 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:46 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:46 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #108
16/05/24 12:07:46 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #108
16/05/24 12:07:46 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:47 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:47 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #109
16/05/24 12:07:47 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #109
16/05/24 12:07:47 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:48 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:48 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #110
16/05/24 12:07:48 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #110
16/05/24 12:07:48 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:49 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:49 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #111
16/05/24 12:07:49 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #111
16/05/24 12:07:49 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:50 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:50 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #112
16/05/24 12:07:50 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #112
16/05/24 12:07:50 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:51 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:51 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #113
16/05/24 12:07:51 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #113
16/05/24 12:07:51 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:52 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:52 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #114
16/05/24 12:07:52 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #114
16/05/24 12:07:52 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:53 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:53 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #115
16/05/24 12:07:53 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #115
16/05/24 12:07:53 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: closed
16/05/24 12:07:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: stopped, remaining connections 1
16/05/24 12:07:54 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #116
16/05/24 12:07:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #116
16/05/24 12:07:54 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:55 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:55 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #117
16/05/24 12:07:55 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #117
16/05/24 12:07:55 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:56 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:56 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #118
16/05/24 12:07:56 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #118
16/05/24 12:07:56 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:57 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:57 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #119
16/05/24 12:07:57 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #119
16/05/24 12:07:57 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:07:58 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:58 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #120
16/05/24 12:07:58 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #120
16/05/24 12:07:58 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:07:59 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:07:59 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #121
16/05/24 12:07:59 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #121
16/05/24 12:07:59 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:00 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:00 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #122
16/05/24 12:08:00 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #122
16/05/24 12:08:00 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:01 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:01 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #123
16/05/24 12:08:01 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #123
16/05/24 12:08:01 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 3ms
16/05/24 12:08:02 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:02 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #124
16/05/24 12:08:02 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #124
16/05/24 12:08:02 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:03 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #125
16/05/24 12:08:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #125
16/05/24 12:08:03 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:04 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: -1 offsetInBlock: 0 lastPacketInBlock: false lastByteOffsetInBlock: 0
16/05/24 12:08:04 DEBUG DFSClient: DFSClient seqno: -1 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 712483
16/05/24 12:08:04 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:04 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #126
16/05/24 12:08:04 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #126
16/05/24 12:08:04 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:05 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:05 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #127
16/05/24 12:08:05 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #127
16/05/24 12:08:05 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 4ms
16/05/24 12:08:06 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:06 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #128
16/05/24 12:08:06 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #128
16/05/24 12:08:06 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:07 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:07 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #129
16/05/24 12:08:07 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #129
16/05/24 12:08:07 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:08 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:08 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #130
16/05/24 12:08:08 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #130
16/05/24 12:08:08 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:09 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:09 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #131
16/05/24 12:08:09 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #131
16/05/24 12:08:09 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:10 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:10 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #132
16/05/24 12:08:10 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #132
16/05/24 12:08:10 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:11 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:11 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #133
16/05/24 12:08:11 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #133
16/05/24 12:08:11 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:12 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:12 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #134
16/05/24 12:08:12 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #134
16/05/24 12:08:12 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:13 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:13 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #135
16/05/24 12:08:13 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #135
16/05/24 12:08:13 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:14 DEBUG Client: The ping interval is 60000 ms.
16/05/24 12:08:14 DEBUG Client: Connecting to dcsac022/10.5.151.122:8020
16/05/24 12:08:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: starting, having connections 2
16/05/24 12:08:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #136
16/05/24 12:08:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #136
16/05/24 12:08:14 DEBUG ProtobufRpcEngine: Call: renewLease took 4ms
16/05/24 12:08:14 DEBUG LeaseRenewer: Lease renewed for client DFSClient_NONMAPREDUCE_-1596888490_1
16/05/24 12:08:14 DEBUG LeaseRenewer: Lease renewer daemon for [DFSClient_NONMAPREDUCE_-1596888490_1] with renew id 1 executed
16/05/24 12:08:14 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #137
16/05/24 12:08:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #137
16/05/24 12:08:14 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:15 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:15 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #138
16/05/24 12:08:15 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #138
16/05/24 12:08:15 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:16 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:16 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #139
16/05/24 12:08:16 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #139
16/05/24 12:08:16 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:17 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:17 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #140
16/05/24 12:08:17 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #140
16/05/24 12:08:17 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:18 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:18 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #141
16/05/24 12:08:18 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #141
16/05/24 12:08:18 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:19 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:19 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #142
16/05/24 12:08:19 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #142
16/05/24 12:08:19 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:20 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:20 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #143
16/05/24 12:08:20 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #143
16/05/24 12:08:20 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:21 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:21 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #144
16/05/24 12:08:21 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #144
16/05/24 12:08:21 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:22 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:22 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #145
16/05/24 12:08:22 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #145
16/05/24 12:08:22 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:23 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:23 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #146
16/05/24 12:08:23 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #146
16/05/24 12:08:23 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: closed
16/05/24 12:08:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: stopped, remaining connections 1
16/05/24 12:08:24 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #147
16/05/24 12:08:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #147
16/05/24 12:08:24 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:25 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:25 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #148
16/05/24 12:08:25 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #148
16/05/24 12:08:25 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:26 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:26 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #149
16/05/24 12:08:26 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #149
16/05/24 12:08:26 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:27 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:27 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #150
16/05/24 12:08:27 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #150
16/05/24 12:08:27 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:28 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:28 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #151
16/05/24 12:08:28 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #151
16/05/24 12:08:28 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:29 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:29 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #152
16/05/24 12:08:29 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #152
16/05/24 12:08:29 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:30 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:30 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #153
16/05/24 12:08:30 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #153
16/05/24 12:08:30 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:31 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:31 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #154
16/05/24 12:08:31 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #154
16/05/24 12:08:31 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:32 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:32 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #155
16/05/24 12:08:32 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #155
16/05/24 12:08:32 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:33 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:33 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #156
16/05/24 12:08:33 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #156
16/05/24 12:08:33 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:34 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: -1 offsetInBlock: 0 lastPacketInBlock: false lastByteOffsetInBlock: 0
16/05/24 12:08:34 DEBUG DFSClient: DFSClient seqno: -1 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 636844
16/05/24 12:08:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:34 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #157
16/05/24 12:08:34 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #157
16/05/24 12:08:34 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 5
16/05/24 12:08:35 INFO ExecutorAllocationManager: Removing executor 5 because it has been idle for 60 seconds (new desired total will be 7)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 8
[Stage 0:===================================================>       (7 + 1) / 8]16/05/24 12:08:35 INFO ExecutorAllocationManager: Removing executor 8 because it has been idle for 60 seconds (new desired total will be 6)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 7
16/05/24 12:08:35 INFO ExecutorAllocationManager: Removing executor 7 because it has been idle for 60 seconds (new desired total will be 5)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 4
16/05/24 12:08:35 INFO ExecutorAllocationManager: Removing executor 4 because it has been idle for 60 seconds (new desired total will be 4)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 6
16/05/24 12:08:35 INFO ExecutorAllocationManager: Removing executor 6 because it has been idle for 60 seconds (new desired total will be 3)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 3
16/05/24 12:08:35 INFO ExecutorAllocationManager: Removing executor 3 because it has been idle for 60 seconds (new desired total will be 2)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Requesting to kill executor(s) 2
16/05/24 12:08:35 INFO ExecutorAllocationManager: Removing executor 2 because it has been idle for 60 seconds (new desired total will be 1)
16/05/24 12:08:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Disabling executor 5.
16/05/24 12:08:35 DEBUG YarnScheduler: Executor 5 on 10.5.151.130 lost, but reason not yet known.
16/05/24 12:08:35 INFO DAGScheduler: Executor lost: 5 (epoch 0)
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Trying to remove executor 5 from BlockManagerMaster.
16/05/24 12:08:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(5, 10.5.151.130, 52204)
16/05/24 12:08:35 INFO BlockManagerMaster: Removed 5 successfully in removeExecutor
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=20, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=99328
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 108318 lastFlushOffset 99446
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 20
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 20
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 20 offsetInBlock: 99328 lastPacketInBlock: false lastByteOffsetInBlock: 108318
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 20 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 660972
16/05/24 12:08:35 INFO YarnScheduler: Executor 5 on 10.5.151.130 killed by driver.
16/05/24 12:08:35 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: ANY
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=21, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=108032
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 108513 lastFlushOffset 108318
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 21
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 21
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 21 offsetInBlock: 108032 lastPacketInBlock: false lastByteOffsetInBlock: 108513
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 21 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 507763
16/05/24 12:08:35 INFO ExecutorAllocationManager: Existing executor 5 has been removed (new total is 7)
16/05/24 12:08:35 DEBUG ExecutorAllocationManager: Executor 5 is no longer pending to be removed (6 left)
16/05/24 12:08:35 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #158
16/05/24 12:08:35 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #158
16/05/24 12:08:35 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Disabling executor 7.
16/05/24 12:08:35 DEBUG YarnScheduler: Executor 7 on 10.5.151.125 lost, but reason not yet known.
16/05/24 12:08:35 INFO DAGScheduler: Executor lost: 7 (epoch 0)
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Trying to remove executor 7 from BlockManagerMaster.
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(7, 10.5.151.125, 45433)
16/05/24 12:08:35 INFO BlockManagerMaster: Removed 7 successfully in removeExecutor
16/05/24 12:08:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=22, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=108032
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 108658 lastFlushOffset 108513
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 22
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 22
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 22 offsetInBlock: 108032 lastPacketInBlock: false lastByteOffsetInBlock: 108658
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 22 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 511270
16/05/24 12:08:35 INFO YarnScheduler: Executor 7 on 10.5.151.125 killed by driver.
16/05/24 12:08:35 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: ANY
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=23, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=108544
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 108853 lastFlushOffset 108658
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 23
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 23
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 23 offsetInBlock: 108544 lastPacketInBlock: false lastByteOffsetInBlock: 108853
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 23 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 495673
16/05/24 12:08:35 INFO ExecutorAllocationManager: Existing executor 7 has been removed (new total is 6)
16/05/24 12:08:35 DEBUG ExecutorAllocationManager: Executor 7 is no longer pending to be removed (5 left)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Disabling executor 2.
16/05/24 12:08:35 DEBUG YarnScheduler: Executor 2 on 10.5.151.133 lost, but reason not yet known.
16/05/24 12:08:35 INFO DAGScheduler: Executor lost: 2 (epoch 0)
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Trying to remove executor 2 from BlockManagerMaster.
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(2, 10.5.151.133, 46700)
16/05/24 12:08:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:35 INFO BlockManagerMaster: Removed 2 successfully in removeExecutor
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=24, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=108544
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 108998 lastFlushOffset 108853
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 24
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 24
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 24 offsetInBlock: 108544 lastPacketInBlock: false lastByteOffsetInBlock: 108998
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 24 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 524177
16/05/24 12:08:35 INFO YarnScheduler: Executor 2 on 10.5.151.133 killed by driver.
16/05/24 12:08:35 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: ANY
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=25, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=108544
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 109193 lastFlushOffset 108998
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 25
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 25
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 25 offsetInBlock: 108544 lastPacketInBlock: false lastByteOffsetInBlock: 109193
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 25 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 498982
16/05/24 12:08:35 INFO ExecutorAllocationManager: Existing executor 2 has been removed (new total is 5)
16/05/24 12:08:35 DEBUG ExecutorAllocationManager: Executor 2 is no longer pending to be removed (4 left)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Disabling executor 3.
16/05/24 12:08:35 DEBUG YarnScheduler: Executor 3 on 10.5.151.129 lost, but reason not yet known.
16/05/24 12:08:35 INFO DAGScheduler: Executor lost: 3 (epoch 0)
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Trying to remove executor 3 from BlockManagerMaster.
16/05/24 12:08:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(3, 10.5.151.129, 50767)
16/05/24 12:08:35 INFO BlockManagerMaster: Removed 3 successfully in removeExecutor
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=26, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=109056
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 109338 lastFlushOffset 109193
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 26
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 26
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 26 offsetInBlock: 109056 lastPacketInBlock: false lastByteOffsetInBlock: 109338
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Disabling executor 8.
16/05/24 12:08:35 DEBUG YarnScheduler: Executor 8 on 10.5.151.134 lost, but reason not yet known.
16/05/24 12:08:35 INFO DAGScheduler: Executor lost: 8 (epoch 0)
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Trying to remove executor 8 from BlockManagerMaster.
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(8, 10.5.151.134, 47769)
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 26 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 513216
16/05/24 12:08:35 INFO BlockManagerMaster: Removed 8 successfully in removeExecutor
16/05/24 12:08:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=27, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=109056
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 109483 lastFlushOffset 109338
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 27
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 27
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 27 offsetInBlock: 109056 lastPacketInBlock: false lastByteOffsetInBlock: 109483
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 27 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 482217
16/05/24 12:08:35 INFO YarnScheduler: Executor 3 on 10.5.151.129 killed by driver.
16/05/24 12:08:35 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: ANY
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=28, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=109056
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 109678 lastFlushOffset 109483
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 28
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 28
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 28 offsetInBlock: 109056 lastPacketInBlock: false lastByteOffsetInBlock: 109678
16/05/24 12:08:35 INFO YarnScheduler: Executor 8 on 10.5.151.134 killed by driver.
16/05/24 12:08:35 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: ANY
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 28 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 488431
16/05/24 12:08:35 INFO ExecutorAllocationManager: Existing executor 3 has been removed (new total is 4)
16/05/24 12:08:35 DEBUG ExecutorAllocationManager: Executor 3 is no longer pending to be removed (3 left)
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=29, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=109568
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 109873 lastFlushOffset 109678
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 29
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 29
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 29 offsetInBlock: 109568 lastPacketInBlock: false lastByteOffsetInBlock: 109873
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 29 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 492942
16/05/24 12:08:35 INFO ExecutorAllocationManager: Existing executor 8 has been removed (new total is 3)
16/05/24 12:08:35 DEBUG ExecutorAllocationManager: Executor 8 is no longer pending to be removed (2 left)
16/05/24 12:08:35 INFO YarnClientSchedulerBackend: Disabling executor 6.
16/05/24 12:08:35 DEBUG YarnScheduler: Executor 6 on 10.5.151.127 lost, but reason not yet known.
16/05/24 12:08:35 INFO DAGScheduler: Executor lost: 6 (epoch 0)
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Trying to remove executor 6 from BlockManagerMaster.
16/05/24 12:08:35 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(6, 10.5.151.127, 43177)
16/05/24 12:08:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:35 INFO BlockManagerMaster: Removed 6 successfully in removeExecutor
16/05/24 12:08:35 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=30, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=109568
16/05/24 12:08:35 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 110018 lastFlushOffset 109873
16/05/24 12:08:35 DEBUG DFSClient: Queued packet 30
16/05/24 12:08:35 DEBUG DFSClient: Waiting for ack for: 30
16/05/24 12:08:35 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 30 offsetInBlock: 109568 lastPacketInBlock: false lastByteOffsetInBlock: 110018
16/05/24 12:08:35 DEBUG DFSClient: DFSClient seqno: 30 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 492515
16/05/24 12:08:36 INFO YarnScheduler: Executor 6 on 10.5.151.127 killed by driver.
16/05/24 12:08:36 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: ANY
16/05/24 12:08:36 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=31, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=109568
16/05/24 12:08:36 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 110213 lastFlushOffset 110018
16/05/24 12:08:36 DEBUG DFSClient: Queued packet 31
16/05/24 12:08:36 DEBUG DFSClient: Waiting for ack for: 31
16/05/24 12:08:36 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 31 offsetInBlock: 109568 lastPacketInBlock: false lastByteOffsetInBlock: 110213
16/05/24 12:08:36 DEBUG DFSClient: DFSClient seqno: 31 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 492875
16/05/24 12:08:36 INFO ExecutorAllocationManager: Existing executor 6 has been removed (new total is 2)
16/05/24 12:08:36 DEBUG ExecutorAllocationManager: Executor 6 is no longer pending to be removed (1 left)
16/05/24 12:08:36 INFO YarnClientSchedulerBackend: Disabling executor 4.
16/05/24 12:08:36 DEBUG YarnScheduler: Executor 4 on 10.5.151.123 lost, but reason not yet known.
16/05/24 12:08:36 INFO DAGScheduler: Executor lost: 4 (epoch 0)
16/05/24 12:08:36 INFO BlockManagerMasterEndpoint: Trying to remove executor 4 from BlockManagerMaster.
16/05/24 12:08:36 INFO BlockManagerMasterEndpoint: Removing block manager BlockManagerId(4, 10.5.151.123, 39707)
16/05/24 12:08:36 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:36 INFO BlockManagerMaster: Removed 4 successfully in removeExecutor
16/05/24 12:08:36 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=32, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=110080
16/05/24 12:08:36 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 110358 lastFlushOffset 110213
16/05/24 12:08:36 DEBUG DFSClient: Queued packet 32
16/05/24 12:08:36 DEBUG DFSClient: Waiting for ack for: 32
16/05/24 12:08:36 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 32 offsetInBlock: 110080 lastPacketInBlock: false lastByteOffsetInBlock: 110358
16/05/24 12:08:36 INFO YarnScheduler: Executor 4 on 10.5.151.123 killed by driver.
16/05/24 12:08:36 DEBUG TaskSetManager: Valid locality levels for TaskSet 0.0: ANY
16/05/24 12:08:36 DEBUG DFSClient: DFSClient seqno: 32 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 536065
16/05/24 12:08:36 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=33, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=110080
16/05/24 12:08:36 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 110553 lastFlushOffset 110358
16/05/24 12:08:36 DEBUG DFSClient: Queued packet 33
16/05/24 12:08:36 DEBUG DFSClient: Waiting for ack for: 33
16/05/24 12:08:36 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 33 offsetInBlock: 110080 lastPacketInBlock: false lastByteOffsetInBlock: 110553
16/05/24 12:08:36 DEBUG DFSClient: DFSClient seqno: 33 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 510608
16/05/24 12:08:36 INFO ExecutorAllocationManager: Existing executor 4 has been removed (new total is 1)
16/05/24 12:08:36 DEBUG ExecutorAllocationManager: Executor 4 is no longer pending to be removed (0 left)
16/05/24 12:08:36 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:36 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #159
16/05/24 12:08:36 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #159
16/05/24 12:08:36 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:37 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:37 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #160
16/05/24 12:08:37 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #160
16/05/24 12:08:37 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:38 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:38 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #161
16/05/24 12:08:38 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #161
16/05/24 12:08:38 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:39 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:39 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #162
16/05/24 12:08:39 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #162
16/05/24 12:08:39 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:40 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:40 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #163
16/05/24 12:08:40 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #163
16/05/24 12:08:40 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:41 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:41 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #164
16/05/24 12:08:41 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #164
16/05/24 12:08:41 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:42 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:42 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #165
16/05/24 12:08:42 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #165
16/05/24 12:08:42 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:43 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:43 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #166
16/05/24 12:08:43 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #166
16/05/24 12:08:43 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:44 DEBUG Client: The ping interval is 60000 ms.
16/05/24 12:08:44 DEBUG Client: Connecting to dcsac022/10.5.151.122:8020
16/05/24 12:08:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #167
16/05/24 12:08:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: starting, having connections 2
16/05/24 12:08:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #167
16/05/24 12:08:44 DEBUG ProtobufRpcEngine: Call: renewLease took 4ms
16/05/24 12:08:44 DEBUG LeaseRenewer: Lease renewed for client DFSClient_NONMAPREDUCE_-1596888490_1
16/05/24 12:08:44 DEBUG LeaseRenewer: Lease renewer daemon for [DFSClient_NONMAPREDUCE_-1596888490_1] with renew id 1 executed
16/05/24 12:08:44 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #168
16/05/24 12:08:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #168
16/05/24 12:08:44 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:45 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:45 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #169
16/05/24 12:08:45 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #169
16/05/24 12:08:45 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:46 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:46 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #170
16/05/24 12:08:46 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #170
16/05/24 12:08:46 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:47 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:47 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #171
16/05/24 12:08:47 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #171
16/05/24 12:08:47 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:48 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:48 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #172
16/05/24 12:08:48 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #172
16/05/24 12:08:48 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:49 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:49 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #173
16/05/24 12:08:49 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #173
16/05/24 12:08:49 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:50 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:50 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #174
16/05/24 12:08:50 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #174
16/05/24 12:08:50 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:51 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:51 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #175
16/05/24 12:08:51 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #175
16/05/24 12:08:51 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:52 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:52 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #176
16/05/24 12:08:52 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #176
16/05/24 12:08:52 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:53 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:53 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #177
16/05/24 12:08:53 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #177
16/05/24 12:08:53 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: closed
16/05/24 12:08:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: stopped, remaining connections 1
16/05/24 12:08:54 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #178
16/05/24 12:08:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #178
16/05/24 12:08:54 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:55 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:55 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #179
16/05/24 12:08:55 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #179
16/05/24 12:08:55 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:56 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:56 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #180
16/05/24 12:08:56 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #180
16/05/24 12:08:56 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:08:57 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:57 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #181
16/05/24 12:08:57 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #181
16/05/24 12:08:57 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:58 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:58 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #182
16/05/24 12:08:58 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #182
16/05/24 12:08:58 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:08:59 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:08:59 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #183
16/05/24 12:08:59 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #183
16/05/24 12:08:59 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:00 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:00 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #184
16/05/24 12:09:00 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #184
16/05/24 12:09:00 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:01 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:01 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #185
16/05/24 12:09:01 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #185
16/05/24 12:09:01 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:02 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:02 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #186
16/05/24 12:09:02 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #186
16/05/24 12:09:02 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:03 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #187
16/05/24 12:09:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #187
16/05/24 12:09:03 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:04 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:04 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #188
16/05/24 12:09:04 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #188
16/05/24 12:09:04 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:05 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:05 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #189
16/05/24 12:09:05 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #189
16/05/24 12:09:05 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:06 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: -1 offsetInBlock: 0 lastPacketInBlock: false lastByteOffsetInBlock: 0
16/05/24 12:09:06 DEBUG DFSClient: DFSClient seqno: -1 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 633664
16/05/24 12:09:06 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:06 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #190
16/05/24 12:09:06 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #190
16/05/24 12:09:06 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:07 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:07 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #191
16/05/24 12:09:07 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #191
16/05/24 12:09:07 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:08 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:08 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #192
16/05/24 12:09:08 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #192
16/05/24 12:09:08 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:09 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:09 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #193
16/05/24 12:09:09 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #193
16/05/24 12:09:09 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:10 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:10 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #194
16/05/24 12:09:10 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #194
16/05/24 12:09:10 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:11 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:11 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #195
16/05/24 12:09:11 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #195
16/05/24 12:09:11 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:12 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:12 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #196
16/05/24 12:09:12 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #196
16/05/24 12:09:12 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:13 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:13 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #197
16/05/24 12:09:13 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #197
16/05/24 12:09:13 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:14 DEBUG Client: The ping interval is 60000 ms.
16/05/24 12:09:14 DEBUG Client: Connecting to dcsac022/10.5.151.122:8020
16/05/24 12:09:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: starting, having connections 2
16/05/24 12:09:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #198
16/05/24 12:09:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #198
16/05/24 12:09:14 DEBUG ProtobufRpcEngine: Call: renewLease took 4ms
16/05/24 12:09:14 DEBUG LeaseRenewer: Lease renewed for client DFSClient_NONMAPREDUCE_-1596888490_1
16/05/24 12:09:14 DEBUG LeaseRenewer: Lease renewer daemon for [DFSClient_NONMAPREDUCE_-1596888490_1] with renew id 1 executed
16/05/24 12:09:14 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #199
16/05/24 12:09:14 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #199
16/05/24 12:09:14 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:15 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:15 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #200
16/05/24 12:09:15 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #200
16/05/24 12:09:15 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:16 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:16 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #201
16/05/24 12:09:16 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #201
16/05/24 12:09:16 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:17 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:17 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #202
16/05/24 12:09:17 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #202
16/05/24 12:09:17 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:18 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:18 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #203
16/05/24 12:09:18 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #203
16/05/24 12:09:18 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:19 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:19 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #204
16/05/24 12:09:19 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #204
16/05/24 12:09:19 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:20 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:20 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #205
16/05/24 12:09:20 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #205
16/05/24 12:09:20 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:21 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:21 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #206
16/05/24 12:09:21 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #206
16/05/24 12:09:21 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:22 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:22 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #207
16/05/24 12:09:22 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #207
16/05/24 12:09:22 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:23 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:23 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #208
16/05/24 12:09:23 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #208
16/05/24 12:09:23 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: closed
16/05/24 12:09:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: stopped, remaining connections 1
16/05/24 12:09:24 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #209
16/05/24 12:09:24 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #209
16/05/24 12:09:24 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:25 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:25 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #210
16/05/24 12:09:25 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #210
16/05/24 12:09:25 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:26 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:26 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #211
16/05/24 12:09:26 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #211
16/05/24 12:09:26 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:27 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:27 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #212
16/05/24 12:09:27 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #212
16/05/24 12:09:27 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:28 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:28 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #213
16/05/24 12:09:28 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #213
16/05/24 12:09:28 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:29 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:29 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #214
16/05/24 12:09:29 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #214
16/05/24 12:09:29 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:30 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:30 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #215
16/05/24 12:09:30 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #215
16/05/24 12:09:30 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:31 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:31 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #216
16/05/24 12:09:31 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #216
16/05/24 12:09:31 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:32 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:32 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #217
16/05/24 12:09:32 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #217
16/05/24 12:09:32 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:33 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:33 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #218
16/05/24 12:09:33 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #218
16/05/24 12:09:33 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:34 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:34 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #219
16/05/24 12:09:34 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #219
16/05/24 12:09:34 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
[Stage 0:===================================================>       (7 + 1) / 8]16/05/24 12:09:35 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:35 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #220
16/05/24 12:09:35 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #220
16/05/24 12:09:35 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:36 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: -1 offsetInBlock: 0 lastPacketInBlock: false lastByteOffsetInBlock: 0
16/05/24 12:09:36 DEBUG DFSClient: DFSClient seqno: -1 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 619861
16/05/24 12:09:36 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:36 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #221
16/05/24 12:09:36 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #221
16/05/24 12:09:36 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:37 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:37 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #222
16/05/24 12:09:37 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #222
16/05/24 12:09:37 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 4ms
16/05/24 12:09:38 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:38 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #223
16/05/24 12:09:38 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #223
16/05/24 12:09:38 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:39 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:39 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #224
16/05/24 12:09:39 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #224
16/05/24 12:09:39 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:40 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:40 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #225
16/05/24 12:09:40 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #225
16/05/24 12:09:40 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:41 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:41 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #226
16/05/24 12:09:41 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #226
16/05/24 12:09:41 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:42 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:42 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #227
16/05/24 12:09:42 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #227
16/05/24 12:09:42 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:43 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:43 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #228
16/05/24 12:09:43 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #228
16/05/24 12:09:43 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:44 DEBUG Client: The ping interval is 60000 ms.
16/05/24 12:09:44 DEBUG Client: Connecting to dcsac022/10.5.151.122:8020
16/05/24 12:09:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: starting, having connections 2
16/05/24 12:09:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #229
16/05/24 12:09:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #229
16/05/24 12:09:44 DEBUG ProtobufRpcEngine: Call: renewLease took 4ms
16/05/24 12:09:44 DEBUG LeaseRenewer: Lease renewed for client DFSClient_NONMAPREDUCE_-1596888490_1
16/05/24 12:09:44 DEBUG LeaseRenewer: Lease renewer daemon for [DFSClient_NONMAPREDUCE_-1596888490_1] with renew id 1 executed
16/05/24 12:09:44 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #230
16/05/24 12:09:44 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #230
16/05/24 12:09:44 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:45 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:45 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #231
16/05/24 12:09:45 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #231
16/05/24 12:09:45 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:46 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:46 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #232
16/05/24 12:09:46 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #232
16/05/24 12:09:46 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:47 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:47 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #233
16/05/24 12:09:47 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #233
16/05/24 12:09:47 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:48 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:48 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #234
16/05/24 12:09:48 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #234
16/05/24 12:09:48 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:49 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:49 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #235
16/05/24 12:09:49 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #235
16/05/24 12:09:49 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:50 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:50 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #236
16/05/24 12:09:50 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #236
16/05/24 12:09:50 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 2ms
16/05/24 12:09:51 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:51 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #237
16/05/24 12:09:51 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #237
16/05/24 12:09:51 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:52 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:52 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #238
16/05/24 12:09:52 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #238
16/05/24 12:09:52 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:53 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:53 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #239
16/05/24 12:09:53 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #239
16/05/24 12:09:53 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: closed
16/05/24 12:09:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: stopped, remaining connections 1
16/05/24 12:09:54 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #240
16/05/24 12:09:54 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #240
16/05/24 12:09:54 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:55 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:55 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #241
16/05/24 12:09:55 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #241
16/05/24 12:09:55 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:56 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:56 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #242
16/05/24 12:09:56 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #242
16/05/24 12:09:56 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:57 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:57 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #243
16/05/24 12:09:57 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #243
16/05/24 12:09:57 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:58 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:58 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #244
16/05/24 12:09:58 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #244
16/05/24 12:09:58 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:09:59 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:09:59 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #245
16/05/24 12:09:59 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #245
16/05/24 12:09:59 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:10:00 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:10:00 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #246
16/05/24 12:10:00 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #246
16/05/24 12:10:00 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:10:01 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:10:01 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #247
16/05/24 12:10:01 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #247
16/05/24 12:10:01 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:10:02 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:10:02 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith sending #248
16/05/24 12:10:02 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8032 from keith got value #248
16/05/24 12:10:02 DEBUG ProtobufRpcEngine: Call: getApplicationReport took 1ms
16/05/24 12:10:03 INFO SparkContext: Invoking stop() from shutdown hook
16/05/24 12:10:03 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=34, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=110080
16/05/24 12:10:03 DEBUG DFSClient: DFSClient flush():  bytesCurBlock 110619 lastFlushOffset 110553
16/05/24 12:10:03 DEBUG DFSClient: Queued packet 34
16/05/24 12:10:03 DEBUG DFSClient: Waiting for ack for: 34
16/05/24 12:10:03 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 34 offsetInBlock: 110080 lastPacketInBlock: false lastByteOffsetInBlock: 110619
16/05/24 12:10:03 DEBUG DFSClient: DFSClient seqno: 34 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 768311
16/05/24 12:10:03 DEBUG YarnScheduler: parentName: default, name: TaskSet_0, runningTasks: 1
16/05/24 12:10:03 INFO SparkUI: Stopped Spark web UI at http://10.5.151.121:56042
16/05/24 12:10:03 DEBUG DFSClient: DFSClient writeChunk allocating new packet seqno=35, src=/user/spark/applicationHistory/application_1464015602645_0083.inprogress, packetSize=65532, chunksPerPacket=127, bytesCurBlock=110592
16/05/24 12:10:03 DEBUG DFSClient: Queued packet 35
16/05/24 12:10:03 DEBUG DFSClient: Queued packet 36
16/05/24 12:10:03 DEBUG DFSClient: Waiting for ack for: 36
16/05/24 12:10:03 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 35 offsetInBlock: 110592 lastPacketInBlock: false lastByteOffsetInBlock: 110619
16/05/24 12:10:03 DEBUG DFSClient: DFSClient seqno: 35 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 470266
16/05/24 12:10:03 DEBUG DFSClient: DataStreamer block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458 sending packet packet seqno: 36 offsetInBlock: 110619 lastPacketInBlock: true lastByteOffsetInBlock: 110619
16/05/24 12:10:03 DEBUG DFSClient: DFSClient seqno: 36 reply: 0 reply: 0 reply: 0 downstreamAckTimeNanos: 2415979
16/05/24 12:10:03 DEBUG DFSClient: Closing old block BP-1669306261-10.5.151.122-1458279933593:blk_1075282159_1541458
16/05/24 12:10:03 DEBUG Client: The ping interval is 60000 ms.
16/05/24 12:10:03 DEBUG Client: Connecting to dcsac022/10.5.151.122:8020
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith: starting, having connections 2
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #249
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #249
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: complete took 14ms
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #250
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #250
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: getFileInfo took 1ms
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #251
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #251
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: rename took 6ms
16/05/24 12:10:03 INFO DAGScheduler: ResultStage 0 (parquet at <console>:22) failed in 149.140 s
16/05/24 12:10:03 INFO DAGScheduler: Job 0 failed: parquet at <console>:22, took 149.412419 s
16/05/24 12:10:03 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerStageCompleted(org.apache.spark.scheduler.StageInfo@7e288fa4)
16/05/24 12:10:03 ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event SparkListenerJobEnd(0,1464106203491,JobFailed(org.apache.spark.SparkException: Job 0 cancelled because SparkContext was shut down))
16/05/24 12:10:03 INFO YarnClientSchedulerBackend: Shutting down all executors
16/05/24 12:10:03 INFO YarnClientSchedulerBackend: Interrupting monitor thread
16/05/24 12:10:03 INFO YarnClientSchedulerBackend: Asking each executor to shut down
16/05/24 12:10:03 DEBUG AbstractService: Service: org.apache.hadoop.yarn.client.api.impl.YarnClientImpl entered state STOPPED
16/05/24 12:10:03 DEBUG Client: stopping client from cache: org.apache.hadoop.ipc.Client@177d440e
16/05/24 12:10:03 INFO YarnClientSchedulerBackend: Stopped
16/05/24 12:10:03 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/05/24 12:10:03 INFO MemoryStore: MemoryStore cleared
16/05/24 12:10:03 INFO BlockManager: BlockManager stopped
16/05/24 12:10:03 INFO BlockManagerMaster: BlockManagerMaster stopped
16/05/24 12:10:03 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/05/24 12:10:03 INFO SparkContext: Successfully stopped SparkContext
16/05/24 12:10:03 INFO ShutdownHookManager: Shutdown hook called
16/05/24 12:10:03 INFO ShutdownHookManager: Deleting directory /tmp/spark-ebe3285d-ad81-4daa-80ae-356063d761c9
16/05/24 12:10:03 INFO ShutdownHookManager: Deleting directory /tmp/spark-705b01e2-b4e6-44c9-b0df-18bfef0fb29a
16/05/24 12:10:03 INFO ShutdownHookManager: Deleting directory /tmp/spark-251f5c07-a6ae-485f-a487-0a5307fbd48c
16/05/24 12:10:03 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/05/24 12:10:03 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #252
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #252
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: getFileInfo took 0ms
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #253
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #253
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: delete took 10ms
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #254
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #254
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: getFileInfo took 1ms
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #255
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #255
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: getFileInfo took 1ms
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #256
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #256
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: delete took 7ms
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith sending #257
16/05/24 12:10:03 DEBUG Client: IPC Client (737852016) connection to dcsac022/10.5.151.122:8020 from keith got value #257
16/05/24 12:10:03 DEBUG ProtobufRpcEngine: Call: getFileInfo took 1ms
16/05/24 12:10:03 DEBUG Client: stopping client from cache: org.apache.hadoop.ipc.Client@177d440e