CDH4.3.0.2 - Yarn - Spark (trunk)

[Vipul]

I'm unable to successfully run the SparkPi example in my YARN cluster. 

I did whatever has been specified here (didn't change anything anywhere) : http://spark.incubator.apache.org/docs/0.7.0/running-on-yarn.html

and added HADOOP_CONF_DIR as well. (btw, on sbt/sbt assembly - the jar file it generates is spark-core-assembly-0.6.0.jar)

I get the following exception in my container : 

Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
	at org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
	at org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster(AMRMProtocolPBClientImpl.java:103)
	at spark.deploy.yarn.ApplicationMaster.registerApplicationMaster(ApplicationMaster.scala:123)
	at spark.deploy.yarn.ApplicationMaster.spark$deploy$yarn$ApplicationMaster$$runImpl(ApplicationMaster.scala:52)
	at spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:42)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
	at spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:40)
	at spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:340)
	at spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
Caused by: com.google.protobuf.ServiceException: java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: callId, status; Host Details : local host is: "rd17d01ls-vm0109.rd.geo.apple.com/17.134.172.65"; destination host is: "rd17d01ls-vm0110.rd.geo.apple.com":8030; 
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:212)
	at $Proxy7.registerApplicationMaster(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster(AMRMProtocolPBClientImpl.java:100)
	... 9 more
Caused by: java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: callId, status; Host Details : local host is: "rd17d01ls-vm0109.rd.geo.apple.com/17.134.172.65"; destination host is: "rd17d01ls-vm0110.rd.geo.apple.com":8030; 
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:761)
	at org.apache.hadoop.ipc.Client.call(Client.java:1239)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
	... 11 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: callId, status
	at com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:81)
	at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto$Builder.buildParsed(RpcPayloadHeaderProtos.java:1094)
	at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto$Builder.access$1300(RpcPayloadHeaderProtos.java:1028)
	at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:986)
	at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:946)
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:844)

Any solutions anyone? 

[Arindam Paul]

Hi Vipul,

Any luck here ? Have you found a solution to your problem ? If so, please post it here..