Hi,
After launch storm-yarn via "storm-yarn launch storm-0.9.0-wip21/conf/storm.yaml", the new yarn application
shuts down a few seconds later. The status of the application shows "Failed".
The Yarn UI shows:
Application application_1411179375629_0005 failed 2 times due to AM Container for appattempt_1411179375629_0005_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
The container log shows the following:
14/09/19 20:44:55 INFO yarn.MasterServer: Got a registration response maximumCapability { memory: 8192 virtual_cores: 32 } queue: "default"
14/09/19 20:44:55 INFO yarn.MasterServer: Max Capability <memory:8192, vCores:32>
14/09/19 20:44:55 INFO yarn.StormAMRMClient: Max Capability is now <memory:8192, vCores:32>
14/09/19 20:44:55 INFO yarn.MasterServer: Starting HB thread
14/09/19 20:44:55 INFO yarn.MasterServer: Starting launcher
14/09/19 20:44:55 INFO yarn.MasterServer: Starting Master Thrift Server
14/09/19 20:44:55 ERROR auth.ThriftServer: ThriftServer is being stopped due to: org.apache.thrift7.transport.TTransportException: Could not create ServerSocket on address
0.0.0.0/0.0.0.0:9000.
org.apache.thrift7.transport.TTransportException: Could not create ServerSocket on address
0.0.0.0/0.0.0.0:9000.
at org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:89)
at org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:68)
at org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:61)
at backtype.storm.security.auth.SimpleTransportPlugin.getServer(SimpleTransportPlugin.java:47)
at backtype.storm.security.auth.ThriftServer.serve(ThriftServer.java:52)
at com.yahoo.storm.yarn.MasterServer.main(MasterServer.java:175)
By netstat it shows port 9000 is being used by Namenode of HDFS, how should I work around this?
Meanwhile, I see most cases people use storm-yarn in user "hdfs", is it mandatory?
Thanks a lot!