storm-yarn0.9.0 doesn't work on hdp2.06 VMS

[杨勇]

env：
1.A(hadoop81.test)\B(hadoop82.test)\C(hadoop83.test) Virtual machines，4g RAM+2cpus
2.Ambari1.4.2 install HDP2.06
3.A:Storm-Yarn(storm-0.9.0-wip21)、NN   B:RM
4.STORM-YARN_HOME:/usr/local/src/storm-yarn-master
5.STORM_HOME:/usr/local/src/storm-0.9.0-wip21/
6.STORM_ZIP_HOME:hdfs://hadoop81.test:8020/lib/storm/0.9.0-wip21
7.edit the /usr/local/src/storm-0.9.0-wip21/conf/storm.yaml:
 storm.zookeeper.servers:
     - "hadoop81.test"
     - "hadoop82.test"
     - "hadoop83.test"
 nimbus.host: "hadoop81.test"
 master.host: "hadoop81.test"
 supervisor.worker.start.timeout.secs: 360
 master.container.size-mb: 1024

 
major install steps:
1.edit the pom.xml :
    <properties>
        <storm.version>0.9.0-wip21</storm.version>
        <!--hadoop.version>2.2.0</hadoop.version-->
        <hadoop.version>2.2.0.2.0.6.0-76</hadoop.version>
    </properties>
2.cp lib/storm.zip  /usr/local/src/storm-0.9.0-wip21/
3.hdfs dfs –put storm.zip  /lib/storm/0.9.0-wip2/
4.export PATH=$PATH:/usr/local/src/maven/bin:/usr/local/src/storm-0.9.0-wip21/bin:/usr/local/src/storm-yarn-master/bin
5.cd storm-yarn-master
   mvn package
6.storm-yarn launch /usr/local/src/storm-0.9.0-wip21/conf/storm.yaml --stormZip /lib/storm/0.9.0-wip21/storm.zip --appname storm-on-yarn-test --output target/appId.txt
7.yarn application -list
8.storm-yarn getStormConfig /usr/local/src/storm-0.9.0-wip21/conf/storm.yaml --appId application_1390526485888_0015 --output ~/.storm/storm.yaml
9.storm jar lib/storm-starter-0.0.1-SNAPSHOT.jar storm.starter.WordCountTopology WordCountTopology -c nimbus.host=10.1.1.81

log info:
1.hadoop81.test /hadoop/yarn/log/application_1390526485888_0015/container_1390526485888_0015_01_000001/nimbus.log
 Setting new assignment for topology id WordCountTopology-1-1390704628: #backtype.storm.daemon.common.Assignment{:master-code-dir "storm-local/nimbus/stormdist/WordCountTopology-1-1390704628",
 :node->host {"b731bdb6-4b09-4d32-ab91-78cbac21733a" "hadoop82.test"}, :executor->node+port {[2 2] ["b731bdb6-4b09-4d32-ab91-78cbac21733a" 6702], [3 3] ["b731bdb6-4b09-4d32-ab91-78cbac21733a" 6701], [4 4]
 2014-01-30 09:35:35 b.s.d.nimbus [INFO] Executor WordCountTopology-1-1390704628:[2 2] not alive
2014-01-30 09:35:35 b.s.d.nimbus [INFO] Executor WordCountTopology-1-1390704628:[3 3] not alive
2014-01-30 09:35:35 b.s.d.nimbus [INFO] Executor WordCountTopology-1-1390704628:[4 4] not alive
2.hadoop82.test  /hadoop/yarn/log/application_1390526485888_0015/container_1390526485888_0015_01_000002/supervisor.log
there aren't workerlogs in this directory at all.

2014-01-31 16:56:26 b.s.d.supervisor [INFO] 32818308-a03b-4912-918c-d8eeb57fd8c4 still hasn't started
2014-01-31 16:56:26 b.s.d.supervisor [INFO] 32818308-a03b-4912-918c-d8eeb57fd8c4 still hasn't started
2014-01-31 16:56:27 b.s.d.supervisor [INFO] 32818308-a03b-4912-918c-d8eeb57fd8c4 still hasn't started
2014-01-31 16:56:27 b.s.d.supervisor [INFO] 32818308-a03b-4912-918c-d8eeb57fd8c4 still hasn't started
2014-01-31 16:56:28 b.s.d.supervisor [INFO] 32818308-a03b-4912-918c-d8eeb57fd8c4 still hasn't started
2014-01-31 16:56:28 b.s.d.supervisor [INFO] 32818308-a03b-4912-918c-d8eeb57fd8c4 still hasn't started
2014-01-31 16:56:29 b.s.d.supervisor [INFO] Worker 32818308-a03b-4912-918c-d8eeb57fd8c4 failed to start
2014-01-31 16:56:29 b.s.d.supervisor [INFO] Worker 8e6dbd52-89b8-4b45-bb68-ddd8ad86693d failed to start
2014-01-31 16:56:29 b.s.d.supervisor [INFO] Worker 709f18d2-63f3-4ec8-9c60-8066310942fd failed to start
2014-01-31 16:56:29 b.s.d.supervisor [INFO] Shutting down and clearing state for id 709f18d2-63f3-4ec8-9c60-8066310942fd. Current supervisor time: 1391158589. State: :disallowed, Heartbeat: nil

[Bobby Evans]

I am not really sure.  The logs show that most everything came up, except the worker process.  They show that the worker failed to launch for some reason and then it looks like the topology was killed or the worker was rescheduled to a different node.

Without seeing the log for the worker process itself it will be difficult to debug.  It could also be an issue where the process is never launched and sadly the way the supervisor is written right now anything written to stderr and stdout before it comes up fully is lost.

—Bobby

--
You received this message because you are subscribed to the Google Groups "storm-yarn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-yarn+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[杨勇]

Hi,Bobby
    What should i do?

在 2014年2月1日星期六UTC+8上午12时47分20秒，evans写道：

[Bobby Evans]

If it were me I would first look at pulling in something to storm that would log stdout/stderr for the processes being launched.

https://github.com/nathanmarz/storm/pull/595

I am not sure if the patch applies to the version of storm you are using.  The other thing I would do is look for the worker logs, and see what might be going on there.