what is the usage for port 9000?

[bertt...@gmail.com]

1、when execute: storm-yarn launch <config.yaml>, i got a .

2、when i execute:storm-yarn startUi -appId <appId>,i got an error: Attaching to h2:9000 to talk to app master application_1376015903368_0010, connection refused!

i saw  in the sourcecode "master_defaults.yaml", there is the concerned config: 

master.host: "localhost"

master.thrift.port: 9000  

can anybody tell me:

1. how to turn on port 9000?

2.what is the usage of port 9000?

3.do i need to change the "master_defaults.yaml"? 

thank you very much!

[bertt...@gmail.com]

paste the config file:

########### These MUST be filled in for a storm configuration

 storm.zookeeper.servers:

      - "192.168.9.121"

#     - "server2"

# 

 nimbus.host: "192.168.9.32"

 master.initial-num-supervisors: 1

 ui.port: 8082

# 

# 

# ##### These may optionally be filled in:

#    

## List of custom serializations

# topology.kryo.register:

#     - org.mycompany.MyType

#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer

#

## List of custom kryo decorators

# topology.kryo.decorators:

#     - org.mycompany.MyDecorator

#

## Locations of the drpc servers

# drpc.servers:

#     - "server1"

#     - "server2"

## Metrics Consumers

# topology.metrics.consumer.register:

#    - class: "org.mycompany.MyMetricsConsumer"

#      argument: 

#         - endpoint: "metrics-collector.mycompany.org"

#      parallelism.hint: 1

在 2013年8月12日星期一UTC+8上午12时31分53秒，bertt...@gmail.com写道：

[bertt...@gmail.com]

paste the log after launch:

./storm-yarn launch /home/hadoop/storm-yarn-2.1.0/conf/storm-yarn.yaml 

STORM_BIN=/home/hadoop/storm/bin

/usr/java/jdk1.7.0_25/bin/java -cp :/home/hadoop/storm-yarn-2.1.0/bin/../target/storm-yarn-1.0-alpha.jar:/home/hadoop/hdp2/conf:/home/hadoop/hdp2/conf:/home/hadoop/hdp2/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-yarn/lib/*:/home/hadoop/storm/storm-core-0.9.0-wip19.jar:/home/hadoop/storm/storm-netty-0.9.0-wip19.jar:/home/hadoop/storm/storm-console-logging-0.9.0-wip19.jar:/home/hadoop/storm/lib/commons-logging-1.1.1.jar:/home/hadoop/storm/lib/junit-3.8.1.jar:/home/hadoop/storm/lib/compojure-1.1.3.jar:/home/hadoop/storm/lib/commons-lang-2.5.jar:/home/hadoop/storm/lib/netty-3.6.3.Final.jar:/home/hadoop/storm/lib/servlet-api-2.5.jar:/home/hadoop/storm/lib/jgrapht-0.8.3.jar:/home/hadoop/storm/lib/minlog-1.2.jar:/home/hadoop/storm/lib/clout-1.0.1.jar:/home/hadoop/storm/lib/snakeyaml-1.11.jar:/home/hadoop/storm/lib/tools.cli-0.2.2.jar:/home/hadoop/storm/lib/jetty-util-6.1.26.jar:/home/hadoop/storm/lib/zookeeper-3.3.3.jar:/home/hadoop/storm/lib/tools.macro-0.1.0.jar:/home/hadoop/storm/lib/objenesis-1.2.jar:/home/hadoop/storm/lib/httpclient-4.1.1.jar:/home/hadoop/storm/lib/json-simple-1.1.jar:/home/hadoop/storm/lib/asm-4.0.jar:/home/hadoop/storm/lib/httpcore-4.1.jar:/home/hadoop/storm/lib/jline-0.9.94.jar:/home/hadoop/storm/lib/libthrift7-0.7.0.jar:/home/hadoop/storm/lib/ring-devel-0.3.11.jar:/home/hadoop/storm/lib/curator-framework-1.0.1.jar:/home/hadoop/storm/lib/carbonite-1.5.0.jar:/home/hadoop/storm/lib/commons-io-1.4.jar:/home/hadoop/storm/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm/lib/hiccup-0.3.6.jar:/home/hadoop/storm/lib/disruptor-2.10.1.jar:/home/hadoop/storm/lib/jzmq-2.1.0.jar:/home/hadoop/storm/lib/clojure-1.4.0.jar:/home/hadoop/storm/lib/commons-exec-1.1.jar:/home/hadoop/storm/lib/joda-time-2.0.jar:/home/hadoop/storm/lib/logback-classic-1.0.6.jar:/home/hadoop/storm/lib/curator-client-1.0.1.jar:/home/hadoop/storm/lib/clj-time-0.4.1.jar:/home/hadoop/storm/lib/tools.logging-0.2.3.jar:/home/hadoop/storm/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm/lib/kryo-2.17.jar:/home/hadoop/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm/lib/clj-stacktrace-0.2.2.jar:/home/hadoop/storm/lib/jetty-6.1.26.jar:/home/hadoop/storm/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm/lib/ring-core-1.1.5.jar:/home/hadoop/storm/lib/guava-13.0.jar:/home/hadoop/storm/lib/mockito-all-1.9.5.jar:/home/hadoop/storm/lib/core.incubator-0.1.0.jar:/home/hadoop/storm/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm/lib/logback-core-1.0.6.jar:/home/hadoop/storm/lib/commons-codec-1.4.jar:/home/hadoop/.storm -Dstorm.home=/home/hadoop/storm/bin/.. com.yahoo.storm.yarn.Client launch /home/hadoop/storm-yarn-2.1.0/conf/storm-yarn.yaml

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/home/hadoop/storm/lib/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

13/08/11 23:59:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

13/08/11 23:59:51 INFO yarn.StormOnYarn: Copy App Master jar from local filesystem and add to local environment

13/08/11 23:59:53 INFO yarn.StormOnYarn: Set the environment for the application master

13/08/11 23:59:53 INFO yarn.StormOnYarn: Setting up app master command:[java, -Dstorm.home=./storm/storm-0.9.0-wip19/, com.yahoo.storm.yarn.MasterServer, 1><LOG_DIR>/stdout, 2><LOG_DIR>/stderr]

13/08/11 23:59:53 INFO impl.YarnClientImpl: Submitted application application_1376015903368_0010 to ResourceManager at /192.168.9.32:8050

application_1376015903368_0010

在 2013年8月12日星期一UTC+8上午12时45分05秒，bertt...@gmail.com写道：

[bertt...@gmail.com]

paster the log after launch

./storm-yarn launch /home/hadoop/storm-yarn-2.1.0/conf/storm-yarn.yaml 

STORM_BIN=/home/hadoop/storm/bin

/usr/java/jdk1.7.0_25/bin/java -cp :/home/hadoop/storm-yarn-2.1.0/bin/../target/storm-yarn-1.0-alpha.jar:/home/hadoop/hdp2/conf:/home/hadoop/hdp2/conf:/home/hadoop/hdp2/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-yarn/lib/*:/home/hadoop/storm/storm-core-0.9.0-wip19.jar:/home/hadoop/storm/storm-netty-0.9.0-wip19.jar:/home/hadoop/storm/storm-console-logging-0.9.0-wip19.jar:/home/hadoop/storm/lib/commons-logging-1.1.1.jar:/home/hadoop/storm/lib/junit-3.8.1.jar:/home/hadoop/storm/lib/compojure-1.1.3.jar:/home/hadoop/storm/lib/commons-lang-2.5.jar:/home/hadoop/storm/lib/netty-3.6.3.Final.jar:/home/hadoop/storm/lib/servlet-api-2.5.jar:/home/hadoop/storm/lib/jgrapht-0.8.3.jar:/home/hadoop/storm/lib/minlog-1.2.jar:/home/hadoop/storm/lib/clout-1.0.1.jar:/home/hadoop/storm/lib/snakeyaml-1.11.jar:/home/hadoop/storm/lib/tools.cli-0.2.2.jar:/home/hadoop/storm/lib/jetty-util-6.1.26.jar:/home/hadoop/storm/lib/zookeeper-3.3.3.jar:/home/hadoop/storm/lib/tools.macro-0.1.0.jar:/home/hadoop/storm/lib/objenesis-1.2.jar:/home/hadoop/storm/lib/httpclient-4.1.1.jar:/home/hadoop/storm/lib/json-simple-1.1.jar:/home/hadoop/storm/lib/asm-4.0.jar:/home/hadoop/storm/lib/httpcore-4.1.jar:/home/hadoop/storm/lib/jline-0.9.94.jar:/home/hadoop/storm/lib/libthrift7-0.7.0.jar:/home/hadoop/storm/lib/ring-devel-0.3.11.jar:/home/hadoop/storm/lib/curator-framework-1.0.1.jar:/home/hadoop/storm/lib/carbonite-1.5.0.jar:/home/hadoop/storm/lib/commons-io-1.4.jar:/home/hadoop/storm/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm/lib/hiccup-0.3.6.jar:/home/hadoop/storm/lib/disruptor-2.10.1.jar:/home/hadoop/storm/lib/jzmq-2.1.0.jar:/home/hadoop/storm/lib/clojure-1.4.0.jar:/home/hadoop/storm/lib/commons-exec-1.1.jar:/home/hadoop/storm/lib/joda-time-2.0.jar:/home/hadoop/storm/lib/logback-classic-1.0.6.jar:/home/hadoop/storm/lib/curator-client-1.0.1.jar:/home/hadoop/storm/lib/clj-time-0.4.1.jar:/home/hadoop/storm/lib/tools.logging-0.2.3.jar:/home/hadoop/storm/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm/lib/kryo-2.17.jar:/home/hadoop/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm/lib/clj-stacktrace-0.2.2.jar:/home/hadoop/storm/lib/jetty-6.1.26.jar:/home/hadoop/storm/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm/lib/ring-core-1.1.5.jar:/home/hadoop/storm/lib/guava-13.0.jar:/home/hadoop/storm/lib/mockito-all-1.9.5.jar:/home/hadoop/storm/lib/core.incubator-0.1.0.jar:/home/hadoop/storm/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm/lib/logback-core-1.0.6.jar:/home/hadoop/storm/lib/commons-codec-1.4.jar:/home/hadoop/.storm -Dstorm.home=/home/hadoop/storm/bin/.. com.yahoo.storm.yarn.Client launch /home/hadoop/storm-yarn-2.1.0/conf/storm-yarn.yaml

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/home/hadoop/storm/lib/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

13/08/11 23:59:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

13/08/11 23:59:51 INFO yarn.StormOnYarn: Copy App Master jar from local filesystem and add to local environment

13/08/11 23:59:53 INFO yarn.StormOnYarn: Set the environment for the application master

13/08/11 23:59:53 INFO yarn.StormOnYarn: Setting up app master command:[java, -Dstorm.home=./storm/storm-0.9.0-wip19/, com.yahoo.storm.yarn.MasterServer, 1><LOG_DIR>/stdout, 2><LOG_DIR>/stderr]

13/08/11 23:59:53 INFO impl.YarnClientImpl: Submitted application application_1376015903368_0010 to ResourceManager at /192.168.9.32:8050

application_1376015903368_0010

在 2013年8月12日星期一UTC+8上午12时45分05秒，bertt...@gmail.com写道：

paste the config file:

[bertt...@gmail.com]

paster the log after startUi:

[hadoop@h1 bin]$ storm-yarn startUI -appId application_1376015903368_0010

STORM_BIN=/home/hadoop/storm/bin

/usr/java/jdk1.7.0_25/bin/java -cp :/home/hadoop/storm-yarn-2.1.0/bin/../target/storm-yarn-1.0-alpha.jar:/home/hadoop/hdp2/conf:/home/hadoop/hdp2/conf:/home/hadoop/hdp2/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-yarn/lib/*:/home/hadoop/storm/storm-core-0.9.0-wip19.jar:/home/hadoop/storm/storm-netty-0.9.0-wip19.jar:/home/hadoop/storm/storm-console-logging-0.9.0-wip19.jar:/home/hadoop/storm/lib/commons-logging-1.1.1.jar:/home/hadoop/storm/lib/junit-3.8.1.jar:/home/hadoop/storm/lib/compojure-1.1.3.jar:/home/hadoop/storm/lib/commons-lang-2.5.jar:/home/hadoop/storm/lib/netty-3.6.3.Final.jar:/home/hadoop/storm/lib/servlet-api-2.5.jar:/home/hadoop/storm/lib/jgrapht-0.8.3.jar:/home/hadoop/storm/lib/minlog-1.2.jar:/home/hadoop/storm/lib/clout-1.0.1.jar:/home/hadoop/storm/lib/snakeyaml-1.11.jar:/home/hadoop/storm/lib/tools.cli-0.2.2.jar:/home/hadoop/storm/lib/jetty-util-6.1.26.jar:/home/hadoop/storm/lib/zookeeper-3.3.3.jar:/home/hadoop/storm/lib/tools.macro-0.1.0.jar:/home/hadoop/storm/lib/objenesis-1.2.jar:/home/hadoop/storm/lib/httpclient-4.1.1.jar:/home/hadoop/storm/lib/json-simple-1.1.jar:/home/hadoop/storm/lib/asm-4.0.jar:/home/hadoop/storm/lib/httpcore-4.1.jar:/home/hadoop/storm/lib/jline-0.9.94.jar:/home/hadoop/storm/lib/libthrift7-0.7.0.jar:/home/hadoop/storm/lib/ring-devel-0.3.11.jar:/home/hadoop/storm/lib/curator-framework-1.0.1.jar:/home/hadoop/storm/lib/carbonite-1.5.0.jar:/home/hadoop/storm/lib/commons-io-1.4.jar:/home/hadoop/storm/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm/lib/hiccup-0.3.6.jar:/home/hadoop/storm/lib/disruptor-2.10.1.jar:/home/hadoop/storm/lib/jzmq-2.1.0.jar:/home/hadoop/storm/lib/clojure-1.4.0.jar:/home/hadoop/storm/lib/commons-exec-1.1.jar:/home/hadoop/storm/lib/joda-time-2.0.jar:/home/hadoop/storm/lib/logback-classic-1.0.6.jar:/home/hadoop/storm/lib/curator-client-1.0.1.jar:/home/hadoop/storm/lib/clj-time-0.4.1.jar:/home/hadoop/storm/lib/tools.logging-0.2.3.jar:/home/hadoop/storm/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm/lib/kryo-2.17.jar:/home/hadoop/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm/lib/clj-stacktrace-0.2.2.jar:/home/hadoop/storm/lib/jetty-6.1.26.jar:/home/hadoop/storm/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm/lib/ring-core-1.1.5.jar:/home/hadoop/storm/lib/guava-13.0.jar:/home/hadoop/storm/lib/mockito-all-1.9.5.jar:/home/hadoop/storm/lib/core.incubator-0.1.0.jar:/home/hadoop/storm/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm/lib/logback-core-1.0.6.jar:/home/hadoop/storm/lib/commons-codec-1.4.jar:/home/hadoop/.storm -Dstorm.home=/home/hadoop/storm/bin/.. com.yahoo.storm.yarn.Client startUI -appId application_1376015903368_0010

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/home/hadoop/storm/lib/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

13/08/12 00:07:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

13/08/12 00:07:27 INFO yarn.StormOnYarn: application report for application_1376015903368_0010 :h2:9000

13/08/12 00:07:27 INFO yarn.StormOnYarn: Attaching to h2:9000 to talk to app master application_1376015903368_0010

Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused

at com.yahoo.storm.yarn.MasterClient.getConfiguredClient(MasterClient.java:42)

at com.yahoo.storm.yarn.StormOnYarn.getClient(StormOnYarn.java:118)

at com.yahoo.storm.yarn.StormMasterCommand.process(StormMasterCommand.java:75)

at com.yahoo.storm.yarn.Client.execute(Client.java:139)

at com.yahoo.storm.yarn.Client.main(Client.java:145)

Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused

at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)

at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)

at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:66)

at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:46)

at com.yahoo.storm.yarn.MasterClient.<init>(MasterClient.java:48)

at com.yahoo.storm.yarn.MasterClient.getConfiguredClient(MasterClient.java:36)

... 4 more

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:579)

at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)

... 9 more

在 2013年8月12日星期一UTC+8上午12时46分59秒，bertt...@gmail.com写道：

[Andy Feng]

We plan to add a "Get Started Guide" to storm-yarn soon. Here is the basic steps to use storm-yarn:

1. storm-yarn launch <master.yaml> -appname <appname> --output <file for appID>
2. storm-yarn getStormConfig <naster.yaml> -appId <appID from step 1> -output <storm.yaml>

3. cp <storm.yaml from step 2> .storm/storm.yaml
4. storm jar <Topology JAR> <MainClass> ...
5. storm kill <Topology ID for step 4>
6. storm-yarn shutdown <master.yaml> -appId <appID from step 2>

Appmaster should be launched in step 1. At that point, port 9000 is ready for storm-yarn commands against this app master (ex step 2 and step 6 above)

Before you perform step 4, you need to get storm.yaml via step 2. Step 3 is placing the received storm.yaml in the right place.

Please don't modify master_defaults.yaml. You should create your own master.yaml with the following info such as:

   - ZK servers and port 

   

Andy

[bertt...@gmail.com]

thanks, i think the guide will help us a lot.
在 2013年8月12日星期一UTC+8上午4时25分22秒，Andy Feng写道：

[bertt...@gmail.com]

Hi,andy:

在 2013年8月12日星期一UTC+8上午4时25分22秒，Andy Feng写道：

[bertt...@gmail.com]

hi,andy:

  can you give me an example of master.yaml? thank you!

在 2013年8月12日星期一UTC+8上午4时25分22秒，Andy Feng写道：