These are the yarn logs. I see there are exceptions around yarn user not being able to write to hdfs file system - not sure how to fix that as storm-yarn seems to be executed as yarn user.
There is another error with bind exception for starting the Master Thrift server which seems to be happening because the port might already be in use. Is there a configuration I can change to use a different port ?
Container: container_1385621683413_0008_02_000001 on mm-hdp-dn-95338.phx-os1.stratus.dev.ebay.com_45454
=========================================================================================================
LogType: nimbus.log
LogLength: 8772
Log Contents:
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.7.0_45
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:java.home=/usr/java/jdk1.7.0_45/jre
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:java.class.path=conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/storm-core-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-netty-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/lib/math.numeric-tower-0.0.1.jar:./storm/storm-0.9.0-wip21/lib/tools.logging-0.2.3.jar:./storm/storm-0.9.0-wip21/lib/log4j-over-slf4j-1.6.6.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5-20081211.jar:./storm/storm-0.9.0-wip21/lib/tools.cli-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/commons-codec-1.4.jar:./storm/storm-0.9.0-wip21/lib/clojure-1.4.0.jar:./storm/storm-0.9.0-wip21/lib/minlog-1.2.jar:./storm/storm-0.9.0-wip21/lib/reflectasm-1.07-shaded.jar:./storm/storm-0.9.0-wip21/lib/libthrift7-0.7.0-2.jar:./storm/storm-0.9.0-wip21/lib/tools.macro-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/json-simple-1.1.jar:./storm/storm-0.9.0-wip21/lib/ring-servlet-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/slf4j-api-1.6.5.jar:./storm/storm-0.9.0-wip21/lib/commons-lang-2.5.jar:./storm/storm-0.9.0-wip21/lib/ring-jetty-adapter-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/clout-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/jline-0.9.94.jar:./storm/storm-0.9.0-wip21/lib/logback-core-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/objenesis-1.2.jar:./storm/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/curator-client-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/asm-4.0.jar:./storm/storm-0.9.0-wip21/lib/jzmq-2.1.0.jar:./storm/storm-0.9.0-wip21/lib/jgrapht-0.8.3.jar:./storm/storm-0.9.0-wip21/lib/snakeyaml-1.11.jar:./storm/storm-0.9.0-wip21/lib/carbonite-1.5.0.jar:./storm/storm-0.9.0-wip21/lib/mockito-all-1.9.5.jar:./storm/storm-0.9.0-wip21/lib/netty-3.6.3.Final.jar:./storm/storm-0.9.0-wip21/lib/ring-devel-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/commons-io-1.4.jar:./storm/storm-0.9.0-wip21/lib/compojure-1.1.3.jar:./storm/storm-0.9.0-wip21/lib/zookeeper-3.3.3.jar:./storm/storm-0.9.0-wip21/lib/disruptor-2.10.1.jar:./storm/storm-0.9.0-wip21/lib/clj-time-0.4.1.jar:./storm/storm-0.9.0-wip21/lib/jetty-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/jetty-util-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/curator-framework-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/commons-logging-1.1.1.jar:./storm/storm-0.9.0-wip21/lib/guava-13.0.jar:./storm/storm-0.9.0-wip21/lib/commons-exec-1.1.jar:./storm/storm-0.9.0-wip21/lib/commons-fileupload-1.2.1.jar:./storm/storm-0.9.0-wip21/lib/junit-3.8.1.jar:./storm/storm-0.9.0-wip21/lib/ring-core-1.1.5.jar:./storm/storm-0.9.0-wip21/lib/core.incubator-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/kryo-2.17.jar:./storm/storm-0.9.0-wip21/lib/hiccup-0.3.6.jar:./storm/storm-0.9.0-wip21/lib/joda-time-2.0.jar:./storm/storm-0.9.0-wip21/lib/clj-stacktrace-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/httpclient-4.1.1.jar:./storm/storm-0.9.0-wip21/lib/httpcore-4.1.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5.jar
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:
os.name=Linux
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:os.version=2.6.32-279.22.1.el6.x86_64
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:
user.name=yarn
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/yarn
2013-12-02 21:40:20 o.a.z.ZooKeeper [INFO] Client environment:user.dir=/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/container_1385621683413_0008_02_000001
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.version=1.7.0_45
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.vendor=Oracle Corporation
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.home=/usr/java/jdk1.7.0_45/jre
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.class.path=conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/storm-core-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-netty-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/lib/math.numeric-tower-0.0.1.jar:./storm/storm-0.9.0-wip21/lib/tools.logging-0.2.3.jar:./storm/storm-0.9.0-wip21/lib/log4j-over-slf4j-1.6.6.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5-20081211.jar:./storm/storm-0.9.0-wip21/lib/tools.cli-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/commons-codec-1.4.jar:./storm/storm-0.9.0-wip21/lib/clojure-1.4.0.jar:./storm/storm-0.9.0-wip21/lib/minlog-1.2.jar:./storm/storm-0.9.0-wip21/lib/reflectasm-1.07-shaded.jar:./storm/storm-0.9.0-wip21/lib/libthrift7-0.7.0-2.jar:./storm/storm-0.9.0-wip21/lib/tools.macro-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/json-simple-1.1.jar:./storm/storm-0.9.0-wip21/lib/ring-servlet-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/slf4j-api-1.6.5.jar:./storm/storm-0.9.0-wip21/lib/commons-lang-2.5.jar:./storm/storm-0.9.0-wip21/lib/ring-jetty-adapter-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/clout-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/jline-0.9.94.jar:./storm/storm-0.9.0-wip21/lib/logback-core-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/objenesis-1.2.jar:./storm/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/curator-client-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/asm-4.0.jar:./storm/storm-0.9.0-wip21/lib/jzmq-2.1.0.jar:./storm/storm-0.9.0-wip21/lib/jgrapht-0.8.3.jar:./storm/storm-0.9.0-wip21/lib/snakeyaml-1.11.jar:./storm/storm-0.9.0-wip21/lib/carbonite-1.5.0.jar:./storm/storm-0.9.0-wip21/lib/mockito-all-1.9.5.jar:./storm/storm-0.9.0-wip21/lib/netty-3.6.3.Final.jar:./storm/storm-0.9.0-wip21/lib/ring-devel-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/commons-io-1.4.jar:./storm/storm-0.9.0-wip21/lib/compojure-1.1.3.jar:./storm/storm-0.9.0-wip21/lib/zookeeper-3.3.3.jar:./storm/storm-0.9.0-wip21/lib/disruptor-2.10.1.jar:./storm/storm-0.9.0-wip21/lib/clj-time-0.4.1.jar:./storm/storm-0.9.0-wip21/lib/jetty-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/jetty-util-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/curator-framework-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/commons-logging-1.1.1.jar:./storm/storm-0.9.0-wip21/lib/guava-13.0.jar:./storm/storm-0.9.0-wip21/lib/commons-exec-1.1.jar:./storm/storm-0.9.0-wip21/lib/commons-fileupload-1.2.1.jar:./storm/storm-0.9.0-wip21/lib/junit-3.8.1.jar:./storm/storm-0.9.0-wip21/lib/ring-core-1.1.5.jar:./storm/storm-0.9.0-wip21/lib/core.incubator-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/kryo-2.17.jar:./storm/storm-0.9.0-wip21/lib/hiccup-0.3.6.jar:./storm/storm-0.9.0-wip21/lib/joda-time-2.0.jar:./storm/storm-0.9.0-wip21/lib/clj-stacktrace-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/httpclient-4.1.1.jar:./storm/storm-0.9.0-wip21/lib/httpcore-4.1.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5.jar
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.io.tmpdir=/tmp
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.compiler=<NA>
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:
os.name=Linux
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.arch=amd64
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.version=2.6.32-279.22.1.el6.x86_64
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:
user.name=yarn
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.home=/home/yarn
2013-12-02 21:40:20 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.dir=/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/container_1385621683413_0008_02_000001
LogType: stderr
LogLength: 8501
Log Contents:
21:40:16,335 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
21:40:16,336 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
21:40:16,340 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:40:16,341 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
21:40:16,341 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:40:16,341 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/hadoop/yarn/filecache/11/storm.zip/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar!/logback.xml]
21:40:16,712 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
21:40:16,723 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeFilter scanning period to 1 minutes
21:40:16,723 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will scan for changes in [[/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]] every 60 seconds.
21:40:16,723 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Adding ReconfigureOnChangeFilter as a turbo filter
21:40:16,740 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:40:16,755 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [A1]
21:40:17,012 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@476d50ae - No compression will be used
21:40:17,042 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:40:17,186 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - Active log file name: /hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_02_000001/nimbus.log
21:40:17,186 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - File property is set to [/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_02_000001/nimbus.log]
21:40:17,187 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:40:17,187 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ACCESS]
21:40:17,189 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@59cbf424 - No compression will be used
21:40:17,196 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:40:17,197 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - Active log file name: access.log
21:40:17,197 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - File property is set to [access.log]
21:40:17,197 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
21:40:17,198 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [A1] to Logger[ROOT]
21:40:17,198 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [backtype.storm.security.auth.authorizer] to false
21:40:17,199 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - backtype.storm.security.auth.authorizer level set to INFO
21:40:17,199 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ACCESS] to Logger[backtype.storm.security.auth.authorizer]
21:40:17,199 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
21:40:17,213 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@5d09958a - Registering current configuration as safe fallback point
21:40:17,022 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
21:40:17,023 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
21:40:17,023 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:40:17,025 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
21:40:17,025 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:40:17,025 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/hadoop/yarn/filecache/11/storm.zip/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar!/logback.xml]
21:40:17,152 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
21:40:17,157 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeFilter scanning period to 1 minutes
21:40:17,157 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will scan for changes in [[/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]] every 60 seconds.
21:40:17,157 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Adding ReconfigureOnChangeFilter as a turbo filter
21:40:17,164 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:40:17,175 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [A1]
21:40:17,321 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@e7f21d6 - No compression will be used
21:40:17,331 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:40:17,387 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - Active log file name: /hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_02_000001/ui.log
21:40:17,387 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - File property is set to [/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_02_000001/ui.log]
21:40:17,388 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:40:17,388 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ACCESS]
21:40:17,390 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@3c867009 - No compression will be used
21:40:17,392 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:40:17,393 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - Active log file name: access.log
21:40:17,393 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - File property is set to [access.log]
21:40:17,393 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
21:40:17,393 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [A1] to Logger[ROOT]
21:40:17,395 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [backtype.storm.security.auth.authorizer] to false
21:40:17,395 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - backtype.storm.security.auth.authorizer level set to INFO
21:40:17,395 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ACCESS] to Logger[backtype.storm.security.auth.authorizer]
21:40:17,395 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
21:40:17,397 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@1e257dea - Registering current configuration as safe fallback point
LogType: stdout
LogLength: 9764
Log Contents:
13/12/02 21:40:11 INFO yarn.MasterServer: Starting the AM!!!!
13/12/02 21:40:12 INFO yarn.MasterServer: appAttemptID from env:appattempt_1385621683413_0008_000002
13/12/02 21:40:12 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies : 500
13/12/02 21:40:13 INFO yarn.StormMasterServerHandler: Storm master host:10.9.210.218
13/12/02 21:40:13 INFO yarn.MasterServer: launch nimbus
13/12/02 21:40:13 INFO yarn.StormMasterServerHandler: starting nimbus...
13/12/02 21:40:13 INFO yarn.MasterServer: launch ui
13/12/02 21:40:13 INFO yarn.StormMasterServerHandler: starting UI...
13/12/02 21:40:13 INFO yarn.MasterServer: launch 1 supervisors
13/12/02 21:40:13 INFO yarn.StormMasterServerHandler: adding 1 supervisors...
13/12/02 21:40:13 INFO yarn.StormAMRMClient: Added 1 supervisors, but not requesting containers now.
13/12/02 21:40:13 INFO yarn.StormMasterServerHandler: Running: /usr/java/jdk1.7.0_45/bin/java -server -Dstorm.home=./storm/storm-0.9.0-wip21/ -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=storm.yaml -cp conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/*:./storm/storm-0.9.0-wip21/lib/* -Xmx1024m -Dlogfile.name=/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_02_000001/nimbus.log backtype.storm.daemon.nimbus
13/12/02 21:40:13 INFO yarn.StormMasterServerHandler: Running: /usr/java/jdk1.7.0_45/bin/java -server -Dstorm.home=./storm/storm-0.9.0-wip21/ -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=storm.yaml -cp conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/*:./storm/storm-0.9.0-wip21/lib/* -Xmx768m -Dstorm.options=nimbus.host=localhost -Dlogfile.name=/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_02_000001/ui.log backtype.storm.ui.core
13/12/02 21:40:14 INFO yarn.MasterServer: Got a registration response maximumCapability { memory: 6144 virtual_cores: 32 }
13/12/02 21:40:14 INFO yarn.MasterServer: Max Capability <memory:6144, vCores:32>
13/12/02 21:40:14 INFO yarn.StormAMRMClient: Max Capability is now <memory:6144, vCores:32>
13/12/02 21:40:14 INFO yarn.MasterServer: Starting HB thread
13/12/02 21:40:14 INFO yarn.MasterServer: Starting launcher
13/12/02 21:40:14 INFO yarn.MasterServer: Starting Master Thrift Server
13/12/02 21:40:16 INFO yarn.MasterServer: HB: Received allocated containers (1)
13/12/02 21:40:16 INFO yarn.MasterServer: HB: Supervisors are to run, so queueing (1) containers...
13/12/02 21:40:16 INFO yarn.MasterServer: LAUNCHER: Taking container with id (container_1385621683413_0008_02_000002) from the queue.
13/12/02 21:40:16 INFO yarn.MasterServer: LAUNCHER: Supervisors are to run, so launching container id (container_1385621683413_0008_02_000002)
13/12/02 21:40:18 ERROR yarn.MasterServer: Launcher thread I/O exception :
org.apache.hadoop.security.AccessControlException: Permission denied: user=yarn, access=WRITE, inode="/user":hdfs:hdfs:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:234)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:214)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:158)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5152)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5134)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5108)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3368)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3338)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3310)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:694)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:502)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59598)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2352)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2321)
at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:828)
at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:824)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:78)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:824)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:817)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1929)
at com.yahoo.storm.yarn.Util.createConfigurationFileInFs(Util.java:139)
at com.yahoo.storm.yarn.StormAMRMClient.launchSupervisorOnContainer(StormAMRMClient.java:185)
at com.yahoo.storm.yarn.MasterServer$2.run(MasterServer.java:199)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=yarn, access=WRITE, inode="/user":hdfs:hdfs:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:234)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:214)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:158)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5152)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5134)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5108)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3368)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3338)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3310)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:694)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:502)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59598)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042)
at org.apache.hadoop.ipc.Client.call(Client.java:1347)
at org.apache.hadoop.ipc.Client.call(Client.java:1300)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:188)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:467)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2350)
... 10 more
LogType: ui.log
LogLength: 0
Log Contents:
Container: container_1385621683413_0008_01_000001 on mm-hdp-rm-95334.phx-os1.stratus.dev.ebay.com_45454
=========================================================================================================
LogType: nimbus.log
LogLength: 15793
Log Contents:
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.7.0_45
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:java.home=/usr/java/jdk1.7.0_45/jre
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:java.class.path=conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/storm-core-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-netty-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/lib/math.numeric-tower-0.0.1.jar:./storm/storm-0.9.0-wip21/lib/tools.logging-0.2.3.jar:./storm/storm-0.9.0-wip21/lib/log4j-over-slf4j-1.6.6.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5-20081211.jar:./storm/storm-0.9.0-wip21/lib/tools.cli-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/commons-codec-1.4.jar:./storm/storm-0.9.0-wip21/lib/clojure-1.4.0.jar:./storm/storm-0.9.0-wip21/lib/minlog-1.2.jar:./storm/storm-0.9.0-wip21/lib/reflectasm-1.07-shaded.jar:./storm/storm-0.9.0-wip21/lib/libthrift7-0.7.0-2.jar:./storm/storm-0.9.0-wip21/lib/tools.macro-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/json-simple-1.1.jar:./storm/storm-0.9.0-wip21/lib/ring-servlet-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/slf4j-api-1.6.5.jar:./storm/storm-0.9.0-wip21/lib/commons-lang-2.5.jar:./storm/storm-0.9.0-wip21/lib/ring-jetty-adapter-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/clout-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/jline-0.9.94.jar:./storm/storm-0.9.0-wip21/lib/logback-core-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/objenesis-1.2.jar:./storm/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/curator-client-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/asm-4.0.jar:./storm/storm-0.9.0-wip21/lib/jzmq-2.1.0.jar:./storm/storm-0.9.0-wip21/lib/jgrapht-0.8.3.jar:./storm/storm-0.9.0-wip21/lib/snakeyaml-1.11.jar:./storm/storm-0.9.0-wip21/lib/carbonite-1.5.0.jar:./storm/storm-0.9.0-wip21/lib/mockito-all-1.9.5.jar:./storm/storm-0.9.0-wip21/lib/netty-3.6.3.Final.jar:./storm/storm-0.9.0-wip21/lib/ring-devel-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/commons-io-1.4.jar:./storm/storm-0.9.0-wip21/lib/compojure-1.1.3.jar:./storm/storm-0.9.0-wip21/lib/zookeeper-3.3.3.jar:./storm/storm-0.9.0-wip21/lib/disruptor-2.10.1.jar:./storm/storm-0.9.0-wip21/lib/clj-time-0.4.1.jar:./storm/storm-0.9.0-wip21/lib/jetty-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/jetty-util-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/curator-framework-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/commons-logging-1.1.1.jar:./storm/storm-0.9.0-wip21/lib/guava-13.0.jar:./storm/storm-0.9.0-wip21/lib/commons-exec-1.1.jar:./storm/storm-0.9.0-wip21/lib/commons-fileupload-1.2.1.jar:./storm/storm-0.9.0-wip21/lib/junit-3.8.1.jar:./storm/storm-0.9.0-wip21/lib/ring-core-1.1.5.jar:./storm/storm-0.9.0-wip21/lib/core.incubator-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/kryo-2.17.jar:./storm/storm-0.9.0-wip21/lib/hiccup-0.3.6.jar:./storm/storm-0.9.0-wip21/lib/joda-time-2.0.jar:./storm/storm-0.9.0-wip21/lib/clj-stacktrace-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/httpclient-4.1.1.jar:./storm/storm-0.9.0-wip21/lib/httpcore-4.1.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5.jar
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:
os.name=Linux
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:os.version=2.6.32-279.22.1.el6.x86_64
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:
user.name=yarn
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/yarn
2013-12-02 21:41:24 o.a.z.ZooKeeper [INFO] Client environment:user.dir=/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/container_1385621683413_0008_01_000001
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.version=1.7.0_45
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.vendor=Oracle Corporation
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.home=/usr/java/jdk1.7.0_45/jre
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.class.path=conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/storm-core-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/storm-netty-0.9.0-wip21.jar:./storm/storm-0.9.0-wip21/lib/math.numeric-tower-0.0.1.jar:./storm/storm-0.9.0-wip21/lib/tools.logging-0.2.3.jar:./storm/storm-0.9.0-wip21/lib/log4j-over-slf4j-1.6.6.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5-20081211.jar:./storm/storm-0.9.0-wip21/lib/tools.cli-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/commons-codec-1.4.jar:./storm/storm-0.9.0-wip21/lib/clojure-1.4.0.jar:./storm/storm-0.9.0-wip21/lib/minlog-1.2.jar:./storm/storm-0.9.0-wip21/lib/reflectasm-1.07-shaded.jar:./storm/storm-0.9.0-wip21/lib/libthrift7-0.7.0-2.jar:./storm/storm-0.9.0-wip21/lib/tools.macro-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/json-simple-1.1.jar:./storm/storm-0.9.0-wip21/lib/ring-servlet-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/slf4j-api-1.6.5.jar:./storm/storm-0.9.0-wip21/lib/commons-lang-2.5.jar:./storm/storm-0.9.0-wip21/lib/ring-jetty-adapter-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/clout-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/jline-0.9.94.jar:./storm/storm-0.9.0-wip21/lib/logback-core-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/objenesis-1.2.jar:./storm/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar:./storm/storm-0.9.0-wip21/lib/curator-client-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/asm-4.0.jar:./storm/storm-0.9.0-wip21/lib/jzmq-2.1.0.jar:./storm/storm-0.9.0-wip21/lib/jgrapht-0.8.3.jar:./storm/storm-0.9.0-wip21/lib/snakeyaml-1.11.jar:./storm/storm-0.9.0-wip21/lib/carbonite-1.5.0.jar:./storm/storm-0.9.0-wip21/lib/mockito-all-1.9.5.jar:./storm/storm-0.9.0-wip21/lib/netty-3.6.3.Final.jar:./storm/storm-0.9.0-wip21/lib/ring-devel-0.3.11.jar:./storm/storm-0.9.0-wip21/lib/commons-io-1.4.jar:./storm/storm-0.9.0-wip21/lib/compojure-1.1.3.jar:./storm/storm-0.9.0-wip21/lib/zookeeper-3.3.3.jar:./storm/storm-0.9.0-wip21/lib/disruptor-2.10.1.jar:./storm/storm-0.9.0-wip21/lib/clj-time-0.4.1.jar:./storm/storm-0.9.0-wip21/lib/jetty-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/jetty-util-6.1.26.jar:./storm/storm-0.9.0-wip21/lib/curator-framework-1.0.1.jar:./storm/storm-0.9.0-wip21/lib/commons-logging-1.1.1.jar:./storm/storm-0.9.0-wip21/lib/guava-13.0.jar:./storm/storm-0.9.0-wip21/lib/commons-exec-1.1.jar:./storm/storm-0.9.0-wip21/lib/commons-fileupload-1.2.1.jar:./storm/storm-0.9.0-wip21/lib/junit-3.8.1.jar:./storm/storm-0.9.0-wip21/lib/ring-core-1.1.5.jar:./storm/storm-0.9.0-wip21/lib/core.incubator-0.1.0.jar:./storm/storm-0.9.0-wip21/lib/kryo-2.17.jar:./storm/storm-0.9.0-wip21/lib/hiccup-0.3.6.jar:./storm/storm-0.9.0-wip21/lib/joda-time-2.0.jar:./storm/storm-0.9.0-wip21/lib/clj-stacktrace-0.2.2.jar:./storm/storm-0.9.0-wip21/lib/httpclient-4.1.1.jar:./storm/storm-0.9.0-wip21/lib/httpcore-4.1.jar:./storm/storm-0.9.0-wip21/lib/servlet-api-2.5.jar
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.io.tmpdir=/tmp
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:java.compiler=<NA>
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:
os.name=Linux
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.arch=amd64
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:os.version=2.6.32-279.22.1.el6.x86_64
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:
user.name=yarn
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.home=/home/yarn
2013-12-02 21:41:24 o.a.z.s.ZooKeeperServer [INFO] Server environment:user.dir=/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/container_1385621683413_0008_01_000001
2013-12-02 21:41:25 b.s.d.nimbus [INFO] Starting Nimbus with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000, "topology.skip.missing.kryo.registrations" false, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "
nimbusui.startup.ms" 10000, "topology.trident.batch.emit.interval.millis" 500, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "storm.zip.visibility" "PUBLIC", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "storm-local", "storm.messaging.netty.buffer_size" 1048576, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.messaging.netty.min_wait_ms" 1000, "storm.zookeeper.port" 2181, "master.container.size-mb" 5120, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "master.thrift.port" 9000, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "yarn.report.wait.millis" 10000, "master.initial-num-supervisors" 1, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 100, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "master.heartbeat.interval.millis" 1000, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "
topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "master.host" "localhost", "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "storm.zip.path" "/lib/storm/0.9.0-wip21/storm.zip", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "storm.messaging.netty.max_wait_ms" 5000, "drpc.request.timeout.secs" 600, "master.timeout.secs" 1000, "storm.local.mode.zmq" false, "ui.port" 7070, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "distributed", "topology.optimize" true, "topology.max.task.parallelism" nil, "master.container.priority" 0}
2013-12-02 21:41:25 b.s.d.nimbus [INFO] Using default scheduler
2013-12-02 21:41:25 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2013-12-02 21:41:25 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost:2181 sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@389ea6dd
2013-12-02 21:41:25 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181
2013-12-02 21:41:25 o.a.z.ClientCnxn [INFO] Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2013-12-02 21:41:25 o.a.z.ClientCnxn [INFO] Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x3426e17ea96002a, negotiated timeout = 20000
2013-12-02 21:41:25 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2013-12-02 21:41:25 o.a.z.ZooKeeper [INFO] Session: 0x3426e17ea96002a closed
2013-12-02 21:41:25 o.a.z.ClientCnxn [INFO] EventThread shut down
2013-12-02 21:41:25 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
2013-12-02 21:41:25 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost:2181/storm sessionTimeout=20000 watcher=com.netflix.curator.ConnectionState@213b0eef
2013-12-02 21:41:25 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181
2013-12-02 21:41:25 o.a.z.ClientCnxn [INFO] Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2013-12-02 21:41:25 o.a.z.ClientCnxn [INFO] Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x3426e17ea96002b, negotiated timeout = 20000
2013-12-02 21:41:25 b.s.d.nimbus [INFO] Starting Nimbus server...
2013-12-02 21:41:46 b.s.d.nimbus [ERROR] Error when processing event
java.io.IOException: Unable to create directory storm-local/nimbus
at org.apache.commons.io.FileUtils.forceMkdir(FileUtils.java:1488) ~[commons-io-1.4.jar:1.4]
at backtype.storm.config$master_local_dir.invoke(config.clj:112) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.config$master_stormdist_root.invoke(config.clj:118) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.nimbus$code_ids.invoke(nimbus.clj:737) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.nimbus$cleanup_storm_ids.invoke(nimbus.clj:746) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.nimbus$do_cleanup$fn__5456.invoke(nimbus.clj:811) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.nimbus$do_cleanup.invoke(nimbus.clj:810) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.nimbus$fn__5508$exec_fn__1224__auto____5509$fn__5514.invoke(nimbus.clj:890) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.timer$schedule_recurring$this__3005.invoke(timer.clj:77) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.timer$mk_timer$fn__2988$fn__2989.invoke(timer.clj:33) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.timer$mk_timer$fn__2988.invoke(timer.clj:26) [storm-core-0.9.0-wip21.jar:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
2013-12-02 21:41:46 b.s.util [INFO] Halting process: ("Error when processing an event")
LogType: stderr
LogLength: 8501
Log Contents:
21:41:21,526 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
21:41:21,526 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
21:41:21,528 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:41:21,532 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
21:41:21,532 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:41:21,532 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/hadoop/yarn/filecache/11/storm.zip/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar!/logback.xml]
21:41:21,662 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
21:41:21,667 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeFilter scanning period to 1 minutes
21:41:21,667 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will scan for changes in [[/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]] every 60 seconds.
21:41:21,667 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Adding ReconfigureOnChangeFilter as a turbo filter
21:41:21,676 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:41:21,686 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [A1]
21:41:21,853 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@476d50ae - No compression will be used
21:41:21,863 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:41:21,919 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - Active log file name: /hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_01_000001/nimbus.log
21:41:21,919 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - File property is set to [/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_01_000001/nimbus.log]
21:41:21,922 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:41:21,922 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ACCESS]
21:41:21,924 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@59cbf424 - No compression will be used
21:41:21,927 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:41:21,928 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - Active log file name: access.log
21:41:21,928 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - File property is set to [access.log]
21:41:21,928 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
21:41:21,928 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [A1] to Logger[ROOT]
21:41:21,929 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [backtype.storm.security.auth.authorizer] to false
21:41:21,929 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - backtype.storm.security.auth.authorizer level set to INFO
21:41:21,929 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ACCESS] to Logger[backtype.storm.security.auth.authorizer]
21:41:21,929 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
21:41:21,931 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@5d09958a - Registering current configuration as safe fallback point
21:41:22,245 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
21:41:22,255 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
21:41:22,255 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:41:22,256 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
21:41:22,256 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]
21:41:22,256 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/hadoop/yarn/filecache/11/storm.zip/storm-0.9.0-wip21/storm-console-logging-0.9.0-wip21.jar!/logback.xml]
21:41:22,362 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
21:41:22,365 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Setting ReconfigureOnChangeFilter scanning period to 1 minutes
21:41:22,365 |-INFO in ReconfigureOnChangeFilter{invocationCounter=0} - Will scan for changes in [[/hadoop/yarn/usercache/hdfs/appcache/application_1385621683413_0008/filecache/11/conf/logback.xml]] every 60 seconds.
21:41:22,365 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - Adding ReconfigureOnChangeFilter as a turbo filter
21:41:22,368 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:41:22,380 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [A1]
21:41:22,516 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@e7f21d6 - No compression will be used
21:41:22,526 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:41:22,576 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - Active log file name: /hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_01_000001/ui.log
21:41:22,576 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[A1] - File property is set to [/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_01_000001/ui.log]
21:41:22,578 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:41:22,578 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [ACCESS]
21:41:22,580 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@3c867009 - No compression will be used
21:41:22,581 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:41:22,582 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - Active log file name: access.log
21:41:22,582 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[ACCESS] - File property is set to [access.log]
21:41:22,582 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
21:41:22,582 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [A1] to Logger[ROOT]
21:41:22,583 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [backtype.storm.security.auth.authorizer] to false
21:41:22,583 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - backtype.storm.security.auth.authorizer level set to INFO
21:41:22,583 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [ACCESS] to Logger[backtype.storm.security.auth.authorizer]
21:41:22,583 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
21:41:22,584 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@1e257dea - Registering current configuration as safe fallback point
LogType: stdout
LogLength: 10965
Log Contents:
13/12/02 21:41:16 INFO yarn.MasterServer: Starting the AM!!!!
13/12/02 21:41:17 INFO yarn.MasterServer: appAttemptID from env:appattempt_1385621683413_0008_000001
13/12/02 21:41:17 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies : 500
13/12/02 21:41:18 INFO yarn.StormMasterServerHandler: Storm master host:10.9.209.246
13/12/02 21:41:18 INFO yarn.MasterServer: launch nimbus
13/12/02 21:41:18 INFO yarn.StormMasterServerHandler: starting nimbus...
13/12/02 21:41:18 INFO yarn.MasterServer: launch ui
13/12/02 21:41:18 INFO yarn.StormMasterServerHandler: starting UI...
13/12/02 21:41:18 INFO yarn.MasterServer: launch 1 supervisors
13/12/02 21:41:18 INFO yarn.StormMasterServerHandler: adding 1 supervisors...
13/12/02 21:41:18 INFO yarn.StormAMRMClient: Added 1 supervisors, but not requesting containers now.
13/12/02 21:41:18 INFO yarn.StormMasterServerHandler: Running: /usr/java/jdk1.7.0_45/bin/java -server -Dstorm.home=./storm/storm-0.9.0-wip21/ -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=storm.yaml -cp conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/*:./storm/storm-0.9.0-wip21/lib/* -Xmx768m -Dstorm.options=nimbus.host=localhost -Dlogfile.name=/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_01_000001/ui.log backtype.storm.ui.core
13/12/02 21:41:18 INFO yarn.StormMasterServerHandler: Running: /usr/java/jdk1.7.0_45/bin/java -server -Dstorm.home=./storm/storm-0.9.0-wip21/ -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=storm.yaml -cp conf:./storm/storm-0.9.0-wip21/:./storm/storm-0.9.0-wip21/*:./storm/storm-0.9.0-wip21/lib/* -Xmx1024m -Dlogfile.name=/hadoop/yarn/application_1385621683413_0008/container_1385621683413_0008_01_000001/nimbus.log backtype.storm.daemon.nimbus
13/12/02 21:41:19 INFO yarn.MasterServer: Got a registration response maximumCapability { memory: 6144 virtual_cores: 32 }
13/12/02 21:41:19 INFO yarn.MasterServer: Max Capability <memory:6144, vCores:32>
13/12/02 21:41:19 INFO yarn.StormAMRMClient: Max Capability is now <memory:6144, vCores:32>
13/12/02 21:41:19 INFO yarn.MasterServer: Starting HB thread
13/12/02 21:41:19 INFO yarn.MasterServer: Starting launcher
13/12/02 21:41:19 INFO yarn.MasterServer: Starting Master Thrift Server
Exception in thread "main" java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:181)
at org.mortbay.jetty.bio.SocketConnector.newServerSocket(SocketConnector.java:80)
at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
at org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.Server.doStart(Server.java:235)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at ring.adapter.jetty$run_jetty.invoke(jetty.clj:65)
at backtype.storm.ui.core$start_server_BANG_.invoke(core.clj:810)
at backtype.storm.ui.core$_main.invoke(core.clj:813)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.AFn.applyTo(AFn.java:151)
at backtype.storm.ui.core.main(Unknown Source)
13/12/02 21:41:34 INFO yarn.MasterServer: HB: Received allocated containers (1)
13/12/02 21:41:34 INFO yarn.MasterServer: HB: Supervisors are to run, so queueing (1) containers...
13/12/02 21:41:34 INFO yarn.MasterServer: LAUNCHER: Taking container with id (container_1385621683413_0008_01_000003) from the queue.
13/12/02 21:41:34 INFO yarn.MasterServer: LAUNCHER: Supervisors are to run, so launching container id (container_1385621683413_0008_01_000003)
13/12/02 21:41:35 ERROR yarn.MasterServer: Launcher thread I/O exception :
org.apache.hadoop.security.AccessControlException: Permission denied: user=yarn, access=WRITE, inode="/user":hdfs:hdfs:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:234)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:214)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:158)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5152)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5134)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5108)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3368)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3338)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3310)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:694)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:502)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59598)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2352)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2321)
at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:828)
at org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:824)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:78)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:824)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:817)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1929)
at com.yahoo.storm.yarn.Util.createConfigurationFileInFs(Util.java:139)
at com.yahoo.storm.yarn.StormAMRMClient.launchSupervisorOnContainer(StormAMRMClient.java:185)
at com.yahoo.storm.yarn.MasterServer$2.run(MasterServer.java:199)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=yarn, access=WRITE, inode="/user":hdfs:hdfs:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:234)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:214)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:158)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5152)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5134)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5108)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3368)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3338)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3310)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:694)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:502)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59598)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042)
at org.apache.hadoop.ipc.Client.call(Client.java:1347)
at org.apache.hadoop.ipc.Client.call(Client.java:1300)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:188)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:467)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2350)
... 10 more
LogType: ui.log
LogLength: 372
Log Contents:
2013-12-02 21:41:25 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog
2013-12-02 21:41:25 o.m.log [INFO] jetty-6.1.26
2013-12-02 21:41:25 o.m.log [WARN] failed Server@74a21183: java.net.BindException: Address already in use