mvn package fails with storm-yarn 1.0-alpha

237 views

Skip to first unread message

zhangfenghua

unread,

Oct 16, 2013, 11:23:02 PM10/16/13

to storm...@googlegroups.com

yarn version：

Hadoop 2.0.0-cdh4.4.0

Subversion file:///data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hadoop-2.0.0-cdh4.4.0/src/hadoop-common-project/hadoop-common -r c0eba6cd38c984557e96a16ccd7356b7de835e79

Compiled by jenkins on Tue Sep 3 19:33:17 PDT 2013

From source with checksum ac7e170aa709b3ace13dc5f775487180

This command was run using /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0.jar

yarn CLASSPATH

/etc/hadoop/conf:/etc/hadoop/conf:/etc/hadoop/conf:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop/lib/*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop/.//*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop-hdfs/.//*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop-yarn/lib/*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop-yarn/.//*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-mapreduce/lib/*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-mapreduce/.//*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop-yarn/.//*:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec/../../hadoop-yarn/lib/*

mvn package

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building storm-yarn 1.0-alpha

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ storm-yarn ---

[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!

[INFO] Copying 2 resources

[INFO]

[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ storm-yarn ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ storm-yarn ---

[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!

[INFO] Copying 2 resources

[INFO]

[INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @ storm-yarn ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-jar-plugin:2.4:jar (default) @ storm-yarn ---

[INFO]

[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ storm-yarn ---

[INFO] Surefire report directory: /usr/local/storm-yarn/target/surefire-reports

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running com.yahoo.storm.yarn.TestStormCluster

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/root/.m2/repository/ch/qos/logback/logback-classic/1.0.6/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/root/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

11:19:12,623 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]

11:19:12,623 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback-test.xml] at [file:/usr/local/storm-yarn/target/test-classes/logback-test.xml]

11:19:12,851 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]

11:19:12,859 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]

11:19:12,915 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property

11:19:13,024 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO

11:19:13,025 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]

11:19:13,027 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.

11:19:13,030 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@4ece3b01 - Registering current configuration as safe fallback point

SLF4J: Actual binding is of type [ch.qos.logback.classic.selector.DefaultContextSelector]

11:19:13.044 [main] INFO c.yahoo.storm.yarn.EmbeddedZKServer - Starting up embedded Zookeeper server

11:19:13.473 [main] INFO c.yahoo.storm.yarn.EmbeddedZKServer - Zookeeper port allocated:60000

11:19:14.437 [main] INFO com.yahoo.storm.yarn.TestConfig - storm_home: lib/storm-0.9.0-wip21

11:19:14.446 [main] INFO c.yahoo.storm.yarn.TestStormCluster - Will be using storm found on PATH at lib/storm-0.9.0-wip21

11:19:14.906 [main] INFO c.yahoo.storm.yarn.TestStormCluster - OK: Nimbus does not seem to be running.

11:19:14.907 [main] INFO c.yahoo.storm.yarn.TestStormCluster - OK: UI does not seem to be running.

11:19:15.663 [main] INFO c.y.s.yarn.StormMasterServerHandler - Storm master host:172.21.40.67

11:19:15.685 [main] INFO com.yahoo.storm.yarn.MasterServer - launch nimbus

11:19:15.685 [main] INFO c.y.s.yarn.StormMasterServerHandler - starting nimbus...

11:19:15.687 [main] INFO com.yahoo.storm.yarn.MasterServer - launch ui

11:19:15.687 [main] INFO c.y.s.yarn.StormMasterServerHandler - starting UI...

11:19:15.688 [main] INFO com.yahoo.storm.yarn.MasterServer - launch 1 supervisors

11:19:15.688 [main] INFO c.y.s.yarn.StormMasterServerHandler - adding 1 supervisors...

11:19:15.737 [Thread-1] INFO c.y.s.yarn.StormMasterServerHandler - Running: /usr/local/jdk/bin/java -server -Dstorm.home=lib/storm-0.9.0-wip21 -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=storm.yaml -cp conf:lib/storm-0.9.0-wip21:lib/storm-0.9.0-wip21/*:lib/storm-0.9.0-wip21/lib/* -Xmx1024m -Dlogfile.name=null/nimbus.log backtype.storm.daemon.nimbus

11:19:15.737 [Thread-2] INFO c.y.s.yarn.StormMasterServerHandler - Running: /usr/local/jdk/bin/java -server -Dstorm.home=lib/storm-0.9.0-wip21 -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=storm.yaml -cp conf:lib/storm-0.9.0-wip21:lib/storm-0.9.0-wip21/*:lib/storm-0.9.0-wip21/lib/* -Xmx768m -Dstorm.options=nimbus.host=localhost -Dlogfile.name=null/ui.log backtype.storm.ui.core

11:19:15.748 [main] INFO c.yahoo.storm.yarn.TestStormCluster - Sleep to wait for the server to startup

11:19:15.778 [Thread-3] ERROR b.storm.security.auth.ThriftServer - ThriftServer is being stopped due to: org.apache.thrift7.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:9000.

org.apache.thrift7.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:9000.

at org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:89) ~[libthrift7-0.7.0-2.jar:0.7.0-2]

at org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:68) ~[libthrift7-0.7.0-2.jar:0.7.0-2]

at org.apache.thrift7.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:61) ~[libthrift7-0.7.0-2.jar:0.7.0-2]

at backtype.storm.security.auth.SimpleTransportPlugin.getServer(SimpleTransportPlugin.java:47) ~[storm-core-0.9.0-wip21.jar:na]

at backtype.storm.security.auth.ThriftServer.serve(ThriftServer.java:52) ~[storm-core-0.9.0-wip21.jar:na]

at com.yahoo.storm.yarn.TestStormCluster$1.run(TestStormCluster.java:77) [test-classes/:na]

at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]

2433 [main] INFO backtype.storm.daemon.nimbus - Starting Nimbus with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.builtin.metrics.bucket.size.secs" 60, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000, "topology.skip.missing.kryo.registrations" false, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "nimbusui.startup.ms" 10000, "topology.trident.batch.emit.interval.millis" 500, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "storm-local", "storm.messaging.netty.buffer_size" 1048576, "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.messaging.netty.min_wait_ms" 1000, "storm.zookeeper.port" 60000, "master.container.size-mb" 5120, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000, "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" nil, "master.thrift.port" 9000, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8, "yarn.report.wait.millis" 10000, "master.initial-num-supervisors" 1, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "storm.messaging.netty.max_retries" 100, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "master.heartbeat.interval.millis" 1000, "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "master.host" "localhost", "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "storm.messaging.netty.max_wait_ms" 5000, "drpc.request.timeout.secs" 600, "master.timeout.secs" 1000, "storm.local.mode.zmq" false, "ui.port" 7070, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "distributed", "topology.optimize" true, "topology.max.task.parallelism" nil, "master.container.priority" 0}

zhangfenghua

unread,

Oct 16, 2013, 11:39:24 PM10/16/13

to storm...@googlegroups.com

when I change the master.thrift.port: 9000 to master.thrift.port: 9090,the output Changed.

11:33:52.062 [Thread-239] INFO org.mortbay.log - Extract jar:file:/home/storm/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.1.0-beta/hadoop-yarn-common-2.1.0-beta.jar!/webapps/node to /tmp/Jetty_datanode10_50031_node____.fuc5qa/webapp

Oct 17, 2013 11:33:52 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider

INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"

Oct 17, 2013 11:33:53 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider

INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton"

11:33:53.529 [Thread-39] INFO org.mortbay.log - Started SelectChannelConnector@datanode10:52507

11:33:53.529 [Thread-239] INFO org.mortbay.log - Started SelectChannelConnector@datanode10:50031

11:33:53.531 [Thread-39] INFO o.apache.hadoop.yarn.webapp.WebApps - Web app /node started at 52507

11:33:53.531 [Thread-239] INFO o.apache.hadoop.yarn.webapp.WebApps - Web app /node started at 50031

Oct 17, 2013 11:33:53 AM com.google.inject.servlet.GuiceFilter setPipeline

WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected.

11:33:53.584 [Thread-39] INFO o.apache.hadoop.yarn.webapp.WebApps - Registered webapp guice modules

Oct 17, 2013 11:33:53 AM com.google.inject.servlet.GuiceFilter setPipeline

11:33:53.586 [Thread-239] INFO o.apache.hadoop.yarn.webapp.WebApps - Registered webapp guice modules

11:33:53.623 [Thread-39] INFO o.a.hadoop.yarn.util.RackResolver - Resolved datanode10 to /default-rack

11:33:53.623 [Thread-239] INFO o.a.hadoop.yarn.util.RackResolver - Resolved datanode10 to /default-rack

11:33:53.629 [Thread-39] INFO o.a.h.y.s.r.ResourceTrackerService - NodeManager from node datanode10(cmPort: 49347 httpPort: 52507) registered with capability: <memory:4096, vCores:8>, assigned nodeId datanode10:49347

11:33:53.629 [Thread-239] INFO o.a.h.y.s.r.ResourceTrackerService - NodeManager from node datanode10(cmPort: 60968 httpPort: 50031) registered with capability: <memory:4096, vCores:8>, assigned nodeId datanode10:60968

11:33:53.630 [Thread-39] INFO o.a.h.y.s.n.s.NMContainerTokenSecretManager - Rolling master-key for container-tokens, got key with id 2076303895

11:33:53.630 [Thread-239] INFO o.a.h.y.s.n.s.NMContainerTokenSecretManager - Rolling master-key for container-tokens, got key with id 2076303895

11:33:53.651 [Thread-239] INFO o.a.h.y.s.n.s.NMTokenSecretManagerInNM - Rolling master-key for nm-tokens, got key with id :-711402670

11:33:53.652 [Thread-239] INFO o.a.h.y.s.n.NodeStatusUpdaterImpl - Registered with ResourceManager as datanode10:60968 with total resource of <memory:4096, vCores:8>

11:33:53.652 [Thread-239] INFO o.a.h.y.s.n.NodeStatusUpdaterImpl - Notifying ContainerManager to unblock new container-requests

11:33:53.652 [Thread-39] INFO o.a.h.y.s.n.s.NMTokenSecretManagerInNM - Rolling master-key for nm-tokens, got key with id :-711402670

11:33:53.653 [Thread-39] INFO o.a.h.y.s.n.NodeStatusUpdaterImpl - Registered with ResourceManager as datanode10:49347 with total resource of <memory:4096, vCores:8>

11:33:53.653 [Thread-39] INFO o.a.h.y.s.n.NodeStatusUpdaterImpl - Notifying ContainerManager to unblock new container-requests

11:33:53.669 [AsyncDispatcher event handler] INFO o.a.h.y.s.r.rmnode.RMNodeImpl - datanode10:49347 Node Transitioned from NEW to RUNNING

11:33:53.670 [AsyncDispatcher event handler] INFO o.a.h.y.s.r.rmnode.RMNodeImpl - datanode10:60968 Node Transitioned from NEW to RUNNING

11:33:53.677 [ResourceManager Event Processor] INFO o.a.h.y.s.r.s.c.CapacityScheduler - Added node datanode10:49347 clusterResource: <memory:4096, vCores:8>

11:33:53.677 [ResourceManager Event Processor] INFO o.a.h.y.s.r.s.c.CapacityScheduler - Added node datanode10:60968 clusterResource: <memory:8192, vCores:16>

11:33:55.874 [main] INFO com.yahoo.storm.yarn.TestConfig - storm_home: lib/storm-0.9.0-wip21

11:33:55.876 [main] INFO com.yahoo.storm.yarn.TestIntegration - Will be using storm found on PATH at lib/storm-0.9.0-wip21

11:33:55.929 [main] INFO com.yahoo.storm.yarn.TestIntegration - bin/storm-yarn launch ./conf/storm.yaml --stormZip lib/storm.zip --appname storm-on-yarn-test --output target/appId.txt

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/usr/local/storm-yarn/lib/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/client/api/YarnClient

at com.yahoo.storm.yarn.StormOnYarn.<init>(StormOnYarn.java:80)

at com.yahoo.storm.yarn.StormOnYarn.<init>(StormOnYarn.java:75)

at com.yahoo.storm.yarn.StormOnYarn.launchApplication(StormOnYarn.java:371)

at com.yahoo.storm.yarn.LaunchCommand.process(LaunchCommand.java:55)

at com.yahoo.storm.yarn.Client.execute(Client.java:139)

at com.yahoo.storm.yarn.Client.main(Client.java:145)

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.client.api.YarnClient

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 6 more

11:34:12.474 [main] ERROR com.yahoo.storm.yarn.TestIntegration - setup failure

java.io.FileNotFoundException: target/appId.txt (No such file or directory)

at java.io.FileInputStream.open(Native Method) ~[na:1.7.0_45]

at java.io.FileInputStream.<init>(FileInputStream.java:146) ~[na:1.7.0_45]

at java.io.FileInputStream.<init>(FileInputStream.java:101) ~[na:1.7.0_45]

at java.io.FileReader.<init>(FileReader.java:58) ~[na:1.7.0_45]

at com.yahoo.storm.yarn.TestIntegration.setup(TestIntegration.java:101) ~[test-classes/:na]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_45]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_45]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_45]

at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_45]

at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) [junit-4.10.jar:na]

at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) [junit-4.10.jar:na]

at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) [junit-4.10.jar:na]

at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) [junit-4.10.jar:na]

at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) [junit-4.10.jar:na]

at org.junit.runners.ParentRunner.run(ParentRunner.java:300) [junit-4.10.jar:na]

at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53) [surefire-junit4-2.10.jar:2.10]

at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123) [surefire-junit4-2.10.jar:2.10]

at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104) [surefire-junit4-2.10.jar:2.10]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_45]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_45]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_45]

at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_45]

at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164) [surefire-api-2.10.jar:2.10]

at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110) [surefire-booter-2.10.jar:2.10]

at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175) [surefire-booter-2.10.jar:2.10]

at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107) [surefire-booter-2.10.jar:2.10]

at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68) [surefire-booter-2.10.jar:2.10]

11:34:12.475 [main] INFO com.yahoo.storm.yarn.TestIntegration - java.lang.NullPointerException

11:34:12.475 [main] INFO c.yahoo.storm.yarn.EmbeddedZKServer - shutdown embedded zookeeper server with port 60000

11:34:12.479 [main] INFO com.yahoo.storm.yarn.TestIntegration - shutdown MiniYarn cluster

11:34:12.488 [main] INFO org.mortbay.log - Stopped SelectChannelConnector@datanode10:0

11:34:12.590 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 60968

11:34:12.594 [IPC Server listener on 60968] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 60968

11:34:12.594 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.598 [main] INFO o.a.h.y.s.n.c.m.ContainersMonitorImpl - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread

11:34:12.599 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 60239

11:34:12.600 [IPC Server listener on 60239] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 60239

11:34:12.600 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.601 [Public Localizer] INFO o.a.h.y.s.n.c.l.ResourceLocalizationService - Public cache exiting

11:34:12.609 [main] INFO org.mortbay.log - Stopped SelectChannelConnector@datanode10:0

11:34:12.613 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 49347

11:34:12.615 [IPC Server listener on 49347] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 49347

11:34:12.616 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.619 [main] INFO o.a.h.y.s.n.c.m.ContainersMonitorImpl - Neither virutal-memory nor physical-memory monitoring is needed. Not running the monitor-thread

11:34:12.620 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 54851

11:34:12.621 [IPC Server listener on 54851] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 54851

11:34:12.623 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.623 [Public Localizer] INFO o.a.h.y.s.n.c.l.ResourceLocalizationService - Public cache exiting

11:34:12.628 [main] INFO org.mortbay.log - Stopped SelectChannelConnector@datanode10:0

11:34:12.735 [Thread[Thread-21,5,main]] ERROR o.a.h.s.t.d.AbstractDelegationTokenSecretManager - InterruptedExcpetion recieved for ExpiredTokenRemover thread java.lang.InterruptedException: sleep interrupted

11:34:12.737 [main] INFO o.a.h.m.impl.MetricsSystemImpl - Stopping NodeManager metrics system...

11:34:12.743 [main] INFO o.a.h.m.impl.MetricsSystemImpl - NodeManager metrics system stopped.

11:34:12.744 [main] INFO o.a.h.m.impl.MetricsSystemImpl - NodeManager metrics system shutdown complete.

11:34:12.745 [ApplicationMaster Launcher] WARN o.a.h.y.s.r.a.ApplicationMasterLauncher - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning.

11:34:12.746 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 50842

11:34:12.747 [IPC Server listener on 50842] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 50842

11:34:12.747 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 34056

11:34:12.747 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.791 [IPC Server listener on 34056] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 34056

11:34:12.791 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.792 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 55279

11:34:12.798 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.798 [IPC Server listener on 55279] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 55279

11:34:12.798 [main] INFO org.apache.hadoop.ipc.Server - Stopping server on 48967

11:34:12.805 [Ping Checker] INFO o.a.h.y.u.AbstractLivelinessMonitor - NMLivelinessMonitor thread interrupted

11:34:12.806 [ResourceManager Event Processor] ERROR o.a.h.y.s.r.ResourceManager - Returning, interrupted : java.lang.InterruptedException

11:34:12.806 [IPC Server Responder] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server Responder

11:34:12.805 [IPC Server listener on 48967] INFO org.apache.hadoop.ipc.Server - Stopping IPC Server listener on 48967

11:34:12.815 [Ping Checker] INFO o.a.h.y.u.AbstractLivelinessMonitor - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted

11:34:12.814 [Ping Checker] INFO o.a.h.y.u.AbstractLivelinessMonitor - AMLivelinessMonitor thread interrupted

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 27.402 sec <<< FAILURE!

Results :

Failed tests: com.yahoo.storm.yarn.TestIntegration: expected:<null> but was:<java.io.FileNotFoundException: target/appId.txt (No such file or directory)>

Tests run: 3, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 4:44.183s

[INFO] Finished at: Thu Oct 17 11:34:13 CST 2013

[INFO] Final Memory: 14M/466M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.10:test (default-test) on project storm-yarn: There are test failures.

[ERROR]

[ERROR] Please refer to /usr/local/storm-yarn/target/surefire-reports for the individual test results.

[ERROR] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Andy Feng

unread,

Oct 17, 2013, 12:28:36 AM10/17/13

to zhangfenghua, storm...@googlegroups.com

Which branch did you checkout the code? Master or 0.23?

Andy Feng

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "storm-yarn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-yarn+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

zhangfenghua

unread,

Oct 17, 2013, 12:58:22 AM10/17/13

to Andy Feng, storm...@googlegroups.com

hi Andy Feng：

I checkouted the Master branch。

zhangfenghua

unread,

Oct 17, 2013, 2:57:34 AM10/17/13

to Andy Feng, storm...@googlegroups.com

Now I use https://github.com/yahoo/storm-yarn/tree/hadoop-0.23 the branch,and try to execute the command:

mvn -f pom-cdh43.xml package //if you are using Hadoop 2.0.0-cdh4.3.x from cloudera

though our hadoop cluster version is hadoop-2.0.0-cdh4.4.0,but it builded success.

but the output contains summery ERROR message like

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/storm-yarn/lib/storm-0.9.0-wip21/lib/logback-classic-1.0.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

14:52:31.152 [mainhread] org.apache.hadoop.yarn.service.AbstractService.init(AbstractService.java:81)evel org.apache.hadoop.yarn.service.AbstractService.init(AbstractService.java:81)ogger{36} - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.sg
14:52:31.617 [mainhread] org.apache.hadoop.util.NativeCodeLoader.<clinit>(NativeCodeLoader.java:62)evel org.apache.hadoop.util.NativeCodeLoader.<clinit>(NativeCodeLoader.java:62)ogger{36} - Unable to load native-hadoop library for your platform... using builtin-java classes where applicablesg
14:52:31.960 [mainhread] org.apache.hadoop.yarn.service.AbstractService.start(AbstractService.java:94)evel org.apache.hadoop.yarn.service.AbstractService.start(AbstractService.java:94)ogger{36} - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.sg
14:52:42.160 [mainhread] com.yahoo.storm.yarn.StormOnYarn.getClient(StormOnYarn.java:122)evel com.yahoo.storm.yarn.StormOnYarn.getClient(StormOnYarn.java:122)ogger{36} - No host/port returned for Application Master application_1381992618387_0001sg
14:52:42.265 [mainhread] org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:113)evel org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:113)ogger{36} - Service:org.apache.hadoop.yarn.client.YarnClientImpl is stopped.sg
Exception in thread "main" java.lang.NullPointerException
        at com.yahoo.storm.yarn.StormMasterCommand.process(StormMasterCommand.java:119)

        at com.yahoo.storm.yarn.Client.execute(Client.java:139)
        at com.yahoo.storm.yarn.Client.main(Client.java:145)

NullPointerException

zhangfenghua

From: Andy Feng

Date: 2013-10-17 12:28

To: zhangfenghua

CC: storm...@googlegroups.com

Subject: Re: mvn package fails with storm-yarn 1.0-alpha

Reply all

Reply to author

Forward

0 new messages