I am new to Storm just trying to understand and run the storm-starter
example WordCount on local host. I am getting following error..
Running: export STORM_JAR=storm-starter-0.0.1-SNAPSHOT.jar; java -
client -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -
cp /home/manish/storm-dev/storm-0.6.0/storm-0.6.0.jar:/home/manish/
storm-dev/storm-0.6.0/lib/hiccup-0.3.6.jar:/home/manish/storm-dev/
storm-0.6.0/lib/jetty-util-6.1.26.jar:/home/manish/storm-dev/
storm-0.6.0/lib/commons-fileupload-1.2.1.jar:/home/manish/storm-dev/
storm-0.6.0/lib/clj-time-0.3.0.jar:/home/manish/storm-dev/storm-0.6.0/
lib/clout-0.4.1.jar:/home/manish/storm-dev/storm-0.6.0/lib/
clojure-1.2.0.jar:/home/manish/storm-dev/storm-0.6.0/lib/slf4j-
api-1.5.8.jar:/home/manish/storm-dev/storm-0.6.0/lib/ring-jetty-
adapter-0.3.11.jar:/home/manish/storm-dev/storm-0.6.0/lib/
tools.macro-0.1.0.jar:/home/manish/storm-dev/storm-0.6.0/lib/
jetty-6.1.26.jar:/home/manish/storm-dev/storm-0.6.0/lib/
compojure-0.6.4.jar:/home/manish/storm-dev/storm-0.6.0/lib/
reflectasm-1.01.jar:/home/manish/storm-dev/storm-0.6.0/lib/servlet-
api-2.5-20081211.jar:/home/manish/storm-dev/storm-0.6.0/lib/
kryo-1.04.jar:/home/manish/storm-dev/storm-0.6.0/lib/
core.incubator-0.1.0.jar:/home/manish/storm-dev/storm-0.6.0/lib/
log4j-1.2.16.jar:/home/manish/storm-dev/storm-0.6.0/lib/
jline-0.9.94.jar:/home/manish/storm-dev/storm-0.6.0/lib/commons-
codec-1.3.jar:/home/manish/storm-dev/storm-0.6.0/lib/commons-
lang-2.5.jar:/home/manish/storm-dev/storm-0.6.0/lib/commons-
logging-1.1.1.jar:/home/manish/storm-dev/storm-0.6.0/lib/
jvyaml-1.0.0.jar:/home/manish/storm-dev/storm-0.6.0/lib/ring-
servlet-0.3.11.jar:/home/manish/storm-dev/storm-0.6.0/lib/
jzmq-2.1.0.jar:/home/manish/storm-dev/storm-0.6.0/lib/asm-3.2.jar:/
home/manish/storm-dev/storm-0.6.0/lib/slf4j-log4j12-1.5.8.jar:/home/
manish/storm-dev/storm-0.6.0/lib/clojure-contrib-1.2.0.jar:/home/
manish/storm-dev/storm-0.6.0/lib/carbonite-1.0.0.jar:/home/manish/
storm-dev/storm-0.6.0/lib/minlog-1.2.jar:/home/manish/storm-dev/
storm-0.6.0/lib/httpcore-4.0.1.jar:/home/manish/storm-dev/storm-0.6.0/
lib/commons-io-1.4.jar:/home/manish/storm-dev/storm-0.6.0/lib/ring-
core-0.3.10.jar:/home/manish/storm-dev/storm-0.6.0/lib/
zookeeper-3.3.2.jar:/home/manish/storm-dev/storm-0.6.0/lib/joda-
time-1.6.jar:/home/manish/storm-dev/storm-0.6.0/lib/json-
simple-1.1.jar:/home/manish/storm-dev/storm-0.6.0/lib/servlet-
api-2.5.jar:/home/manish/storm-dev/storm-0.6.0/lib/
httpclient-4.0.1.jar:/home/manish/storm-dev/storm-0.6.0/lib/commons-
exec-1.1.jar:/home/manish/storm-dev/storm-0.6.0/lib/
libthrift7-0.7.0.jar:/home/manish/storm-dev/storm-0.6.0/lib/
junit-3.8.1.jar:storm-starter-0.0.1-SNAPSHOT.jar:/home/manish/.storm:/
home/manish/storm-dev/storm-0.6.0/bin storm.starter.WordCountTopology
0 [main] INFO backtype.storm.zookeeper - Starting inprocess
zookeeper at port 2181 and dir /tmp/
23af594e-862f-46ec-9b89-52bab68ed742
157 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper
state update: :connected:none
215 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper
state update: :connected:none
221 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper
state update: :connected:none
236 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper
state update: :connected:none
316 [main] INFO backtype.storm.daemon.supervisor - Starting
supervisor with id 69714b4d-6977-46a1-85de-35e68ef0ea50
329 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper
state update: :connected:none
344 [main] INFO backtype.storm.daemon.supervisor - Starting
supervisor with id 9434b025-30f4-4cc2-8b87-7e894d47283c
403 [main] INFO backtype.storm.daemon.nimbus - Received topology
submission for word-count with conf {"topology.kryo.register" nil,
"storm.id" "word-count-1-1324371220", "topology.max.task.parallelism"
3, "topology.debug" true}
548 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:1 timed out
549 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:2 timed out
550 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:3 timed out
551 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:4 timed out
552 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:5 timed out
553 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:6 timed out
554 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:7 timed out
555 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:8 timed out
556 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:9 timed out
558 [main] INFO backtype.storm.daemon.nimbus - Task word-
count-1-1324371220:10 timed out
561 [main] INFO backtype.storm.daemon.nimbus - Reassigning word-
count-1-1324371220 to 1 slots
674 [main] INFO backtype.storm.daemon.nimbus - Reassign ids: [1 2 3
4 5 6 7 8 9 10]
691 [main] INFO backtype.storm.daemon.nimbus - Available slots:
(["9434b025-30f4-4cc2-8b87-7e894d47283c" 4]
["9434b025-30f4-4cc2-8b87-7e894d47283c" 5]
["9434b025-30f4-4cc2-8b87-7e894d47283c" 6]
["69714b4d-6977-46a1-85de-35e68ef0ea50" 1]
["69714b4d-6977-46a1-85de-35e68ef0ea50" 2]
["69714b4d-6977-46a1-85de-35e68ef0ea50" 3])
730 [main] INFO backtype.storm.daemon.nimbus - Setting new
assignment for storm id word-count-1-1324371220:
#:backtype.storm.daemon.common.Assignment{:master-code-dir "/tmp/
52f572fd-7260-42dd-9135-54d146f567a9/nimbus/stormdist/word-
count-1-1324371220", :node->host
{"9434b025-30f4-4cc2-8b87-7e894d47283c" "Iris"}, :task->node+port {1
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 2
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 3
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 4
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 5
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 6
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 7
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 8
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 9
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4], 10
["9434b025-30f4-4cc2-8b87-7e894d47283c" 4]}, :task->start-time-secs {1
1324371221, 2 1324371221, 3 1324371221, 4 1324371221, 5 1324371221, 6
1324371221, 7 1324371221, 8 1324371221, 9 1324371221, 10 1324371221}}
746 [main] INFO backtype.storm.daemon.nimbus - Activating word-
count: word-count-1-1324371220
762 [Thread-11] INFO backtype.storm.daemon.supervisor - Downloading
code for storm id word-count-1-1324371220 from /tmp/
52f572fd-7260-42dd-9135-54d146f567a9/nimbus/stormdist/word-
count-1-1324371220
764 [Thread-11] INFO backtype.storm.daemon.supervisor - Copying
resources at jar:file:/home/manish/storm-starter/nathanmarz-storm-
starter-4186391/target/storm-starter-0.0.1-SNAPSHOT.jar!/resources to /
tmp/5e9aa013-a694-4589-af56-c21c02783be5/supervisor/stormdist/word-
count-1-1324371220/resources
765 [Thread-11] ERROR backtype.storm.event - Error when processing
event backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668@7593c366
java.io.FileNotFoundException: Source 'file:/home/manish/storm-starter/
nathanmarz-storm-starter-4186391/target/storm-starter-0.0.1-
SNAPSHOT.jar!/resources' does not exist
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:770)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:745)
at backtype.storm.daemon.supervisor
$fn__3785.invoke(supervisor.clj:394)
at clojure.lang.MultiFn.invoke(MultiFn.java:171)
at backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668.invoke(supervisor.clj:
263)
at backtype.storm.event$event_manager
$fn__2068$fn__2069.invoke(event.clj:25)
at backtype.storm.event$event_manager
$fn__2068.invoke(event.clj:22)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:636)
769 [Thread-6] INFO backtype.storm.daemon.supervisor - Downloading
code for storm id word-count-1-1324371220 from /tmp/
52f572fd-7260-42dd-9135-54d146f567a9/nimbus/stormdist/word-
count-1-1324371220
771 [Thread-6] INFO backtype.storm.daemon.supervisor - Copying
resources at jar:file:/home/manish/storm-starter/nathanmarz-storm-
starter-4186391/target/storm-starter-0.0.1-SNAPSHOT.jar!/resources to /
tmp/09c8b6c3-b5d1-40c8-8861-224b86c043d4/supervisor/stormdist/word-
count-1-1324371220/resources
771 [Thread-6] ERROR backtype.storm.event - Error when processing
event backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668@65957a34
java.io.FileNotFoundException: Source 'file:/home/manish/storm-starter/
nathanmarz-storm-starter-4186391/target/storm-starter-0.0.1-
SNAPSHOT.jar!/resources' does not exist
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:770)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:745)
at backtype.storm.daemon.supervisor
$fn__3785.invoke(supervisor.clj:394)
at clojure.lang.MultiFn.invoke(MultiFn.java:171)
at backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668.invoke(supervisor.clj:
263)
at backtype.storm.event$event_manager
$fn__2068$fn__2069.invoke(event.clj:25)
at backtype.storm.event$event_manager
$fn__2068.invoke(event.clj:22)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:636)
817 [Thread-6] INFO backtype.storm.util - Halting process: ("Error
when processing an event")
817 [Thread-11] INFO backtype.storm.util - Halting process: ("Error
when processing an event")
Not sure what is wrong please help me in debugging this.
Thanks,
Manish
I am using Linux
Linux Iris 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200
x86_64 x86_64 x86_64 GNU/Linux.
I am using your storm-starter example I have compiled in using maven
and it has created two jars storm-starter-0.0.1-SNAPSHOT.jar and storm-
starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar in target dirtectory.
I have installed storm 0.6.0 and added bin in path and then ran the
following command
/storm-starter/nathanmarz-storm-starter-4186391/target> storm jar
storm-starter-0.0.1-SNAPSHOT.jar storm.starter.Word.Count.Topology
Then it gave the error.
same error is coming when I am running
/storm-starter/nathanmarz-storm-starter-4186391/target> storm jar
storm-starter-0.0.1-SNAPSHOT.jar storm.starter.ExclamationTopology
"906 [Thread-11] ERROR backtype.storm.event - Error when processing
event backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668@68c95daa
java.io.FileNotFoundException: Source 'file:/home/manish/storm-starter/
nathanmarz-storm-starter-4186391/target/storm-starter-0.0.1-
SNAPSHOT.jar!/resources' does not exist
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:770)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:745)
at backtype.storm.daemon.supervisor
$fn__3785.invoke(supervisor.clj:394)
at clojure.lang.MultiFn.invoke(MultiFn.java:171)
at backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668.invoke(supervisor.clj:
263)
at backtype.storm.event$event_manager
$fn__2068$fn__2069.invoke(event.clj:25)
at backtype.storm.event$event_manager
$fn__2068.invoke(event.clj:22)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:636)
915 [Thread-6] INFO backtype.storm.daemon.supervisor - Downloading
code for storm id test-1-1324374200 from /tmp/649e2c3f-c2d6-4c2a-
b0c5-96957195e503/nimbus/stormdist/test-1-1324374200
916 [Thread-6] INFO backtype.storm.daemon.supervisor - Copying
resources at jar:file:/home/manish/storm-starter/nathanmarz-storm-
starter-4186391/target/storm-starter-0.0.1-SNAPSHOT.jar!/resources to /
tmp/f204f794-9e0f-45e9-a2b2-f3dc93b371dd/supervisor/stormdist/
test-1-1324374200/resources
916 [Thread-6] ERROR backtype.storm.event - Error when processing
event backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668@711b50a1
java.io.FileNotFoundException: Source 'file:/home/manish/storm-starter/
nathanmarz-storm-starter-4186391/target/storm-starter-0.0.1-
SNAPSHOT.jar!/resources' does not exist
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:866)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:770)
at
org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:745)
at backtype.storm.daemon.supervisor
$fn__3785.invoke(supervisor.clj:394)
at clojure.lang.MultiFn.invoke(MultiFn.java:171)
at backtype.storm.daemon.supervisor
$fn__3567$exec_fn__945__auto____3568$this__3668.invoke(supervisor.clj:
263)
at backtype.storm.event$event_manager
$fn__2068$fn__2069.invoke(event.clj:25)
at backtype.storm.event$event_manager
$fn__2068.invoke(event.clj:22)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:636)
966 [Thread-6] INFO backtype.storm.util - Halting process: ("Error
when processing an event")
967 [Thread-11] INFO backtype.storm.util - Halting process: ("Error
when processing an event")
"
Running with Lein is giving some other error like
manish@Iris:~/storm-starter/nathanmarz-storm-starter-4186391> lein
deps
Exception in thread "main" java.lang.RuntimeException:
java.io.FileNotFoundException: Could not locate leiningen/core/
user__init.class or leiningen/core/user.clj on classpath:
at clojure.lang.Util.runtimeException(Util.java:165)
at clojure.lang.Compiler.eval(Compiler.java:6476)
at clojure.lang.Compiler.eval(Compiler.java:6455)
at clojure.lang.Compiler.load(Compiler.java:6902)
at clojure.lang.RT.loadResourceScript(RT.java:357)
at clojure.lang.RT.loadResourceScript(RT.java:348)
at clojure.lang.RT.load(RT.java:427)
at clojure.lang.RT.load(RT.java:398)
at clojure.core$load$fn__4610.invoke(core.clj:5386)
at clojure.core$load.doInvoke(core.clj:5385)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5200)
at clojure.core$load_lib.doInvoke(core.clj:5237)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:602)
at clojure.core$load_libs.doInvoke(core.clj:5271)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:602)
at clojure.core$require.doInvoke(core.clj:5352)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.main$main_opt.invoke(main.clj:323)
at clojure.main$main.doInvoke(main.clj:426)
at clojure.lang.RestFn.invoke(RestFn.java:436)
at clojure.lang.Var.invoke(Var.java:409)
at clojure.lang.AFn.applyToHelper(AFn.java:167)
at clojure.lang.Var.applyTo(Var.java:518)
at clojure.main.main(main.java:37)
Caused by: java.io.FileNotFoundException: Could not locate leiningen/
core/user__init.class or leiningen/core/user.clj on classpath:
at clojure.lang.RT.load(RT.java:430)
at clojure.lang.RT.load(RT.java:398)
at clojure.core$load$fn__4610.invoke(core.clj:5386)
at clojure.core$load.doInvoke(core.clj:5385)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5200)
at clojure.core$load_lib.doInvoke(core.clj:5237)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:602)
at clojure.core$load_libs.doInvoke(core.clj:5271)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:602)
at clojure.core$require.doInvoke(core.clj:5352)
at clojure.lang.RestFn.invoke(RestFn.java:457)
at leiningen.main
$eval3$loading__4505__auto____4.invoke(main.clj:1)
at leiningen.main$eval3.invoke(main.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6465)
... 25 more
I am novice in this. Please help.
Regards,
Manish
> ...
>
> read more »
I have installed it but created the lein script again and ran lein
getting this error now...
manish@Iris:~/technomancy-leiningen-ac3e84e/bin> lein
Downloading Leiningen now...
% Total % Received % Xferd Average Speed Time Time
Time Current
Dload Upload Total Spent
Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02
--:--:-- 0
curl: (22) The requested URL returned error: 404
Failed to download https://github.com/downloads/technomancy/leiningen/leiningen--standalone.jar
Exception in thread "main" java.lang.NoClassDefFoundError: clojure/
main
Caused by: java.lang.ClassNotFoundException: clojure.main
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:
294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:
336)
Could not find the main class: clojure.main. Program will exit.
> ...
>
> read more »