zookeeper connection timeout

[Aditya]

Hi,

I am trying to create topology, but neither the spouts nor the bolts
are initializing.
I have placed loggers and sysouts in the open and next tuple methods
to verify the same.

The nimbus and supervisor logs seem to be showing no error. The
behaviour seems normal.
But on zookeeper i am getting the following log messages continuously.

I think the session timeouts on zookeeper are the reason for this.
The configuration is default,
in zookeeper -> tick time to 2000
storm.zookeeper.session.timeout: 20000

--------------------------------------------------------
2011-10-13 17:38:35,722 - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1435] - Closed socket connection
for client /192.168.0.172:36555 which had sessionid 0x132fcec7d5e01ea
2011-10-13 17:38:35,760 - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket
connection from /192.168.0.172:36556
2011-10-13 17:38:35,761 - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to
establish new session at /192.168.0.172:36556
2011-10-13 17:38:35,762 - INFO [SyncThread:0:NIOServerCnxn@1580] -
Established session 0x132fcec7d5e01eb with negotiated timeout 20000
for client /192.168.0.172:36556
2011-10-13 17:38:37,702 - WARN [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@634] - EndOfStreamException: Unable
to read additional data from client sessionid 0x132fcec7d5e01eb,
likely client has closed socket
2011-10-13 17:38:37,703 - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1435] - Closed socket connection
for client /192.168.0.172:36556 which had sessionid 0x132fcec7d5e01eb
2011-10-13 17:38:38,742 - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket
connection from /192.168.0.172:36557
2011-10-13 17:38:38,745 - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to
establish new session at /192.168.0.172:36557
2011-10-13 17:38:38,747 - INFO [SyncThread:0:NIOServerCnxn@1580] -
Established session 0x132fcec7d5e01ec with negotiated timeout 10000
for client /192.168.0.172:36557
2011-10-13 17:38:38,760 - INFO
[ProcessThread:-1:PrepRequestProcessor@387] - Processed session
termination for sessionid: 0x132fcec7d5e01ec
2011-10-13 17:38:38,761 - INFO [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1435] - Closed socket connection
for client /192.168.0.172:36557 which had sessionid 0x132fcec7d5e01ec
-----------------------------------------------


In the supervisor logs dir i have many logs which have worker logs
[root@vm2 logs]# ll
total 3308
-rw-r--r-- 1 root root 416787 Oct 13 17:39 supervisor.log
-rw-r--r-- 1 root root 1021127 Oct 13 17:38 worker-6710.log
-rw-r--r-- 1 root root 985749 Oct 13 17:39 worker-6711.log
-rw-r--r-- 1 root root 938137 Oct 13 17:39 worker-6712.log

All of the worker logs end with this message
------------------------------
at backtype.storm.daemon.worker.main(Unknown Source)
2011-10-13 17:39:12 util [INFO] Halting process: ("Error on
initialization")
-----------------------------

I think this is same as the error already posted here

https://gist.github.com/1232610


Also, On the console from where i ran the supervisor, i get these
messages repeatedly

------------------------------------
caution: filename not matched: resources/**
kill 12072: No such process
kill 12095: No such process
kill 12116: No such process
kill 12138: No such process
caution: filename not matched: resources/**
-----------------------------------

1. Why are these errors coming?
2. How do i resolve it?
3. Is there any configuration change i can make to fix this?

The topology i'm running here was running fine on a local cluster, so
i'm assuming the communication/zookeeper is at fault.

Thanks,
Aditya

[Aditya]

This log sequence shows the proper 20 second timeout,
From connection creation to expiry.
There is a topology that should be run at this time, but it is not
being run, every connection is timing out...

2011-10-13 18:07:10,793 - INFO [NIOServerCxn.Factory:

0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket

connection from /192.168.0.172:59769
2011-10-13 18:07:10,794 - INFO [NIOServerCxn.Factory:

0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to

establish new session at /192.168.0.172:59769
2011-10-13 18:07:10,796 - INFO [SyncThread:0:NIOServerCnxn@1580] -
Established session 0x132fd44d46c000d with negotiated timeout 20000
for client /192.168.0.172:59769
2011-10-13 18:07:12,569 - WARN [NIOServerCxn.Factory:

0.0.0.0/0.0.0.0:2181:NIOServerCnxn@634] - EndOfStreamException: Unable

to read additional data from client sessionid 0x132fd44d46c000d,

likely client has closed socket

2011-10-13 18:07:12,569 - INFO [NIOServerCxn.Factory:

0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1435] - Closed socket connection

for client /192.168.0.172:59769 which had sessionid 0x132fd44d46c000d
2011-10-13 18:07:32,006 - INFO [SessionTracker:ZooKeeperServer@314] -
Expiring session 0x132fd44d46c000d, timeout of 20000ms exceeded

[nathanmarz]

I've never seen this error before. Very strange. Can you provide a
sample worker log from a failed worker? Also, can you provide more
information about how your cluster is setup? Information like the
following would be helpful:

1. What OS you're running on
2. How much CPU and memory your machines have
3. The storm.yaml files on the supervisor machines


-Nathan

[Aditya]

Hi Nathan,

The details you wanted
Setup-
1 virtual machine on which all the components are installed and run
(zookeeper, nimbus, 2 supervisors on different worker ports, ui )
I am running a network version the word count topology. The server is
otherwise free, not much load on it due to anything else.

Machine details-> This is a virtual machine running on the server.
This is the output of the commands on the vm
---------
OS -
[root@vm2 ~]# uname -a
Linux asdfg 2.6.18-128.el5xen #1 SMP Wed Jan 21 11:12:42 EST 2009
x86_64 x86_64 x86_64 GNU/Linux

Ram - 2GB

[root@vm2 ~]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 4
model name : AMD Phenom(tm) II X2 550 Processor
stepping : 2
cpu MHz : 3114.162
cache size : 512 KB
-----------------

Storm.yaml on supervisor

----------------

storm.zookeeper.servers:
- "192.168.0.172"
nimbus.host: "192.168.0.172"
storm.local.dir: "/tmp/storm"
supervisor.slots.ports:
- 6710
- 6711
- 6712
- 6713
- 6714

---------------


I think i've figured out why the error is coming. This message was
repeatedly coming in the worker logs.

2011-10-13 18:29:27 worker [ERROR] Error on initialization of server
mk-worker
java.lang.UnsatisfiedLinkError: no jzmq in java.library.path
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
....
....
at backtype.storm.daemon.worker.main(Unknown Source)
2011-10-13 18:29:27 util [INFO] Halting process: ("Error on
initialization")


[root@vm2 storm-0.5.3]# grep UnsatisfiedLinkError logs/worker-6700.log
| wc -l
20
[root@vm2 storm-0.5.3]# grep "Halting process" logs/worker-6700.log |
wc -l
20

I find this odd because i have setup JZMQ following the steps given
here
https://github.com/nathanmarz/jzmq
configure and make worked fine.

Then i ran a make install
Ran a make install after that and it copied zmq.so to /usr/ocal/lib

Now i'm getting a different error

2011-10-17 11:48:47 worker [ERROR] Error on initialization of server
mk-worker
java.lang.UnsatisfiedLinkError: /usr/local/lib/libjzmq.so.0.0.0:
libzmq.so.1: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
.....
....

This error is very odd as

[root@vm2 ~]# ll /usr/local/lib
total 7176
-rw-r--r-- 1 root root 411358 Oct 17 11:42 libjzmq.a
-rwxr-xr-x 1 root root 798 Oct 17 11:42 libjzmq.la
lrwxrwxrwx 1 root root 16 Oct 17 11:42 libjzmq.so -> libjzmq.so.
0.0.0
lrwxrwxrwx 1 root root 16 Oct 17 11:42 libjzmq.so.0 -> libjzmq.so.
0.0.0
-rwxr-xr-x 1 root root 188084 Oct 17 11:42 libjzmq.so.0.0.0
-rw-r--r-- 1 root root 4737500 Oct 4 15:49 libzmq.a
-rwxr-xr-x 1 root root 958 Oct 4 15:49 libzmq.la
lrwxrwxrwx 1 root root 15 Oct 4 15:49 libzmq.so -> libzmq.so.
1.0.0
lrwxrwxrwx 1 root root 15 Oct 4 15:49 libzmq.so.1 -> libzmq.so.
1.0.0
-rwxr-xr-x 1 root root 1969169 Oct 4 15:49 libzmq.so.1.0.0
drwxr-xr-x 2 root root 4096 Oct 4 15:49 pkgconfig

The full logs are here -
(The same log sequence is repeating 20 times for each worker)
===================================
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
GMT
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:host.name=192.168.0.172
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:java.version=1.6.0_17
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:java.vendor=Sun Microsystems Inc.
2011-10-17 11:48:46 ZooKeeper [INFO] Client environment:java.home=/usr/
java/jre1.6.0_17
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:java.class.path=/root/aditya/storm/storm-0.5.3/
stormClient6.jar:/root/aditya/storm/storm-0.5.3/stormClient2.jar:/root/
adity
a/storm/storm-0.5.3/stormClient4.jar:/root/aditya/storm/storm-0.5.3/
stormClient5.jar:/root/aditya/storm/storm-0.5.3/storm-0.5.3.jar:/root/
aditya/storm/storm-0.5.3/stormClient1.jar
:/root/aditya/storm/storm-0.5.3/stormtest.jar:/root/aditya/storm/
storm-0.5.3/stormClient3.jar:/root/aditya/storm/storm-0.5.3/lib/
servlet-api-2.5-20081211.jar:/root/aditya/storm/st
orm-0.5.3/lib/commons-net-1.4.1.jar:/root/aditya/storm/storm-0.5.3/lib/
joda-time-1.6.jar:/root/aditya/storm/storm-0.5.3/lib/clj-
time-0.3.0.jar:/root/aditya/storm/storm-0.5.3/lib/r
ing-servlet-0.3.11.jar:/root/aditya/storm/storm-0.5.3/lib/commons-
httpclient-3.0.1.jar:/root/aditya/storm/storm-0.5.3/lib/
jvyaml-1.0.0.jar:/root/aditya/storm/storm-0.5.3/lib/commo
ns-el-1.0.jar:/root/aditya/storm/storm-0.5.3/lib/ring-core-0.3.10.jar:/
root/aditya/storm/storm-0.5.3/lib/zookeeper-3.3.2.jar:/root/aditya/
storm/storm-0.5.3/lib/commons-cli-1.2.jar
:/root/aditya/storm/storm-0.5.3/lib/json-simple-1.1.jar:/root/aditya/
storm/storm-0.5.3/lib/tools.macro-0.1.0.jar:/root/aditya/storm/
storm-0.5.3/lib/ring-jetty-adapter-0.3.11.jar:/
root/aditya/storm/storm-0.5.3/lib/jzmq-2.1.0.jar:/root/aditya/storm/
storm-0.5.3/lib/slf4j-log4j12-1.5.10.jar:/root/aditya/storm/
storm-0.5.3/lib/hiccup-0.3.6.jar:/root/aditya/storm
/storm-0.5.3/lib/commons-codec-1.3.jar:/root/aditya/storm/storm-0.5.3/
lib/thriftjava-1.0.0.jar:/root/aditya/storm/storm-0.5.3/lib/
jetty-6.1.26.jar:/root/aditya/storm/storm-0.5.3/l
ib/clojure-contrib-1.2.0.jar:/root/aditya/storm/storm-0.5.3/lib/
core.incubator-0.1.0.jar:/root/aditya/storm/storm-0.5.3/lib/
log4j-1.2.16.jar:/root/aditya/storm/storm-0.5.3/lib/com
mons-logging-1.0.4.jar:/root/aditya/storm/storm-0.5.3/lib/commons-
fileupload-1.2.1.jar:/root/aditya/storm/storm-0.5.3/lib/
junit-3.8.1.jar:/root/aditya/storm/storm-0.5.3/lib/common
s-logging-api-1.0.4.jar:/root/aditya/storm/storm-0.5.3/lib/commons-
exec-1.1.jar:/root/aditya/storm/storm-0.5.3/lib/slf4j-api-1.5.10.jar:/
root/aditya/storm/storm-0.5.3/lib/oro-2.0.
8.jar:/root/aditya/storm/storm-0.5.3/lib/jetty-util-6.1.26.jar:/root/
aditya/storm/storm-0.5.3/lib/commons-lang-2.4.jar:/root/aditya/storm/
storm-0.5.3/lib/commons-io-1.4.jar:/root/
aditya/storm/storm-0.5.3/lib/clout-0.4.1.jar:/root/aditya/storm/
storm-0.5.3/lib/compojure-0.6.4.jar:/root/aditya/storm/storm-0.5.3/lib/
servlet-api-2.5.jar:/root/aditya/storm/storm
-0.5.3/lib/jline-0.9.94.jar:/root/aditya/storm/storm-0.5.3/lib/
clojure-1.2.0.jar:/root/aditya/storm/storm-0.5.3/log4j:/root/aditya/
storm/storm-0.5.3/conf:/tmp/storm/supervisor/sto
rmdist/testwithdebug-1-1318832322/stormjar.jar
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:java.io.tmpdir=/tmp
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:java.compiler=<NA>
2011-10-17 11:48:46 ZooKeeper [INFO] Client environment:os.name=Linux
2011-10-17 11:48:46 ZooKeeper [INFO] Client environment:os.arch=amd64
2011-10-17 11:48:46 ZooKeeper [INFO] Client
environment:os.version=2.6.18-128.el5xen
2011-10-17 11:48:46 ZooKeeper [INFO] Client environment:user.name=root
2011-10-17 11:48:46 ZooKeeper [INFO] Client environment:user.home=/
root
2011-10-17 11:48:46 ZooKeeper [INFO] Client environment:user.dir=/root/
aditya/storm/storm-0.5.3/bin
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
GMT
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:host.name=192.168.0.172
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:java.version=1.6.0_17
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:java.vendor=Sun Microsystems Inc.
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:java.home=/usr/java/jre1.6.0_17
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:java.class.path=/root/aditya/storm/storm-0.5.3/
stormClient6.jar:/root/aditya/storm/storm-0.5.3/stormClient2.jar:/root
/aditya/storm/storm-0.5.3/stormClient4.jar:/root/aditya/storm/
storm-0.5.3/stormClient5.jar:/root/aditya/storm/storm-0.5.3/
storm-0.5.3.jar:/root/aditya/storm/storm-0.5.3/stormClien
t1.jar:/root/aditya/storm/storm-0.5.3/stormtest.jar:/root/aditya/storm/
storm-0.5.3/stormClient3.jar:/root/aditya/storm/storm-0.5.3/lib/
servlet-api-2.5-20081211.jar:/root/aditya/st
orm/storm-0.5.3/lib/commons-net-1.4.1.jar:/root/aditya/storm/
storm-0.5.3/lib/joda-time-1.6.jar:/root/aditya/storm/storm-0.5.3/lib/
clj-time-0.3.0.jar:/root/aditya/storm/storm-0.5.3
/lib/ring-servlet-0.3.11.jar:/root/aditya/storm/storm-0.5.3/lib/
commons-httpclient-3.0.1.jar:/root/aditya/storm/storm-0.5.3/lib/
jvyaml-1.0.0.jar:/root/aditya/storm/storm-0.5.3/lib
/commons-el-1.0.jar:/root/aditya/storm/storm-0.5.3/lib/ring-
core-0.3.10.jar:/root/aditya/storm/storm-0.5.3/lib/
zookeeper-3.3.2.jar:/root/aditya/storm/storm-0.5.3/lib/commons-cli-1
.2.jar:/root/aditya/storm/storm-0.5.3/lib/json-simple-1.1.jar:/root/
aditya/storm/storm-0.5.3/lib/tools.macro-0.1.0.jar:/root/aditya/storm/
storm-0.5.3/lib/ring-jetty-adapter-0.3.11
.jar:/root/aditya/storm/storm-0.5.3/lib/jzmq-2.1.0.jar:/root/aditya/
storm/storm-0.5.3/lib/slf4j-log4j12-1.5.10.jar:/root/aditya/storm/
storm-0.5.3/lib/hiccup-0.3.6.jar:/root/aditya
/storm/storm-0.5.3/lib/commons-codec-1.3.jar:/root/aditya/storm/
storm-0.5.3/lib/thriftjava-1.0.0.jar:/root/aditya/storm/storm-0.5.3/
lib/jetty-6.1.26.jar:/root/aditya/storm/storm-0
.5.3/lib/clojure-contrib-1.2.0.jar:/root/aditya/storm/storm-0.5.3/lib/
core.incubator-0.1.0.jar:/root/aditya/storm/storm-0.5.3/lib/
log4j-1.2.16.jar:/root/aditya/storm/storm-0.5.3/l
ib/commons-logging-1.0.4.jar:/root/aditya/storm/storm-0.5.3/lib/
commons-fileupload-1.2.1.jar:/root/aditya/storm/storm-0.5.3/lib/
junit-3.8.1.jar:/root/aditya/storm/storm-0.5.3/lib/
commons-logging-api-1.0.4.jar:/root/aditya/storm/storm-0.5.3/lib/
commons-exec-1.1.jar:/root/aditya/storm/storm-0.5.3/lib/slf4j-
api-1.5.10.jar:/root/aditya/storm/storm-0.5.3/lib/or
o-2.0.8.jar:/root/aditya/storm/storm-0.5.3/lib/jetty-util-6.1.26.jar:/
root/aditya/storm/storm-0.5.3/lib/commons-lang-2.4.jar:/root/aditya/
storm/storm-0.5.3/lib/commons-io-1.4.jar:
/root/aditya/storm/storm-0.5.3/lib/clout-0.4.1.jar:/root/aditya/storm/
storm-0.5.3/lib/compojure-0.6.4.jar:/root/aditya/storm/storm-0.5.3/lib/
servlet-api-2.5.jar:/root/aditya/storm
/storm-0.5.3/lib/jline-0.9.94.jar:/root/aditya/storm/storm-0.5.3/lib/
clojure-1.2.0.jar:/root/aditya/storm/storm-0.5.3/log4j:/root/aditya/
storm/storm-0.5.3/conf:/tmp/storm/supervis
or/stormdist/testwithdebug-1-1318832322/stormjar.jar
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:java.io.tmpdir=/tmp
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:java.compiler=<NA>
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:os.name=Linux
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:os.arch=amd64
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:os.version=2.6.18-128.el5xen
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:user.name=root
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:user.home=/root
2011-10-17 11:48:46 ZooKeeperServer [INFO] Server
environment:user.dir=/root/aditya/storm/storm-0.5.3/bin
2011-10-17 11:48:46 worker [INFO] Launching worker for
testwithdebug-1-1318832322 on 537e6569-5d7e-417d-
affb-35e75c600704:6700 with id 11e6cbf6-2457-4b33-989a-507f2dba963a
2011-10-17 11:48:46 ZooKeeper [INFO] Initiating client connection,
connectString=192.168.0.172:2181/ sessionTimeout=10000
watcher=backtype.storm.zookeeper$mk_client$reify__1233@2f
296d1d
2011-10-17 11:48:46 ClientCnxn [INFO] Opening socket connection to
server /192.168.0.172:2181
2011-10-17 11:48:46 ClientCnxn [INFO] Socket connection established to
192.168.0.172/192.168.0.172:2181, initiating session
2011-10-17 11:48:46 ClientCnxn [INFO] Session establishment complete
on server 192.168.0.172/192.168.0.172:2181, sessionid =
0x1331085b1c30004, negotiated timeout = 4000000
2011-10-17 11:48:46 zookeeper [INFO] Zookeeper state
update: :connected:none
2011-10-17 11:48:46 ZooKeeper [INFO] Session: 0x1331085b1c30004 closed
2011-10-17 11:48:46 ZooKeeper [INFO] Initiating client connection,
connectString=192.168.0.172:2181/storm sessionTimeout=20000
watcher=backtype.storm.zookeeper$mk_client$reify__12
33@530d0eae
2011-10-17 11:48:46 ClientCnxn [INFO] EventThread shut down
2011-10-17 11:48:46 ClientCnxn [INFO] Opening socket connection to
server /192.168.0.172:2181
2011-10-17 11:48:46 ClientCnxn [INFO] Socket connection established to
192.168.0.172/192.168.0.172:2181, initiating session
2011-10-17 11:48:46 ClientCnxn [INFO] Session establishment complete
on server 192.168.0.172/192.168.0.172:2181, sessionid =
0x1331085b1c30005, negotiated timeout = 4000000
2011-10-17 11:48:47 worker [ERROR] Error on initialization of server
mk-worker
java.lang.UnsatisfiedLinkError: /usr/local/lib/libjzmq.so.0.0.0:
libzmq.so.1: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at org.zeromq.ZMQ.<clinit>(ZMQ.java:34)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at zilch.mq__init.load(Unknown Source)
at zilch.mq__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at clojure.lang.RT.loadClassForName(RT.java:1578)
at clojure.lang.RT.load(RT.java:399)
at clojure.lang.RT.load(RT.java:381)
at clojure.core$load$fn__4511.invoke(core.clj:4905)
at clojure.core$load.doInvoke(core.clj:4904)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at clojure.core$load_one.invoke(core.clj:4729)
at clojure.core$load_lib.doInvoke(core.clj:4766)
at clojure.lang.RestFn.applyTo(RestFn.java:143)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$load_libs.doInvoke(core.clj:4800)
at clojure.lang.RestFn.applyTo(RestFn.java:138)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$require.doInvoke(core.clj:4869)
at clojure.lang.RestFn.invoke(RestFn.java:422)
at backtype.storm.messaging.zmq
$loading__4410__auto__.invoke(zmq.clj:1)
at backtype.storm.messaging.zmq__init.load(Unknown Source)
at backtype.storm.messaging.zmq__init.<clinit>(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at clojure.lang.RT.loadClassForName(RT.java:1578)
at clojure.lang.RT.load(RT.java:399)
at clojure.lang.RT.load(RT.java:381)
at clojure.core$load$fn__4511.invoke(core.clj:4905)
at clojure.core$load.doInvoke(core.clj:4904)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at clojure.core$load_one.invoke(core.clj:4729)
at clojure.core$load_lib.doInvoke(core.clj:4766)
at clojure.lang.RestFn.applyTo(RestFn.java:143)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$load_libs.doInvoke(core.clj:4800)
at clojure.lang.RestFn.applyTo(RestFn.java:138)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$require.doInvoke(core.clj:4869)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at backtype.storm.messaging.loader
$mk_zmq_context.doInvoke(loader.clj:8)
at clojure.lang.RestFn.invoke(RestFn.java:437)
at backtype.storm.daemon.worker
$fn__3031$exec_fn__855__auto____3032.invoke(worker.clj:109)
at clojure.lang.AFn.applyToHelper(AFn.java:187)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:540)
at backtype.storm.daemon.worker
$fn__3031$mk_worker__3169.doInvoke(worker.clj:78)
at clojure.lang.RestFn.invoke(RestFn.java:513)
at backtype.storm.daemon.worker$_main.invoke(worker.clj:244)
at clojure.lang.AFn.applyToHelper(AFn.java:174)
at clojure.lang.AFn.applyTo(AFn.java:151)
at backtype.storm.daemon.worker.main(Unknown Source)
2011-10-17 11:48:47 util [INFO] Halting process: ("Error on
initialization")
2011-10-17 11:49:21 ZooKeeper [INFO] Client
environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32
GMT
2011-10-17 11:49:21 ZooKeeper [INFO] Client
environment:host.name=192.168.0.172
2011-10-17 11:49:21 ZooKeeper [INFO] Client
environment:java.version=1.6.0_17
2011-10-17 11:49:21 ZooKeeper [INFO] Client
environment:java.vendor=Sun Microsystems Inc.
2011-10-17 11:49:21 ZooKeeper [INFO] Client environment:java.home=/usr/
java/jre1.6.0_17

=================================

Sorry about the messy copying, this forum does not give me any option
to attach the logs....

Thanks!
Aditya

[Aditya]

I think this problem was solved here
http://groups.google.com/group/storm-user/browse_thread/thread/33a84304016da7a5/902363b3e5d6c495?#902363b3e5d6c495

I'll try following these steps and getting back to you..

> herehttps://github.com/nathanmarz/jzmq