Bolt not emitting data
428 views
Skip to first unread message
Casey Sybrandy
Nov 10, 2011, 5:05:50 PM11/10/11
to storm-user
Hello,
I have a small topology that I'm trying to test, however I'm having a
bit of an issue with my bolt not emitting data. I think it may not be
receiving any data, though the spout is reporting that it has emitted
and transmitted data. What could be causing this issue?
I'm using storm-0.5.5-SNAPSHOT and have installed the versions of 0MQ
and jzmq pulled down by the install script that came with storm,
though it may have been the 0.5.4 version. I'm currently running the
cluster on one machine with one supervisor. The UI reports that a
number of tuples have been emitted, however this number seems off,
though that could be unrelated to main issue. Regardless, the bolt
should have emitted data.
Thanks.
I have a small topology that I'm trying to test, however I'm having a
bit of an issue with my bolt not emitting data. I think it may not be
receiving any data, though the spout is reporting that it has emitted
and transmitted data. What could be causing this issue?
I'm using storm-0.5.5-SNAPSHOT and have installed the versions of 0MQ
and jzmq pulled down by the install script that came with storm,
though it may have been the 0.5.4 version. I'm currently running the
cluster on one machine with one supervisor. The UI reports that a
number of tuples have been emitted, however this number seems off,
though that could be unrelated to main issue. Regardless, the bolt
should have emitted data.
Thanks.
Nathan Marz
Nov 11, 2011, 1:44:46 AM11/11/11
to storm...@googlegroups.com
Can you send me the code for the topology you're trying to run? Are you able to run the storm-starter topologies successfully?
Casey Sybrandy
Nov 14, 2011, 9:21:14 AM11/14/11
to storm-user
Hello,
First, no, I can't send you the code. However, all I'm trying to do
is read data from individual files as they appear on one node and emit
the results on a different node. I had success reading the data, but
the second part eludes me.
Second, I tried two sample topologies this morning and neither of them
emitted any data. I currently have the Reach Topology running for 10
minutes and no data has been emitted. I'm even trying this on a pair
of servers with 0.5.4 with no success. However, I am getting a number
of "No such process" messages from both supervisors.
Casey
First, no, I can't send you the code. However, all I'm trying to do
is read data from individual files as they appear on one node and emit
the results on a different node. I had success reading the data, but
the second part eludes me.
Second, I tried two sample topologies this morning and neither of them
emitted any data. I currently have the Reach Topology running for 10
minutes and no data has been emitted. I'm even trying this on a pair
of servers with 0.5.4 with no success. However, I am getting a number
of "No such process" messages from both supervisors.
Casey
nathanmarz
Nov 14, 2011, 2:46:58 PM11/14/11
to storm-user
I'm going to need more information to help debug this. The
ReachTopology is not a good example to test with, since it doesn't do
anything unless you send DRPC requests. Can you try running
ExclamationTopology on your cluster?
Can you send me all the logs from your cluster from when the topology
wasn't working? This includes nimbus.log, supervisor.log, and all the
worker.log.
-Nathan
ReachTopology is not a good example to test with, since it doesn't do
anything unless you send DRPC requests. Can you try running
ExclamationTopology on your cluster?
Can you send me all the logs from your cluster from when the topology
wasn't working? This includes nimbus.log, supervisor.log, and all the
worker.log.
-Nathan
Casey Sybrandy
Nov 14, 2011, 3:01:17 PM11/14/11
to storm-user
I did test with the ExclamationTopology as well and got the same
results.
As for the log files, for some reason on the real cluster, I'm not
getting any log files in the logs directory, so I can't send you
those. This, I think, is something that I did. As for my local
machine, I just figured out that zookeeper is having an IPv6 vs. IPv4
issue, so I'm trying to resolve that right now so I can get everything
running locally in a pseudo-cluster with one supervisor. As soon as I
do, I'll get you some log files.
Thanks.
Casey
results.
As for the log files, for some reason on the real cluster, I'm not
getting any log files in the logs directory, so I can't send you
those. This, I think, is something that I did. As for my local
machine, I just figured out that zookeeper is having an IPv6 vs. IPv4
issue, so I'm trying to resolve that right now so I can get everything
running locally in a pseudo-cluster with one supervisor. As soon as I
do, I'll get you some log files.
Thanks.
Casey
Nathan Marz
Nov 14, 2011, 11:22:20 PM11/14/11
to storm...@googlegroups.com
OK. it will be useful to see the logs from when ExclamationTopology runs, as the fact that that's not working indicates something unusual with your setup.
Casey Sybrandy
Nov 15, 2011, 9:31:43 AM11/15/11
to storm-user
O.K. It looks like everything is working now. I saw data being
emitted and what not in the ExclamationTopology and my code is showing
data being emitted and acked. I'm still a bit confused here because
there are 10 lines in each file, so I'm expecting to see 33 records
emitted by my spout (I also send an EOF marker), however I'm seeing 40
records being emitted. Then, on the bolt side, I only see 20 acks.
Is this normal?
As for what I did to get this working, here are a couple things that
may/may not have made a difference:
- For the zookeeper directory, I had a trailing slash before and
noticed that it put a double-slash in the path the supervisor was
trying to download the jar from, so I removed it.
- For some reason, zookeeper and storm both tried to use IPv6 vs.
IPv4, so I modified the associated scripts to force java to use IPv4.
I think this made a difference as I reset my nimbus host configuration
variable back to localhost (I tried an IP address at one point in
time) and everything started working. Granted, this may not be
common, but I figured it was worth mentioning in case someone else
runs into this. I also don't know the "correct" way of resolving this
issue without changing the storm script and the zkServer script, so I
won't detail what I did except to mention this flag: -
Djava.net.preferIPv4Stack=true
Casey
emitted and what not in the ExclamationTopology and my code is showing
data being emitted and acked. I'm still a bit confused here because
there are 10 lines in each file, so I'm expecting to see 33 records
emitted by my spout (I also send an EOF marker), however I'm seeing 40
records being emitted. Then, on the bolt side, I only see 20 acks.
Is this normal?
As for what I did to get this working, here are a couple things that
may/may not have made a difference:
- For the zookeeper directory, I had a trailing slash before and
noticed that it put a double-slash in the path the supervisor was
trying to download the jar from, so I removed it.
- For some reason, zookeeper and storm both tried to use IPv6 vs.
IPv4, so I modified the associated scripts to force java to use IPv4.
I think this made a difference as I reset my nimbus host configuration
variable back to localhost (I tried an IP address at one point in
time) and everything started working. Granted, this may not be
common, but I figured it was worth mentioning in case someone else
runs into this. I also don't know the "correct" way of resolving this
issue without changing the storm script and the zkServer script, so I
won't detail what I did except to mention this flag: -
Djava.net.preferIPv4Stack=true
Casey
nathanmarz
Nov 15, 2011, 4:10:30 PM11/15/11
to storm-user
Hey Casey,
Storm by default only increments stats for the Storm UI once every 20
tuples, which is why the numbers look a little off. If you want, you
can change the sampling rate by setting the
"topology.stats.sample.rate" config.
Glad to hear you got things working.
-Nathan
Storm by default only increments stats for the Storm UI once every 20
tuples, which is why the numbers look a little off. If you want, you
can change the sampling rate by setting the
"topology.stats.sample.rate" config.
Glad to hear you got things working.
-Nathan
Casey Sybrandy
Nov 15, 2011, 4:27:24 PM11/15/11
to storm-user
Nathan,
O.K. That's cool. I ended up changing the bolt a bit to write data
to a file and it worked perfectly. I'm still having an issue on our
cluster, but that looks to be a classpath issue.
Anyway, thanks for your help.
Casey
O.K. That's cool. I ended up changing the bolt a bit to write data
to a file and it worked perfectly. I'm still having an issue on our
cluster, but that looks to be a classpath issue.
Anyway, thanks for your help.
Casey
Reply all
Reply to author
Forward
0 new messages