getting error in running simple storm topology

[Kartik Kumar]

Hi,

I am getting error in running one sample programe in two node cluster.
The problem seem to be on the node where nimbus is running.
I am putting whole stack trace here , please help me how to fix.

{RAW_INPUT_DIR=/home/storm/storm-0.5.3/data/,
nimbus.host=10.58.116.133, topology.workers=10, topology.debug=true,
topology.message.timeout.secs=500}
0 [main] INFO backtype.storm.StormSubmitter - Jar not uploaded to
master yet. Submitting jar...
Exception in thread "main" java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:
36)
at
backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:
17)
at backtype.storm.StormSubmitter.submitJar(StormSubmitter.java:
77)
at backtype.storm.StormSubmitter.submitJar(StormSubmitter.java:
70)
at
backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:49)
at
storm.starter.tnp.ThresholdTopology.main(ThresholdTopology.java:60)
Caused by: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:185)
at
org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:
81)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:
34)
... 5 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:
333)
at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.apache.thrift.transport.TSocket.open(TSocket.java:180)
... 7 more
[root@sai133 bin]#

[nathanmarz]

It's failing to connect to the Nimbus server. A few things:

1. Can you send me the nimbus.log file?
2. Do you have nimbus.host configured in your ~/.storm/storm.yaml?
3. Is Nimbus's thrift port open on its node?

-Nathan

[Charanya natesan]

Hi Nathan,

Please find the nimbus.log file and the contents of the storm.yaml file attached here.

The zookeeper is running on port no 2180 on another machine.

We have the storm.yaml file copied to ~/.storm directory also

Regards,
kartik & Charanya

[Nathan Marz]

I see. Nimbus is unable to connect to the Zookeeper server. Do you have the "storm.zookeeper.servers" property set in the storm.yaml on the Nimbus host? If so, can you send me Nimbus's storm.yaml file?

-Nathan

--
Twitter: @nathanmarz
http://nathanmarz.com

[Charanya natesan]

These are the contents of the storm.yaml file.

storm.zookeeper.servers: "10.58.116.132"
storm.local.dir: "/mnt/storm"
nimbus.host: "10.58.116.133"
ui.port: 9090
storm.zookeeper.port: 2180

We changed the storm.zookeeper.port from 2181 to 2180 and tried running "./storm nimbus" again but the nimbus logs say that it is still trying to connect to 2181.

-Charanya

[Nathan Marz]

storm.zookeeper.servers should be a list, like so:

storm.zookeeper.servers:

    - "10.58.116.132"

That should fix your problem.

-Nathan

[Kartik Kumar]

Hi Nathan,

 

Thanks it works now.

 

Regards,

 

Kartik

[Tejas]

Hi Nathan,

We are able to successfully run the topology now, but when try killing
it by running, "./storm kill <name of our topology>", it gives the
below error,

Exception in thread "main" java.lang.NullPointerException
at org.apache.thrift.transport.TSocket.open(TSocket.java:168)
at
org.apache.thrift.transport.TFramedTransport.open(TFramedTransport.java:
81)
at backtype.storm.thrift
$nimbus_client_and_conn.invoke(thrift.clj:43)
at backtype.storm.command.kill_topology
$_main.invoke(kill_topology.clj:10)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.AFn.applyTo(AFn.java:151)
at backtype.storm.command.kill_topology.main(Unknown Source)

We have ~/.storm/storm.yaml configured.

-Tejas

On Dec 8, 12:37 pm, Kartik Kumar <kumkar...@gmail.com> wrote:
> Hi Nathan,
>
> Thanks it works now.
>
> Regards,
>
> Kartik
>
>
>
>
>
>
>
> On Thu, Dec 8, 2011 at 12:55 PM, Nathan Marz <nathan.m...@gmail.com> wrote:
> > storm.zookeeper.servers should be a list, like so:
>
> > storm.zookeeper.servers:
> >     - "10.58.116.132"
>
> > That should fix your problem.
>
> > -Nathan
>

> > On Thu, Dec 8, 2011 at 2:24 AM, Charanya natesan <chara...@gmail.com>wrote:
>
> >> These are the contents of the storm.yaml file.
>
> >> storm.zookeeper.servers: "10.58.116.132"
> >> storm.local.dir: "/mnt/storm"
> >> nimbus.host: "10.58.116.133"
> >> ui.port: 9090
> >> storm.zookeeper.port: 2180
>
> >> We changed the storm.zookeeper.port from 2181 to 2180 and tried running
> >> "./storm nimbus" again but the nimbus logs say that it is still trying to
> >> connect to 2181.
>
> >> -Charanya
>

> >> On Thu, Dec 8, 2011 at 12:46 PM, Nathan Marz <nathan.m...@gmail.com>wrote:
>
> >>> I see. Nimbus is unable to connect to the Zookeeper server. Do you have
> >>> the "storm.zookeeper.servers" property set in the storm.yaml on the Nimbus
> >>> host? If so, can you send me Nimbus's storm.yaml file?
>
> >>> -Nathan
>

> >>> On Thu, Dec 8, 2011 at 2:12 AM, Charanya natesan <chara...@gmail.com>wrote:
>
> >>>> Hi Nathan,
>
> >>>> Please find the nimbus.log file and the contents of the storm.yaml file
> >>>> attached here.
>
> >>>> The zookeeper is running on port no 2180 on another machine.
>
> >>>> We have the storm.yaml file copied to ~/.storm directory also
>
> >>>> Regards,
> >>>> kartik & Charanya
>

[Tejas]

Hi Nathan,

It was due to permissions problem, storm.yaml file was not
accessible..

now it is solved

Thanks,
Tejas