Submit a topology in storm production cluster using IDE

[Abhijit Chanda]

I am facing an issue Must submit topologies using the 'storm' client script so that StormSubmitter knows which jar to upload while submitting a topology to a production cluster using IDE, while the same thing if i perform in command line using storm jar command, its running like heaven. I have seen examples of the same from githublink. 

For submitting topology i am using these set of lines

    conf.put(Config.NIMBUS_HOST, NIMBUS_NODE);
    conf.put(Config.NIMBUS_THRIFT_PORT,6627);
    conf.put(Config.STORM_ZOOKEEPER_PORT,2181);
    conf.put(Config.STORM_ZOOKEEPER_SERVERS,ZOOKEEPER_ID);
    conf.setNumWorkers(20);
    conf.setMaxSpoutPending(5000);
    StormSubmitter submitter = new StormSubmitter();
    submitter.submitTopology("test", conf, builder.createTopology());

Please suggest me where i am making stuff cooked up. 

Thanks & Regards, 

Abhijit Chanda
+91-9748888395

[Abhijit Chanda]

Just a quick follow up, i am using storm 0.8.1

--

Abhijit Chanda
+91-9748888395

[Viral Bajaria]

Are you using "storm jar" to submit your topology to the production cluster ?

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Abhijit Chanda]

i have tried it. but i guess storm jar is the command line way, isn't so?

--

Abhijit Chanda
+91-9748888395

[Viral Bajaria]

Yeah that's for submitting topology from the command line, are you looking to do it via code ? If yes, you need to set a flag that the "storm jar " adds to the java -jar command. I don't have access to my code base right now but can reply sometime tomorrow when I have access to it.

-Viral

[Abhijit Chanda]

Thanks viral. I am looking exactly for the same. For today can u share some link of the same thing so that i can dig into the matter further.

Best Regards,

Abhijit Chanda
+91-9748888395

[Viral Bajaria]

Sorry for the delay. When you run "storm jar" it sets a property flag for storm.jar to the jar that you are submitting. So if you want to programmatically submit a jar then set that flag as follows:

System.setProperty("storm.jar", <path-to-jar>);

[Abhijit Chanda]

Hi Viral,

Thanks a lot for the suggestion. Now i am able to submit a topology in my production cluster. But unlikely its not doing anything which it is supposed do, though the status of the topology is ACTIVE( Information from Storm UI). Is there any trick still left to run the topology? 

[Viral Bajaria]

Not that I know of, does it report any errors ? Do you see any workers assigned to the topology ? If not, can you check nimbus logs and/or supervisor logs ?

Can you also post the screenshot of the storm UI which shows the topology not doing anything ?

-Viral

[Abhijit Chanda]

Hi Viral,

Sorry for the delayed response. 

Kindly find the nimbus.log, supervisor.log, worker.log along with the storm UI screen shots.

Interesting exception found in worker.log 

java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar

and CancelKeyException in zookeper.log, which is understandable. 

[Enno Shioji]

Well, are you bundling Storm jars with your topology? ;) The storm jar of the cluster will be used, so you don't want to include this jar in your job jar.

If you are using maven, you should set the scope of storm dependency to "provided" to avoid it being included in your job jar.

If you are unsure, you could even unzip your job jar to see if the storm jar is included & even manually delete the jar and zip it back to a jar and try if that fixes it (you want to *properly* fix your build after that, though)

[Abhijit Chanda]

Yes i am bundling storm jar with my topology though am using maven. If i don't bundle the jar with my topology, its getting exception what i have already mentioned  in my question.

[Viral Bajaria]

You should not bundle the storm jar when running remotely since the supervisor and workers will have it in its classpath.

What error do you get when you don't bundle it ?

---

From: Abhijit Chanda
Sent: 4/9/2013 3:37 AM
To: storm...@googlegroups.com
Subject: Re: [storm-user] Re: Submit a topology in storm production cluster using IDE

[Abhijit Chanda]

Must submit topologies using the 'storm' client script so that StormSubmitter knows which jar to upload

[pacunar]

I think what Enno means is that you don't need to include storm.jar when running remotely, since that is already in the remote machines. The "jar to upload" in the error is the jar that includes your code.

In your pom.xml be sure your storm dependency is as follows: 

                <dependency>

                        <groupId>storm</groupId>

                        <artifactId>storm</artifactId>

                        <version>0.8.2</version>

                        <!-- keep storm out of the jar-with-dependencies -->

                        <scope>provided</scope>

                </dependency>

[Abhijit Chanda]

Thanks for the response pacunar.  I have the exactly the same dependency tag, but with this dependency tag if i submit a topology then its throwing exception stating  Must submit topologies using the 'storm' client script so that StormSubmitter knows which jar to upload, for which i had asked the question in the forum. Then Viral  suggested me to set the storm jar flag using (mentioned in the discussion) System.setProperty("storm.jar","D:\\storm\\storm-0.8.1.jar");  before submitting the topology, and it worked for me. Now i can submit the topology in my production cluster, but it is not able do anything which it is intend to do. So i check the log files of the nimbus as well as worker. In nimbus node log there is not at all any exception trace, but in worker node's log i am finding this exception statements repeatedly  java.lang.RuntimeException :  Found multiple defaults.yaml resources. You are probably bundling the Storm jars with your topology jar

Any help will be highly appreciated.  

[SKG]

I saw similar message when I run storm ui - Found multiple defaults.yaml resources. You are probably bundling the Storm jars with your topology jar.

Then I just renamed the $STORM_HOME/defaults.yaml to $STORM_HOME/defaults.yaml.new
and the web application started.

Try this on the worker nodes.
I don't whether it is to be done like this as I didn't find in any documentation.

[Viral Bajaria]

Oh wait! My bad! If you are using "java -cp" to submit your topology and want to programmatically submit your jar to the remote cluster then set the property storm.jar to your compiled jar and not the storm jar. The storm jar should only be in your classpath.

Sorry if that wasn't clear in my original email.

[Abhijit Chanda]

Hi Viral,

Thanks for the response, but the things not getting clear for me. Can you please elaborate a little bit  using any example 

[Viral Bajaria]

Let's start with a clean slate. Can you copy/paste the command that you run to execute your topology ? You should be doing System.setProperty("storm.jar", <path-to-your-topology-jar>) and not putting storm-0.8.1.jar in there.

Why do you want to submit a jar to production from IDE ? I just saw your subject again and it seems weird that you want to submit to production from an IDE.

[Abhijit Chanda]

Basically i want to build topology jars in run time and submit them to production cluster. So i think in any IDE using maven, it will be easier task to handle those thing. Does that sounds weird? Am i not correct in my approach? 

      

[pacunar]

You can check this https://groups.google.com/forum/#!topic/storm-user/5JjZQyviRqk 

I don't know how outdated this is, but there are some suggestions to submit a topology without using the storm command, by changing the configuration and using a submitJar method. However, I think you have to generate your jar before and know where it is located. If I understand your last post, you want to build your code generating a JAR and submit the topology automatically using that JAR. I can only think of using separate projects (in your IDE), use one to build the JAR of your code and use the other one to submit the topology ?? I'm only guessing, but you can try :)

cheers, 

Pablo.

[Richards Peter]

The steps in that post are still valid. I am following those steps to submit topologies without storm submit command. But you have to create the jar containing your spout and bolt definitions before submitting the topology.

Richards Peter.

[Abhijit Chanda]

Thanks All. Now i am able to do the things i guess perfectly. 

But my bad, still getting an error in my worker node. Its throwing  Error on initialization of server mk-worker

org.zeromq.ZMQException: Invalid argument(0x16) and with this error all other worker nodes are forcibly closed.  

I googled with the error and got to know that  its may be due to ZeroMQ version mismatch. Right now I am using storm-0.8.1, so what will be suitable versions of dependent tools?

Currently i am using 

Storm-0.8.1

zookeeper-3.4.5

zeromq-3.2.2

jzmq-2.1.0

Also what will be dependent versions for storm-0.8.2? 

 

On Thu, Apr 11, 2013 at 7:54 AM, Richards Peter <hbkri...@gmail.com> wrote:

The steps in that post are still valid. I am following those steps to submit topologies without storm submit command. But you have to create the jar containing your spout and bolt definitions before submitting the topology.

Richards Peter.

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Viral Bajaria]

Get zeromq from here http://download.zeromq.org/zeromq-2.1.7.tar.gz

Storm does not work with zeromq 3.x, there is an open issue which will get rid of the zeromq dependency in a future release.

[Abhijit Chanda]

Thanks viral for the suggestion. Finally i am able to resolve all the probs. 

Thanks a lot, thank u all.