Local Mode without JZMQ

[Kurtis]

Hey Guys,

I'm struggling trying to get this running on Local Mode. I'm going to
post the steps I've taken to try to run the storm-starter project. If
you guys have any clues as to what I'm doing wrong, that would be
great. Thanks.

Firstly, I've read in other places (storm-user group, wiki) that Storm
uses a pure-Java messaging system in Local Mode. I'm hoping that it's
still true. Now for my installation steps:

1. Downloaded lein
2. Git cloned storm
3. Git cloned storm-starter
4. In storm directory, ran 'lein deps'
5. In storm-starter directory, ran 'lein deps'
6. In storm-start directory, ran 'lein uberjar'
7. In storm directory, ran 'lein compile'

This is where I ran in to problems. I get this exception (along with a
pretty long stack trace):

"Caused by: java.lang.UnsatisfiedLinkError: no jzmq in
java.library.path"

I'm assuming that I *do* need to compile storm completely to use it.
I'm also assuming that I need to include the Storm Class Path (storm/
classes) later on when I go to run the storm-starter project. Finally,
I'm assuming (based off reading other posts) that when I go to run the
project, it'll be something along the lines of this command:

./bin/storm jar ../storm-starter/storm-starter-0.0.1-SNAPSHOT-
standalone.jar storm.starter.ExclamationTopology

Side Note: There are several classes already in my /storm/classes/...
directory. I tried to run the above command to execute storm and get
this error:

"Exception in thread "main" java.lang.NoClassDefFoundError: backtype/
storm/command/config_value"

Thanks in advanced!

[nathanmarz]

Hey Kurtis,

There's no need to compile Storm yourself. storm-starter uses Maven to
pull in the Storm code as a dependency. Try this:

1. In storm-starter, run "lein deps && lein compile"
2. Run "lein repl"
3. In the REPL that opens, run the following commands (don't forget
the parens)

(import 'storm.starter.ExclamationTopology)
(ExclamationTopology/main nil)

You should see the topology run for about 10 seconds and then
shutdown.

If lein's not your thing, the following wiki page details how to setup
a project in Eclipse:

https://github.com/nathanmarz/storm/wiki/Creating-a-new-Storm-project

-Nathan

[Kurtis Mullins]

That ran fine so I'm guessing I'm not having any dependency issues. Am I doing something wrong with the way I'm trying to start this Topology? I'll definitely take a look at the Eclipse setup too. That may solve most, if not all, of my problems. Thanks for the quick reply!

[nathanmarz]

Yes, you are. "storm jar" is for submitting topologies to an actual
cluster. You can read more about local mode and remote mode on the
following wiki pages:

https://github.com/nathanmarz/storm/wiki/Local-mode
https://github.com/nathanmarz/storm/wiki/Running-topologies-on-a-production-cluster

-Nathan

[Kurtis]

Sorry if this is a dumb question -- so how do I start them? I looked
through the documentation as you listed and didn't see any examples of
starting topologies locally. The only thing I saw was how to code the
topologies for local deployment. This question might be apparent to
others but I have virtually no experience using Maven or anything else
to import Java dependencies and it's not as easy as just running Java
on a single class file -- at least from what I have tried.

Thanks again!

On Oct 14, 2:45 pm, nathanmarz <nathan.m...@gmail.com> wrote:
> Yes, you are. "storm jar" is for submitting topologies to an actual
> cluster. You can read more about local mode and remote mode on the
> following wiki pages:
>

> https://github.com/nathanmarz/storm/wiki/Local-modehttps://github.com/nathanmarz/storm/wiki/Running-topologies-on-a-prod...

[Kurtis]

One side note -- I looked into the Eclipse setup. According to
https://github.com/nathanmarz/storm/wiki/Creating-a-new-Storm-project
I need to include the Storm jars. However, I can't compile Storm (as
previously mentioned) because of missing dependencies (specifically
JZMQ and by depedency, uzmq as well). So basically I'm pretty lost
when it comes to all of this. If anyone can help me understand what
I'm missing, which might be something obvious, I'd be happy to write
up a little tutorial so that you guys don't have to answer these kinds
of beginner questions again :)

On Oct 15, 4:24 pm, Kurtis <kurtis.mull...@gmail.com> wrote:
> Sorry if this is a dumb question -- so how do I start them? I looked
> through the documentation as you listed and didn't see any examples of
> starting topologies locally. The only thing I saw was how to code the
> topologies for local deployment. This question might be apparent to
> others but I have virtually no experience using Maven or anything else
> to import Java dependencies and it's not as easy as just running Java
> on a single class file -- at least from what I have tried.
>
> Thanks again!
>
> On Oct 14, 2:45 pm, nathanmarz <nathan.m...@gmail.com> wrote:
>
>
>
>
>
>
>
> > Yes, you are. "storm jar" is for submitting topologies to an actual
> > cluster. You can read more about local mode and remote mode on the
> > following wiki pages:
>

> >https://github.com/nathanmarz/storm/wiki/Local-modehttps://github.com......

[Kurtis]

Actually -- I may have read that "Creating-a-new-Storm-project" page
wrong. When it said to include the Storm Jars -- I thought it meant
that Storm itself had to be compiled as a Jar. I forgot that leiningen
would include Storm automatically (at least that's the idea) so I'm
going to give it a try and include /src/jvm from the storm-start
directory and not (as I mis-read) the /src/jvm from the Storm project.
I'd still like to see how to start the storm-starter topologies from
the command line if anyone can clarify those steps for me. Thanks!

On Oct 15, 4:30 pm, Kurtis <kurtis.mull...@gmail.com> wrote:
> One side note -- I looked into the Eclipse setup. According tohttps://github.com/nathanmarz/storm/wiki/Creating-a-new-Storm-project

[nathanmarz]

If you want to stick with leiningen (which is honestly probably the
easiest approach), you can just use the instructions I showed you
earlier for running things from the REPL:

1. In storm-starter, run "lein deps && lein compile"
2. Run "lein repl"
3. In the REPL that opens, run the following commands (don't forget
the parens)
(import 'storm.starter.ExclamationTopology)
(ExclamationTopology/main nil)

You can infer from that how to run other topologies in local mode. As
you can see, there's no need to make jars or use the "storm" client
program.

-Nathan

[Kurtis Mullins]

Thank you so much for your support Nathan! This works exactly as mentioned and I can finally start playing with your awesome project.