supervisord

[Terry Schutte]

Has anyone been able to setup supervisord with storm? I have it
working, but it seems to be eating the worker.log files generated by
the worker processes. I am still seeing the nimbus.log and
supervisor.log files after setting up supervision. I'm mainly
wondering if I missed a flag in the supervisord setup or if this is
simply a side-effect of the multiple spawning of java processes after
the storm supervisor is up and running.

[Mike Stanley]

I have storm running with daemontools.   

Nothing really special about my setup.  

My "/etc/service.d/storm-supervisor/run" script looks something like this:

#!/bin/sh

exec su -l storm -c "cd /opt/storm && ./bin/storm supervisor"

my worker.log files are definitely not being clobbered.

I have a similar run file for nimbus and storm-ui as well.   The only storm related process that is handled slightly differently is zookeeper.

cheers,

... Mike

[Terry Schutte]

Thanks Mike. I'm going to give Monit a try, and if that fails will
give daemontools a go.

For reference, what OS are you running under?

> > the storm supervisor is up and running.- Hide quoted text -
>
> - Show quoted text -

[P. Taylor Goetz]

We're using supervisord, and ran into the exact same issue you did.

If supervisord runs as root, you need to use su to change users and load that users environment.

The relevant section for running a nimbus looks like:

[program:nimbus]
command=su -l storm --command='storm nimbus'
autorestart=true

And for storm supervisors:

[program:supervisor]
command=su -l storm --command='storm supervisor'
autorestart=true

The storm user's environment should have the $STORM_HOME/bin on the PATH

- Taylor

[Terry Schutte]

Taylor, thanks for the tip. I am running as root. I'll give your
suggestion a try today and hopefully that will fix this.

> >> - Show quoted text -- Hide quoted text -

[Terry Schutte]

This fix worked perfectly, thanks again Taylor.

[Terry Schutte]

One more note, in case this helps anyone else in the future. It took
me awhile to figure out at least, but that may be my lack of
experience with zookeeper.

When running zookeeper under supervision, use "zkServer.sh start-
foreground" instead of "zkServer.sh start". The regular start command
forks and exits too many times for supervisord to track the
application state successfully.

[Avijit Paul]

Hi Terry and Mike,

I setup supervisor but keep getting this 

root@yetanotherstorm:/home/ubuntu# supervisorctl

nimbus                           RUNNING    pid 1418, uptime 0:02:06

stormUI                          FATAL      Exited too quickly (process log may have details)

supervisor                       RUNNING    pid 1421, uptime 0:02:06

zookeeper                        FATAL      Exited too quickly (process log may have details)

And in my supervisord.conf file

[program:zookeeper]

command=/zookeeper/bin/zkServer.sh start-foreground

autorestart=true

stopsignal=KILL

[program:nimbus]

command=/storm/bin/storm nimbus

[program:supervisor]

command=/storm/bin/storm supervisor

[program:stormUI]

command=/storm/bin/storm UI

What am I doing wrong?

Regards

Avi

[James Xu]

in "/storm/bin/storm UI", UI should be lowercase: ui, you probably want to run each command individually before use supervisorctl

2012/4/21 Avijit Paul <avijit.p...@gmail.com>

[James Xu]

and in my zookeeper install, zkServer.sh dont have a start-foreground option.

2012/4/21 James Xu <xumingmin...@gmail.com>

[Randy E.]

I tried the command by hand:  zkServer.sh start-foreground

And, it said that it was already running, so I did:  zkServer.sh stop

Then restarted supervisord, and zookeeper stayed up.

FYI