How to launch multiple workers? (ZeroRPC + Circus?)

150 views
Skip to first unread message

Andreu Correa Casablanca

unread,
Jun 17, 2014, 10:23:23 AM6/17/14
to zer...@googlegroups.com
Hello,

I'm new using ZeroRPC. I've built and RPC server using ZeroRPC (and Python).
At this moment I'm launching 4 workers with a bash script that launches four times
the script with the rpc server.

Each worker is binded to a different port using 127.0.0.1 as interface, and then I use HAProxy as a load
balancer to expose the RPC service to the world.

The bash script is something like:

#!/bin/bash

# We check if we are inside the VagrantBox or inside
# the Production Environment.
if [ -z "$RADIANTE_SERVER_DEVENV" ]
then
    export RADIANTE_SERVER_DEVENV=/home/vagrant/devenv
fi

if [ -z "$RADIANTE_SERVER_PATH" ]
then
    export RADIANTE_SERVER_PATH=/vagrant
fi

function stop_workers {
    kill $RAD1_PID
    kill $RAD2_PID
    kill $RAD3_PID
    kill $RAD4_PID
}

# Activating the Python Virtual Environment
source $RADIANTE_SERVER_DEVENV/bin/activate

# Capturing signals to stop the workers
trap 'stop_workers' TERM KILL QUIT

# These four workers are exposed to the world through
# HAProxy (using the port 31415).
#
# The HAProxy settings used in the local testing
# environment are placed in vagrant_assets/haproxy/ .
$RADIANTE_SERVER_PATH/radiante_worker.py 31411 rad1 &
RAD1_PID=$!
$RADIANTE_SERVER_PATH/radiante_worker.py 31412 rad2 &
RAD2_PID=$!
$RADIANTE_SERVER_PATH/radiante_worker.py 31413 rad3 &
RAD3_PID=$!
$RADIANTE_SERVER_PATH/radiante_worker.py 31414 rad4 &
RAD4_PID=$!

wait

# Deactivating the Python Virtual Environment
deactivate


Here I have the HAProxy settings file:

global
    log 127.0.0.1    local0
    log 127.0.0.1    local1 notice
    maxconn 8192
    chroot /var/lib/haproxy
    user haproxy
    group haproxy
    daemon

defaults
    log global
    mode tcp
    option dontlognull
    retries 3
    maxconn 16384
    contimeout 5000
    clitimeout 50000
    srvtimeout 50000

listen radiante :31415
    mode tcp
    option tcplog
    balance roundrobin
   
    server rad1 127.0.0.1:31411
    server rad2 127.0.0.1:31412
    server rad3 127.0.0.1:31413
    server rad4 127.0.0.1:31414


The question is: Exists a better approach? I've been looking around and I've discovered Circus,
but I'm struggling to discover how to bind (or connect :p) the workers to the unix sockets that creates Circus.

Using bash is ugly, If i want to make the script configurable I have to write a lot of crappy and repetitive
code, and using HAProxy adds another layer of complexity in the deployments, because I have more settings
files to take into account, and those settings files became spread around the different directories...

Thanks in advance for your time and attention :) .
Reply all
Reply to author
Forward
0 new messages