Running s4 with zookeeper across multiple machines

[Radhika Renganathan]

Hi,

We are trying to run s4 across two machines with zookeeper running on machine1 and trying to access it from machine 2.

We changed zk_address in s4-core.properties-header as below in machine 2 alone.

gc_opts=-server -XX:+HeapDumpOnOutOfMemoryError
-XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit
mem_opts=-Xms800m -Xmx2000m
remote_debug_enabled=no
pe_container_max_queue_size=8000
listener_max_queue_size=8000
commlayer_mode=dynamic
zk_session_timeout=5000
deamon_mode_enabled=true
zk_address=machine1:2181

Also in machine 1 s4-core.properties-header file we changed

deamon_mode_enabled=true

zk_address=localhost:2181

For testing speech 2 example, we started s4 in machine 1 in ordinary way and in machine 2 as

/root/Desktop/s4/build/s4-image/scripts/start-s4.sh -z machine1:2181 dynamic & 

we checked logs in machine 2 to ensure it is connecting to zookeeper running in machine 1 and it does!

However, we are confused whether to start client adapter in machine 2 as

$S4_IMAGE/scripts/run-client-adapter.sh -z 192.168.116.155:2181 -g s4
-d $S4_IMAGE/s4-core/conf/dynamic/client-stub-conf.xml dynamic &

We checked zkCli utility in machine 1 and it shows only "task-0"

On giving,

head -10 $S4_IMAGE/s4-example-testinput/speeches.txt | sh
$S4_IMAGE/s4-tools-loadgenerator/scripts/generate-load.sh -r 2 -

in machine 1 we get proper output and logs but in machine 2 there is not output and PE and event counts are 0 in logs.

Please help with us with proper way to run across multiple machines.

Also please tell whether we should access shared data for processing by both machines.