Hookbox performance issue and workaround

[marie_dk]

First of all, I am extremely exited to know that Ergo^ and Rob is
going to pick up where mcarter left off with hookbox. Hookbox is too
good to die...

If I had known about the performance issues I would probably think
twice about choosing hookbox, but I am not sure I would have been able
to find another comet server as flexible and easy to work with.

As Ergo^ once said to me, it is still possible to give users a good
experience despite the problems. I will describe my solution, and
maybe others will share there workarounds too...

I am running hookbox on FreeBSD, and using monit (mmonit.org) to check
on hookbox every 10 seconds. This is my monit configuration:

---------------------------------------------
set daemon 10 with start delay 240
set logfile /var/log/monit.log
set mailserver localhost
set alert xx...@xxxxxx.xx
set httpd port 2812 and
use address localhost # only accept connection from localhost
allow localhost # allow localhost to connect to the server
and allow localhost


check process hookbox-prod with pidfile /var/log/hookbox/hookbox-
prod.pid
start program = "/usr/local/etc/rc.d/hookbox-prod.sh start" with
timeout 60 seconds
stop program = "/usr/local/etc/rc.d/hookbox-prod.sh stop"
if failed host 12.34.56.78 port 80 then restart #check comet server
if failed host 10.0.0.123 port 8095 then restart #check REST api
---------------------------------------------

If hookbox stops responding on any of the two interfaces, monit is
going to restart it within 10 seconds.

At the client side I have a loadChat() function to init the chat
(connect at subscribe to channels etc..). If the onClose() event is
fired

hbxConn.onClose = function() {
resetGlobals();
542 if (dojo.byId('preloader')) {
543 dojo.byId('preloader').style.display='block';
544 }
545 var rand_no = Math.floor((3000-1000)*Math.random()) + 1000;
546 var sec = Math.ceil(rand_no / 1000);
547 //console.log('Forbindelsen genoprettes om '+sec+' sekunder');
548 setTimeout('loadChat()',rand_no);
549 }

[marie_dk]

Please disregard my prior post... I was not finished, but my finger
slipped and off it went :-/

Here we go again...

First of all, I am extremely exited to know that Ergo^ and Rob is
going to pick up where mcarter left off with hookbox. Hookbox is too
good to die...

If I had known about the performance issues I would probably think
twice about choosing hookbox, but I am not sure I would have been able
to find another comet server as flexible and easy to work with.

As Ergo^ once said to me, it is still possible to give users a good
experience despite the problems. I will describe my solution, and
maybe others will share there workarounds too...

I am running hookbox on FreeBSD, and using monit (mmonit.org) to check
on hookbox every 10 seconds. This is my monit configuration:

---------------------------------------------
set daemon 10 with start delay 240
set logfile /var/log/monit.log
set mailserver localhost

set alert x...@xxxxxx.xx

set httpd port 2812 and
use address localhost # only accept connection from localhost
allow localhost # allow localhost to connect to the server
and allow localhost

check process hookbox-prod with pidfile /var/log/hookbox/hookbox-
prod.pid
start program = "/usr/local/etc/rc.d/hookbox-prod.sh start" with
timeout 60 seconds
stop program = "/usr/local/etc/rc.d/hookbox-prod.sh stop"
if failed host 12.34.56.78 port 80 then restart #check comet server
if failed host 10.0.0.123 port 8095 then restart #check REST api
---------------------------------------------

If hookbox stops responding on any of the two interfaces, monit is
going to restart it within 10 seconds.

At the client side I have a loadChat() function to init the chat
(connect at subscribe to channels etc..). If the onClose() event is
fired

hbxConn.onClose = function() {
resetGlobals();

if (dojo.byId('preloader')) {
dojo.byId('preloader').style.display='block';

}
var rand_no = Math.floor((3000-1000)*Math.random()) + 1000;

var sec = Math.ceil(rand_no / 1000);

setTimeout('loadChat()',rand_no);
}

I am displaying a loading *.gif to prevent the user from sending any
messages while the chat reconnects. And I also make the user wait for
1 to 3 seconds randomly before reconnecting.

The onClose is not fired until hookbox starts responding again.
Therefore I also count the seconds between messages. If 5 seconds
passes I assume hookbox is unresponsive and show the loading message.

It works rather well. We tested this setup yesterday and most of the
time hookbox is restarted so fast that the user never sees the loading
message. And in any other case, we have ensured that the user is
reconnected and able to continue chatting without intervention.

Any suggestions for improvement are welcome...


/marie_dk

[Andy K]

I have a probably similar setup running on my Ubuntu to work around
the unexpected crashes.
This php script is executed in the background with nohup:
<?php
function is_hookbox_running()
{
$arr = array();
exec("ps ax | grep hookbox", $arr);

$is_running = FALSE;
foreach($arr as $line)
{
if(strpos($line, '/usr/local/bin/hookbox') !== FALSE)
{
$is_running = TRUE;
}
}
return $is_running;
}

function start_hookbox()
{
$arr = array();
exec("nohup /usr/bin/python2.6 /usr/local/bin/hookbox -a apassword -r
anotherpassword --cb-connect=connect --cb-
create_channel=create_channel --cb-destroy_channel=destroy_channel --
cb-subscribe=subscribe --cb-unsubscribe=unsubscribe --cb-
publish=publish --cb-disconnect=disconnect --cbpath=/index.php/hookbox
> out.txt &", $arr);
}

while(TRUE)
{
if(!is_hookbox_running())
{
$fh = fopen('starthookbox.log', 'a');
$fw = fwrite($fh, date('Y-m-d H:i:s')."\n");
fclose($fh);
start_hookbox();
}
sleep(3);
}

Basically it executes ps ax every 3 seconds to look for hookbox
process, if not found, start it again. Right now all the users must
refresh their browser to reconnect and the "online users" list gets
emptied, but you are saying that onClose() would be fired when hookbox
starts again, right? Didn't know about this. Definitely gonna try and
tinker around with this later.

Thanks marie_dk.