Rachet chat application - connection closed immediately after established when the app runs for a pe
441 views
Skip to first unread message
al...@deefuse.com
Aug 13, 2014, 5:08:35 AM8/13/14
to ratch...@googlegroups.com
Hi,
We're using Laravel 4 together with Ratchet to create a chat application. Everything runs normally for about 14-20 hours. After a period of time the chat app stops to run. The connection gets established from the client to the server but right after that the server closes the connection.
No errors get reported in our log files and the fact that we haven't been able to replicate the problem in our development environment doesn't help.
Restarting the chat application on the server fixes the issue for another 14-20 hours.
Supervisor configuration:
[program:chat]
command = bash -c "ulimit -n 10000 && /usr/bin/php /var/www/artisan setup:chat --env=staging"
process_name = chat
numprocs = 1
autostart = true
autorestart = true
user = root
stdout_logfile = /var/www/app/storage/logs/chat_info.log
stdout_logfile_maxbytes = 1MB
stderr_logfile = /var/www/app/storage/logs/chat_error.log
stderr_logfile_maxbytes = 1MB
SetupChatCommand.php (Laravel Setup Chat Command):
<?php
use Illuminate\Console\Command;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Input\InputArgument;
use Application\Chat\Server;
class SetupChatCommand extends Command {
/**
* The console command name.
*
* @var string
*/
protected $name = 'setup:chat';
/**
* The console command description.
*
* @var string
*/
protected $description = 'Setup the chat server';
/**
* Create a new command instance.
*
* @return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* @return mixed
*/
public function fire()
{
$server = new Server();
$server->run();
}
}
Server.php:
<?php namespace Application\Chat;
use React\EventLoop\Factory;
use React\Socket\Server as Reactor;
use Ratchet\Server\IoServer;
use Ratchet\Server\FlashPolicy;
use Ratchet\Http\HttpServer;
use Ratchet\WebSocket\WsServer;
class Server {
const CHAT_PORT = 7778;
const FLASH_PORT = 843;
public function __construct()
{
$this->_setup();
}
private function _setup()
{
$loop = Factory::create();
$web_socket = new Reactor($loop);
$web_socket->listen(self::CHAT_PORT, '0.0.0.0');
$this->_server = new IoServer(
new HttpServer(
new WsServer(
new Service()
)
)
, $web_socket
, $loop
);
$flash_policy = new FlashPolicy();
$flash_policy->addAllowedAccess('*', self::CHAT_PORT);
$flash_socket = new Reactor($loop);
$flash_socket->listen(self::FLASH_PORT, '0.0.0.0');
$flash_server = new IoServer($flash_policy, $flash_socket);
}
public function run()
{
$this->_server->run();
}
}
Service.php:
<?php namespace Application\Chat;
use SplObjectStorage;
use Ratchet\ConnectionInterface;
use Ratchet\MessageComponentInterface;
class Service implements MessageComponentInterface {
public function __construct()
{
$this->_setupClients();
}
/**
* Clients
*/
private $_clients = null;
private function _setupClients()
{
$this->_clients = new SplObjectStorage();
}
public function getClientByConnection($connection)
{
foreach ($this->_clients as $client)
{
if($client->getConnection() === $connection)
{
return $client;
}
}
return null;
}
public function getClientsByRoom($room)
{
$clients = array();
foreach ($this->_clients as $client)
{
if($client->getRoom()->id === $room->id)
{
array_push($clients, $client);
}
}
return $clients;
}
/**
* Input
*/
private function _handleInput($client, $input)
{
if(empty($input) || empty($input['type']) || empty($input['content'])) return;
switch ($input['type'])
{
case 'session.set':
$this->_handleInputSetSession($client, $input['content']);
break;
case 'room.set':
$this->_handleInputSetRoom($client, $input['content']);
break;
case 'message':
$this->_handleInputMessage($client, $input['content']);
break;
}
}
private function _handleInputSetSession($client, $input)
{
$client->setSession($input);
}
private function _handleInputSetRoom($client, $input)
{
$client->setRoom($input);
}
private function _handleInputMessage($client, $input)
{
$message = $client->message($input);
if($client->hasRoom() && $message)
{
$clients = $this->getClientsByRoom($client->getRoom());
foreach ($clients as $other)
{
if($other !== $client)
{
$other->getConnection()->send(json_encode(array(
'type' => 'message.get'
, 'content' => $message->toArray()
)));
}
}
}
}
/**
* Callbacks
*/
public function onOpen(ConnectionInterface $connection)
{
$client = new Client($connection);
$this->_clients->attach($client);
}
public function onMessage(ConnectionInterface $connection, $input)
{
$client = $this->getClientByConnection($connection);
$input = json_decode($input, true);
$this->_handleInput($client, $input);
}
public function onClose(ConnectionInterface $connection)
{
$client = $this->getClientByConnection($connection);
if($client)
{
$this->_clients->detach($client);
}
}
public function onError(ConnectionInterface $connection, \Exception $e)
{
$client = $this->getClientByConnection($connection);
if($client)
{
$client->getConnection()->close();
}
}
}
Client.php:
<?php namespace Application\Chat;
use App;
use Config;
use Application\Models\ChatRoom;
use Application\Models\ChatRoomUser;
use Application\Models\ChatRoomMessage;
use Application\Models\File;
class Client {
/**
* Constructor & Destructor
*/
public function __construct($connection = null)
{
if($connection)
{
$this->setConnection($connection);
}
}
public function __destruct()
{
if($this->hasRoom())
{
$this->takenUserOfflineForRoomId($this->getRoom()->id);
}
}
/**
* Connection
*/
protected $_connection = null;
public function getConnection()
{
return $this->_connection;
}
public function setConnection($connection)
{
$this->_connection = $connection;
}
/**
* Session
*/
public function setSession($input)
{
Config::set('session.driver', 'database');
$session_id = $input;
$session = App::make('session');
$session->setDefaultDriver(Config::get('session.driver'));
$session->driver()->setId($session_id);
$session->driver()->start();
$cartalyst_session = $session->driver()->get(
Config::get('cartalyst/sentry::cookie.key')
);
if(!empty($cartalyst_session))
{
$this->setUserId($cartalyst_session[0]);
}
else
{
throw new \Exception('User not recognized.');
}
}
/**
* User id
*/
private $_user_id = null;
private function setUserId($id)
{
$this->_user_id = $id;
}
public function getUserId()
{
return $this->_user_id;
}
/**
* Room
*/
private $_room = null;
public function getRoom()
{
return $this->_room;
}
public function setRoom($input)
{
if(empty($input) || empty($input['id']))
{
throw new \Exception('Invalid chat room.');
}
$this->_room = ChatRoom::find($input['id']);
$this->takeUserOnlineForRoomId($this->getRoom()->id);
}
public function hasRoom()
{
if($this->_room)
{
return true;
}
return false;
}
/**
* User room status
*/
public function takeUserOnlineForRoomId($room_id)
{
$chat_room_user = ChatRoomUser::where('chat_room_id', '=', $room_id)
->where('user_id', '=', $this->getUserId())
->first();
if($chat_room_user)
{
$chat_room_user->status = ChatRoomUser::STATUS_ONLINE;
$chat_room_user->save();
}
}
public function takenUserOfflineForRoomId($room_id)
{
$chat_room_user = ChatRoomUser::where('chat_room_id', '=', $room_id)
->where('user_id', '=', $this->getUserId())
->first();
if($chat_room_user)
{
$chat_room_user->status = ChatRoomUser::STATUS_OFFLINE;
$chat_room_user->save();
}
}
/**
* Message
*/
public function message($input)
{
$message = new ChatRoomMessage();
$message->user_id = $this->getUserId();
$message->status = ChatRoomMessage::STATUS_NEW;
$message->content = $input['content'];
$chat_room = $this->getRoom();
$chat_room->messages()->save($message);
$this->_attachInputFile($input, $message);
$message->load('user', 'user.profile', 'user.profile.picture');
return $message;
}
private function _attachInputFile($input, $message)
{
if(empty($input['file']) || empty($input['file']['id'])) return;
$file = File::where('user_id', '=', $this->getUserId())
->where('id', '=', $input['file']['id'])
->first();
if(!$file) return;
$message->file()->save($file);
$message->load('file');
}
}
I tend to think that its something related to the operating system or something that doesn't get cleaned correctly or reused... Again, no errors are being logged and the resource consumption is normal (so the script doesn't run out of RAM or kill the CPU).
Would greatly appreciate any hints or ideas!
Thanks, Alex
cboden
Aug 16, 2014, 9:16:44 AM8/16/14
to ratch...@googlegroups.com, al...@deefuse.com
Can you post the output of the command `composer show -i` please? The most common culprits that I've seen from this behaviour are an older version of React (we've fixed a couple bugs around this recently) and the "ulimit" being reached. Some operating systems have more than one place to set the limit for open file descriptors. I see you are setting a high ulimit in your supervisord file, but I've found one some Linux distributions I also have to configure pam.d to support a higher number of open files as well.
...
al...@deefuse.com
Aug 17, 2014, 7:07:42 AM8/17/14
to ratch...@googlegroups.com, al...@deefuse.com
Hi,
At the moment we're using Ratchet v0.3.0.
Here is the output from "composer show -i":
cartalyst/sentry v2.1.2 PHP 5.3+ Fully-featured Authent...
cboden/ratchet v0.3.0 PHP WebSocket library
ceesvanegmond/minify 1.1 A Laravel 4 package for minifyi...
classpreloader/classpreloader 1.0.2 Helps class loading performance...
d11wtq/boris v1.0.8
evenement/evenement v1.0.0 Événement is a very simple ev...
filp/whoops 1.0.10 php error handling for cool kids
google/google-api-php-client 0.6.2
guzzle/guzzle v3.7.4 Guzzle is a PHP HTTP client lib...
intervention/image dev-master 606228b Image handling and manipulation...
ircmaxell/password-compat 1.0.3 A compatibility library for the...
jeremeamia/SuperClosure 1.0.1 Doing interesting things with c...
laravel/framework v4.1.28 The Laravel Framework.
linkorb/jsmin-php 1.0.0 Unofficial package of jsmin-php
monolog/monolog 1.9.1 Sends your logs to files, socke...
natxet/CssMin 3.0.2 Minifying CSS
nesbot/carbon 1.8.0 A simple API extension for Date...
nikic/php-parser v0.9.4 A PHP parser written in PHP
patchwork/utf8 v1.1.22 Extensive, portable and perform...
pda/pheanstalk dev-master bc9489b PHP client for beanstalkd queue
php-instagram-api/php-instagram-api dev-master 7a796fd PHP Instagram API for PHP 5.3+
phpseclib/phpseclib 0.3.6 PHP Secure Communications Libra...
pimple/pimple v1.1.1 Pimple is a simple Dependency I...
predis/predis v0.8.5 Flexible and feature-complete P...
psr/log 1.0.0 Common interface for logging li...
react/event-loop v0.3.3 Event loop abstraction layer th...
react/socket v0.3.2 Library for building an evented...
react/stream v0.3.3 Basic readable and writable str...
stack/builder v1.0.1 Builder for stack middlewares b...
swiftmailer/swiftmailer v5.1.0 Swiftmailer, free feature-rich ...
symfony/browser-kit v2.4.4 Symfony BrowserKit Component
symfony/console v2.4.4 Symfony Console Component
symfony/css-selector v2.4.4 Symfony CssSelector Component
symfony/debug v2.4.4 Symfony Debug Component
symfony/dom-crawler v2.4.4 Symfony DomCrawler Component
symfony/event-dispatcher v2.4.4 Symfony EventDispatcher Component
symfony/filesystem v2.4.4 Symfony Filesystem Component
symfony/finder v2.4.4 Symfony Finder Component
symfony/http-foundation v2.4.4 Symfony HttpFoundation Component
symfony/http-kernel v2.4.4 Symfony HttpKernel Component
symfony/process v2.4.4 Symfony Process Component
symfony/routing v2.4.4 Symfony Routing Component
symfony/security v2.4.4 Symfony Security Component
symfony/translation v2.4.4 Symfony Translation Component
In regards to the idea you've provided about the ulimit, I will inspect pam.d and I will reply with whatever I can come up with.
Does this version of Ratchet seem old to you? I'm doing to do some experiments to upgrade it and see if everything works.
Thanks for your time.
Best,
Alex
al...@deefuse.com
Aug 17, 2014, 7:10:27 AM8/17/14
to ratch...@googlegroups.com, al...@deefuse.com
Ah, and the linux distro we use is Ubuntu 13.04. I am not sure if this is affected by the scenario that you mentioned but will definitely look into this as well.
On Saturday, August 16, 2014 4:16:44 PM UTC+3, cboden wrote:
cboden
Aug 17, 2014, 7:47:35 AM8/17/14
to ratch...@googlegroups.com, al...@deefuse.com
I think Ubuntu has the pam.d stuff, but I can't remember. I'd definitely hit up Google about "ubuntu open file descriptor limits".
Ratchet is now on version 0.3.2. There has been one direct CPU starvation fix and a couple indirect ones in React. If you can I'd highly suggest using PHP 5.4 (or higher). This will allow you to upgrade to React v0.4. I've had a few reports where people reported the same issue you're having and it turned out to be CPU starvation; they upgraded to the latest releases of Ratchet+React which solved their problem.
al...@deefuse.com
Aug 17, 2014, 8:10:25 AM8/17/14
to ratch...@googlegroups.com, al...@deefuse.com
Hi,
Thanks for your quick reply. I've just update the Ratchet version to 0.3.2. I'll leave this to run for a day or two and see if the app still crashes. We're running PHP 5.4.9 so I will try updating to React v0.4 if this app still crashes.
Will post some progress on this board once I have it.
Thanks again,
Alex
al...@deefuse.com
Aug 18, 2014, 12:07:23 PM8/18/14
to ratch...@googlegroups.com, al...@deefuse.com
Hi again,
I just wanted to post an update on this issue.
I've tried updating Ratchet to V 0.3.2 and also React to V 0.4.
The issue still persists (chat app still crashes).
Below are the exact versions of what we're currently using:
cboden/ratchet v0.3.2
react/event-loop v0.4.1
react/socket v0.4.2
react/stream v0.4.1
I will next try the "ubuntu open file descriptior limits" and will be posting and update on that as well.
Fingers crossed.
Thanks,
Alex
On Sunday, August 17, 2014 2:47:35 PM UTC+3, cboden wrote:
On Sunday, August 17, 2014 2:47:35 PM UTC+3, cboden wrote:
al...@deefuse.com
Aug 27, 2014, 12:58:22 AM8/27/14
to ratch...@googlegroups.com, al...@deefuse.com
Hi cboden,
I've also tried "ubuntu open file descriptor limit" for a few days. The chat app has ran for aproximately 48 hours now prior to manifesting the same behavior.
I've followed these two guides:
And below is the outcome:
cat /proc/sys/fs/file-max
65535
ulimit -Hn
65535
ulimit -Sn
65535
As I mentioned above, this had some effect (the app takes a lot longer to get to that point where it doesn't accept any more new connections). I can now say for sure that the app doesn't crash but it closes connections immediately after they are established.
So right now, I'm in a stage where all the requirements are checked and it still refuses to work indefinetely.
Any more ideas would be welcome.
Thank you,
Alex
On Sunday, August 17, 2014 2:47:35 PM UTC+3, cboden wrote:
cboden
Aug 29, 2014, 7:47:13 AM8/29/14
to ratch...@googlegroups.com, al...@deefuse.com
Check this post. There's pam.d or security limits.conf (among other things!) to look for. Configure all the things! ;)
Reply all
Reply to author
Forward
0 new messages