Issue with 100% CPU usage

[Marko Tukiainen]

Hello all,

I've been experiencing a strange problem with Ratchet. I've set up a fairly simple WAMP server using Ratchet, with very light traffic. Yet, after a (seemingly random) while the script starts consuming 100% of the host machine's CPU without any explanation. It usually takes several hours or more for this issue to present itself, but eventually it does. Restarting the script fixes the issue - until the next time. 

I'm using php5.4.6 on Linux and version 0.2.0 of Ratchet, however this issue was present in the previous version of Ratchet as well. 

Has anyone else encountered anything like this, or is this a bug in my code? Also any ideas on how to start debugging this would be appreciated. 

Thanks. 

[cboden]

hi Marko,

This is a result of Ratchet using stream_select for its event loop.  It's known to be not-so efficient.  Your best option is to install libevent and the php extension for libevent and use that for your event loop in Ratchet.  Libevent uses a different polling method to handle events resulting in less intense CPU usage.

Cheers.

[Marko Tukiainen]

Ah, I see. Somehow I got the impression from the docs that simply using PHP 5.4+ would be more efficient and use libevent by default. I'll give this a try and see what happens. 

[Marko Tukiainen]

Nope, didn't appear to change anything, after a while the script starts eating very close to 100% CPU with marginal actual load. Based on vendor/react/event-loop/React/EventLoop/Factory.php the libevent detection should work automatically, or do I still need to toggle something to take advantage of it? 

[cboden]

Odds are it's still not using libevent.

Instead of doing a $server->run() do a die(var_dump($server->loop)); - that will show you if it's using StreamSelectLoop or LibeventLoop.  If it's using streams libevent probably isn't installed correctly.  You need to install the libevent library on your system and then PECL libevent 0.0.5 extension into PHP.

Cheers.

[Marko Tukiainen]

That was my first instinct as well, so I proceeded to check if the correct eventloop was instantiated. I tested it by using the same method as the EventLoop/Factory.php does, e.g. checking the existence of 'event_base_new' function. And it did exist. 

I also tried your version, and the var_dump shows LibEventLoop. If I run the script with an older version of PHP (which is also without libevent), then it shows StreamSelectLoop. 

So I'm pretty sure libevent is working. 

[cboden]

What happens if you create your own raw PHP script that enters a libevent using native functions listening to a TCP socket ?  Does it still consume 100% of the CPU?

[Marko Tukiainen]

Umm... I'm not sure how to implement one that would still have the same basic functionality as this one. The 100% CPU usage doesn't occur immediately, only after a few hours, which makes me believe that the clients that are connecting to the service are somehow the root cause of it. So I would probably have to implement the entire WAMP server, just without Ratchet, to see if there's a difference - because if there are no connections and no traffic at all, there's no CPU usage either.

[Chris Nguyen]

I have same problem. How to solve it ?

Vào 15:00:45 UTC+7 Thứ hai, ngày 22 tháng mười năm 2012, Marko Tukiainen đã viết:

[Marko Tukiainen]

I actually didn't find a cause for the issue, so I had to make a bit of a hack to "solve" it.

I installed a daemon called "monit" and added the following to /etc/monit/conf.d/wamp.monitrc

check process wamp matching "php5.4.6"

  start program "/usr/local/bin/wamp" uid 1000 gid 1000

  stop program "/usr/bin/killall php5.4.6"

  if totalcpu > 90% for 2 cycles then restart

And set the interval in /etc/monit/monitrc to 15 seconds (set daemon 15). As you can see, monit searches for any processes that match php5.4.6 (I renamed the php binary to that) and checks if the cpu usage exceeds 90% for two cycles = 30 sec, and then just kills and restarts the damn thing.

Obviously this isn't an optimal solution, but since this application isn't critical or very heavily loaded, it has worked just fine. The logs say that the script gets killed two to five times a day on average.

However I've since then created another application that uses the exact same codebase for the wamp server logic, but uses a newer version of php (5.5.6) and a newer version of Ratchet, and a slightly different way of initializing the event loop. I haven't had any problems whatsoever with its CPU usage. 

Marko

[Chris Nguyen]

Dear Marko,
Thank you very much ! I have a hard working day; just installed supervisord, monit and did steps as you guided.
Visit my game (http://caro-game.com).

Vào 23:16:33 UTC+7 Thứ năm, ngày 09 tháng một năm 2014, Marko Tukiainen đã viết: