php-fpm unix-socket contention

[sumitg]

I recently tried listening on unix sockets instead of IP sockets for a slow application I had, and saw that the performance (tested using ab) using unix sockets was far greater. However, on heavy loads, I started getting this in the nginx error logs:

...connect() to unix:/dev/shm/php-fpm.1.socket failed (11: Resource temporarily unavailable) while connecting to upstream...

I made multiple pools in php-fpm.conf, but still get the errors. Many (but not all) requests fail during loads, some requests result in partial data transfers. It seems that there is some sort of contention that's happening over the unix-sockets, which wasn't there in IP-sockets. Can someone with a working unix-sockets based setup help me out here, as the performance gain is quite impressive?

Thanks,

Sumit.

Posted at Nginx Forum: http://forum.nginx.org/read.php?3,4542,4542#msg-4542

[Michael Shadle]

Weird, last I heard there was no real difference between unix sockets
and TCP/IP. Maybe I should be looking into that too?!

What OS and version, distro, etc? What version of PHP?

[sumitg]

Using RHEL5, PHP 5.2.10, php-fpm, nginx 0.7.61

The difference in speed is massive. Also, while using IP Sockets, I was running out of sockets in my loadtests, as two sockets were used per connection, and a lot of them ended up in TIME_WAIT.

The only issue is the lack of reliability - many of my requests during loads fail. I tested some more, and I was able to handle 512 concurrents in various configurations, but if I bump it up to even 550, almost 50% of requests fail. The CPU is not the bottleneck - it's a dual quad-core xeon, 12MB cache. 8GB of RAM on the machie. During tests, there's only about 30% CPU utilization (apache+mod_php loads up all the cores fully, ngix just hums along)

The difference is speed probably comes from the fact that Unix sockets will not have the TCP overhead of IP sockets (which even the loopback interface has). The system does not have to follow framing and packet guidelines of TCP, and processes can effectively negotiate data exchange directly with each other.

Sumit.

Posted at Nginx Forum: http://forum.nginx.org/read.php?3,4542,4552#msg-4552