PHP-FPM socket disappearing

[Proxy One]

Hello,

I installed Jessie on my new server few days ago and moved website that run
previously on Centos 5. I'm using Apache and PHP-FPM. Currently, there is only
one website on that server, but I still configured pool for it. What's
happening is that, after some time, at least once a day, socket that should be
listening for that user disappears, resulting in this error in Apache error log:

FastCGI: incomplete headers (0 bytes) received from server
"/dev/shm/user-php.fcgi
FastCGI: failed to connect to server "/dev/shm/user-php.fcgi

Looking at /dev/shm/ directory, user-php.sock is really missing, but
default php5-fpm.sock is still there.

I have this block in VirtualHost section for that website:
<IfModule mod_fastcgi.c>
Alias /php5-fcgi /dev/shm/pdfconve-php.fcgi
</IfModule>


Pool configured:

[user]
listen = /dev/shm/user-php.sock
listen.owner = user
listen.group = www-data
listen.mode = 0660
user = user
group = user
pm = dynamic
pm.max_children = 40
pm.max_requests = 2048
pm.start_servers = 30
pm.min_spare_servers = 10
pm.max_spare_servers = 35
request_terminate_timeout = 305

And there is also
FastCGIExternalServer /dev/shm/user-php.fcgi -socket /dev/shm/user-php.sock -pass-header Authorization -flush -appConnTimeout 2 -idle-timeout 60
in apache conf.

There is nothing interesting in php-fpm error log. There are some errors like this:
NOTICE: [pool user] child 32719 exited with code 0 after 76324.921427 seconds from start

followed by this line:
NOTICE: [pool user] child 29909 started

but I don't think that's relevant.

Any idea why this is happening and how to prevent it?

Thanks!


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: https://lists.debian.org/20150617213...@angelina.example.com

[Bob Proulx]

Proxy One wrote:
> I installed Jessie on my new server few days ago and moved website
> that run previously on Centos 5.

Welcome! :-)

> I'm using Apache and PHP-FPM.

I have become an Nginx + php5-fpm advocate in recent years. If you
decide you would like to give it a try post something and I will show
my configurations for it. (Not that I am a master of it by any means.
Just a daily user of it.)

> Currently, there is only one website on that server, but I still
> configured pool for it. What's happening is that, after some time,
> at least once a day, socket that should be listening for that user

> disappears, ...
>...

> Looking at /dev/shm/ directory, user-php.sock is really missing, but
> default php5-fpm.sock is still there.

Odd!

> I have this block in VirtualHost section for that website:
> <IfModule mod_fastcgi.c>
> Alias /php5-fcgi /dev/shm/pdfconve-php.fcgi
> </IfModule>

>...

> And there is also
> FastCGIExternalServer /dev/shm/user-php.fcgi -socket /dev/shm/user-php.sock -pass-header Authorization -flush -appConnTimeout 2 -idle-timeout 60
> in apache conf.

Hmm... Is that right? It is working for you so it must be. And
since the disappearing fpm socket shouldn't be related I am going to
ignore my ignorance of the above here.

> Pool configured:
>
> [user]
> listen = /dev/shm/user-php.sock
> listen.owner = user
> listen.group = www-data
> listen.mode = 0660
> user = user
> group = user
> pm = dynamic
> pm.max_children = 40
> pm.max_requests = 2048
> pm.start_servers = 30
> pm.min_spare_servers = 10
> pm.max_spare_servers = 35
> request_terminate_timeout = 305

Why use /dev/shm/user-php.sock as the socket path? The Jessie-style
location would be in /var/run/user-php.sock AFAICS. (I don't see how
that would be related to your socket dissappearing.)

> There is nothing interesting in php-fpm error log. There are some errors like this:
> NOTICE: [pool user] child 32719 exited with code 0 after 76324.921427 seconds from start
>
> followed by this line:
> NOTICE: [pool user] child 29909 started
>
> but I don't think that's relevant.

Is that just hitting your max_requests limit and exiting? In which
case that would be normal operation.

> Any idea why this is happening and how to prevent it?

I am running php5-fpm on several servers. I as a matter of course set
up my own custom pool configurations with a different socket name.
But always in /var/run. I have not had any problems with with the
socket disappearing. One is on Jessie 8, one on Sid, the rest on
Wheezy 7. I haven't seen any endemic problem assocatied with php5-fpm.

The first thing I would try is to move the socket location out of
/dev/shm (which is symlinked to /run/shm in Jessie 8) and see if the
behavior stops. If so then it must be related to that location. If
not then it rules that out as a suspect.

I would tend to also set up a monitor that would run very often, once
every minute wouldn't be too often, to check for the presence of the
socket file. If it dissappears then have it notify me immediately so
I could look to see what else happened around that same time period.
Because if it is getting removed I would think that something must be
removing it and that something will hopefully leave an audit trail.

Bob

[Santiago Vila]

On Wed, Jun 17, 2015 at 11:30:33PM +0200, Proxy One wrote:
> I installed Jessie on my new server few days ago and moved website that run

> previously on Centos 5. I'm using Apache and PHP-FPM. [...]
> [...]
> <IfModule mod_fastcgi.c>

If you are using jessie, your Apache version is >= 2.4, so you could
ditch fastcgi and use mod_proxy and mod_proxy_fcgi instead:

https://wiki.apache.org/httpd/PHP-FPM

This is my very short recipe for that:

a2enmod proxy
a2enmod proxy_fcgi

Edit /etc/apache2/sites-available/yourdomain.conf and include
something like this:

<FilesMatch \.php$>
SetHandler "proxy:unix:/var/run/php5-fpm.sock|fcgi://localhost/"
</FilesMatch>
DirectoryIndex index.php index.html index.htm

Then "service apache2 restart".


[ It worked for me, but then I switched to nginx shortly after that ].

--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: https://lists.debian.org/20150617220...@cantor.unex.es

[Proxy One]

Hi Bob!

On 2015-Jun-17 16:19, Bob Proulx wrote:
> Proxy One wrote:
> > I installed Jessie on my new server few days ago and moved website
> > that run previously on Centos 5.
>
> Welcome! :-)

Thanks! I'm actually using Debian few last years. :)

>
> > I'm using Apache and PHP-FPM.
>
> I have become an Nginx + php5-fpm advocate in recent years. If you
> decide you would like to give it a try post something and I will show
> my configurations for it. (Not that I am a master of it by any means.
> Just a daily user of it.)

I'm using Nginx myself for a few other machines, but this one must stay
with Apache. There are lot of rewrite rules, that I couldn't adapt to
Nginx.

> > Currently, there is only one website on that server, but I still
> > configured pool for it. What's happening is that, after some time,
> > at least once a day, socket that should be listening for that user
> > disappears, ...
> >...
> > Looking at /dev/shm/ directory, user-php.sock is really missing, but
> > default php5-fpm.sock is still there.
>
> Odd!
>
> > I have this block in VirtualHost section for that website:
> > <IfModule mod_fastcgi.c>
> > Alias /php5-fcgi /dev/shm/pdfconve-php.fcgi
> > </IfModule>
> >...
> > And there is also
> > FastCGIExternalServer /dev/shm/user-php.fcgi -socket /dev/shm/user-php.sock -pass-header Authorization -flush -appConnTimeout 2 -idle-timeout 60
> > in apache conf.
>
> Hmm... Is that right? It is working for you so it must be. And
> since the disappearing fpm socket shouldn't be related I am going to
> ignore my ignorance of the above here.

I put that line in conf-enabled/php5-fpm.conf. Otherwise, Apache
wouldn't know it should go to /dev/shm/user-php.sock for fcgi requests.

> > Pool configured:
> >
> > [user]
> > listen = /dev/shm/user-php.sock
> > listen.owner = user
> > listen.group = www-data
> > listen.mode = 0660
> > user = user
> > group = user
> > pm = dynamic
> > pm.max_children = 40
> > pm.max_requests = 2048
> > pm.start_servers = 30
> > pm.min_spare_servers = 10
> > pm.max_spare_servers = 35
> > request_terminate_timeout = 305
>
> Why use /dev/shm/user-php.sock as the socket path? The Jessie-style
> location would be in /var/run/user-php.sock AFAICS. (I don't see how
> that would be related to your socket dissappearing.)

I used that path on Centos machines and it worked. I saw that Debian
uses different default path, but figured it should work either way. I
also used /dev/shm so it's created in RAM (tmpfs). I figured it would be
quicker this way.

> > There is nothing interesting in php-fpm error log. There are some errors like this:
> > NOTICE: [pool user] child 32719 exited with code 0 after 76324.921427 seconds from start
> >
> > followed by this line:
> > NOTICE: [pool user] child 29909 started
> >
> > but I don't think that's relevant.
>
> Is that just hitting your max_requests limit and exiting? In which
> case that would be normal operation.
>
> > Any idea why this is happening and how to prevent it?
>
> I am running php5-fpm on several servers. I as a matter of course set
> up my own custom pool configurations with a different socket name.
> But always in /var/run. I have not had any problems with with the
> socket disappearing. One is on Jessie 8, one on Sid, the rest on
> Wheezy 7. I haven't seen any endemic problem assocatied with php5-fpm.
>
> The first thing I would try is to move the socket location out of
> /dev/shm (which is symlinked to /run/shm in Jessie 8) and see if the
> behavior stops. If so then it must be related to that location. If
> not then it rules that out as a suspect.

I changed path, so all socket are in /var/run now. Will see if that
solves the problem.

> I would tend to also set up a monitor that would run very often, once
> every minute wouldn't be too often, to check for the presence of the
> socket file. If it dissappears then have it notify me immediately so
> I could look to see what else happened around that same time period.
> Because if it is getting removed I would think that something must be
> removing it and that something will hopefully leave an audit trail.
>
> Bob

Good idea. I usually use Zabbix to monitor websites uptime, but PCU on
that server just died.

Thanks for your help, Bob! Much appreciated.

--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: https://lists.debian.org/2015061812...@gvozdika.example.net

[Proxy One]

Hi Santiago!

On 2015-Jun-18 00:03, Santiago Vila wrote:
> On Wed, Jun 17, 2015 at 11:30:33PM +0200, Proxy One wrote:
> > I installed Jessie on my new server few days ago and moved website that run
> > previously on Centos 5. I'm using Apache and PHP-FPM. [...]
> > [...]
> > <IfModule mod_fastcgi.c>
>
> If you are using jessie, your Apache version is >= 2.4, so you could
> ditch fastcgi and use mod_proxy and mod_proxy_fcgi instead:
>
> https://wiki.apache.org/httpd/PHP-FPM
>
> This is my very short recipe for that:
>
> a2enmod proxy
> a2enmod proxy_fcgi
>
> Edit /etc/apache2/sites-available/yourdomain.conf and include
> something like this:
>
> <FilesMatch \.php$>
> SetHandler "proxy:unix:/var/run/php5-fpm.sock|fcgi://localhost/"
> </FilesMatch>
> DirectoryIndex index.php index.html index.htm
>
> Then "service apache2 restart".
>
>
> [ It worked for me, but then I switched to nginx shortly after that ].

I just switched to proxy_fcgi and it's working for me too. To be honest,
it was strange to find that libapache2-mod-fastcgi was in non-free
reporistory, but I just shrugged and installed it, as it was the way of
using php-fpm with Apache I used earlier. Good to know I can remove
libapache2-mod-fastcgi now.

Thanks!

--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: https://lists.debian.org/2015061813...@gvozdika.example.net

[Bob Proulx]

Proxy One wrote:
> > Why use /dev/shm/user-php.sock as the socket path? The Jessie-style
> > location would be in /var/run/user-php.sock AFAICS. (I don't see how
> > that would be related to your socket dissappearing.)
>
> I used that path on Centos machines and it worked. I saw that Debian
> uses different default path, but figured it should work either way. I
> also used /dev/shm so it's created in RAM (tmpfs). I figured it would be
> quicker this way.

On Debian (and I must say Jessie too to bracket the time since this
has been changing in recent history) /run is a tmpfs just like
/dev/shm and /var/run is a symlink that points to /run. Using
/var/run for this will be just as fast as /dev/shm. AFAIK /dev/shm is
the domain for use by libc.

It has also become very popular to use /tmp as a tmpfs too. However
that conflicts with various historical usage. Therefore whether you
use /tmp as a tmpfs is really more of a local system configuration
rather than as a blanket statement.

Bob

[Proxy One]

On 2015-Jun-20 19:42, Bob Proulx wrote:
> Proxy One wrote:
> > > Why use /dev/shm/user-php.sock as the socket path? The Jessie-style
> > > location would be in /var/run/user-php.sock AFAICS. (I don't see how
> > > that would be related to your socket dissappearing.)
> >
> > I used that path on Centos machines and it worked. I saw that Debian
> > uses different default path, but figured it should work either way. I
> > also used /dev/shm so it's created in RAM (tmpfs). I figured it would be
> > quicker this way.
>
> On Debian (and I must say Jessie too to bracket the time since this
> has been changing in recent history) /run is a tmpfs just like
> /dev/shm and /var/run is a symlink that points to /run. Using
> /var/run for this will be just as fast as /dev/shm. AFAIK /dev/shm is
> the domain for use by libc.

You're right, that's also tmpfs, so no speed lost switching to /var/run.

> It has also become very popular to use /tmp as a tmpfs too. However
> that conflicts with various historical usage. Therefore whether you
> use /tmp as a tmpfs is really more of a local system configuration
> rather than as a blanket statement.
>
> Bob

Since changing sockets location, and switching to proxy_fcgi, everything
is working smoothly. Don't know what did the trick, but thanks to both
of you, Bob and Santiago!

--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: https://lists.debian.org/2015062116...@angelina.example.com