SIGPIPE is blocked for cf-execd and descendents resulting in premature subprocess death i.c.w. SSL

[Thomas Hood]

[Re-posting with a more useful subject heading and with additional detail.]

At my employer we have a problem with CFEngine which follows the following scenario.

* cf-execd is for some reason running with SIGPIPE blocked.

* CFEngine restarts sshd (and other server processes) which consequently also have SIGPIPE blocked.

* A user logs in to the machine via sshd and gets a bash shell which also has SIGPIPE blocked.

* The user enters a sudo or strace command and presses Enter.

* Bash creates a child process which starts by looking stuff up via nss_ldap which results in an LDAPS query using OpenSSL.

* In the course of its normal operation this query triggers a SIGPIPE which can't be delivered (because SIGPIPE is blocked) and so becomes pending.

* In its initialisation phase sudo or strace unblocks SIGPIPE.

* The SIGPIPE is delivered to sudo or strace which immediately exits.

Perhaps sudo and strace should clear signals before unblocking them, not sure. Regardless, CFEngine should not be spawning children that have SIGPIPE blocked. CFEngine spawns children that have SIGPIPE blocked because it has SIGPIPE blocked itself. I don't know how it comes to be the case that cf-execd is running with SIGPIPE blocked. If CFEngine is responsible for this then I would be inclined to think that this is a bug.

The following workaround has been adopted for this problem.

cron.d/fixsigblock.cron:

    */15 * * * * root /usr/local/sbin/fixsigblock.sh

The script /usr/local/sbin/fixsigblock.sh checks to see if cf-execd is running with the SIGPIPE bit set in its sigblk mask as read from the line in /proc/<pid>/status starting with 'SigBlk'. If so then it restarts the CFEngine daemons using the /etc/init.d/cfengine3 initscript. Because cron was started at boot with SIGPIPE unblocked, its children also have SIGPIPE unblocked.

Looking at the 3.5.1 and at the current CFEngine source code I see that SIGPIPE gets blocked in client_code.c. I don't see it being unblocked anywhere.

    static AgentConnection *ServerConnection(const char *server, FileCopy fc, int *err, int s)

    {

      AgentConnection *conn = NULL;

      int ret;

      *err = 0;

    #if !defined(__MINGW32__)

      signal(SIGPIPE, SIG_IGN);

    #endif /* !__MINGW32__ */

    #if !defined(__MINGW32__)

      sigset_t signal_mask;

      sigemptyset(&signal_mask);

      sigaddset(&signal_mask, SIGPIPE);

      pthread_sigmask(SIG_BLOCK, &signal_mask, NULL);

    #endif

Is there a bug here?

[bishwa.shrestha]

There's pull-request pending from Loic Pefferkorn that might be related: https://github.com/cfengine/core/pull/1319

Corresponding bug-request:

https://cfengine.com/dev/issues/2971

Could you please verify if this fixes your issue?

-bishwa

[Thomas Hood]

On Friday, January 31, 2014 2:56:45 PM UTC+1, bishwa.shrestha wrote:

There's pull-request pending from Loic Pefferkorn that might be related: https://github.com/cfengine/core/pull/1319

Corresponding bug-request:

https://cfengine.com/dev/issues/2971

Could you please verify if this fixes your issue?

Thanks for the suggestion, but this is not exactly the same issue. 

The cause of issue #2971 seems to be that the signal handler for SIGPIPE is set to SIG_IGN in the children of cf-agent which causes them all to ignore SIGPIPEs when they receive them. The malfunction is their failure to exit on receipt of SIGPIPE. They don't die when they should.

The cause of my issue is that SIGPIPE is *blocked* in the children of cf-execd. The malfunction is that processes exit at some later time when SIGPIPE is unblocked. They die when they shouldn't.

[Steven Kreuzer]

Since I just encountered the same issue on 3.6.4 I assume its safe to
say this has not been fixed?

> --
> You received this message because you are subscribed to the Google Groups
> "help-cfengine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to help-cfengin...@googlegroups.com.
> To post to this group, send email to help-c...@googlegroups.com.
> Visit this group at http://groups.google.com/group/help-cfengine.
> For more options, visit https://groups.google.com/groups/opt_out.

[Dominic D'Apice]

Hi,

We had experimented this issue in 3.4,  we need to schedule the following script in crontab

---------------------------------------------------------------------------

#!/bin/bash

if [[ -n "$(pgrep cf-execd)" ]]; then

   egrep -q "SigBlk:\s+0000000000001000" /proc/`pgrep cf-execd`/status;

   if [[ ${?} = 0 ]] ; then

     service cfengine3 restart;

   fi

fi

---------------------------------------------------------------------------

Dominic

[Dimitrios Apostolou]

The most relevant bug report I managed to find is https://dev.cfengine.com/issues/2971 and seemingly it's fixed in 3.6.x (possible after 3.6.4).

Dimitris

--

You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/help-cfengine.

For more options, visit https://groups.google.com/d/optout.