returnszero expression (with useshell) returning wrong result

[Håkan Olsson]

Hi,

for some reason the following command:

  classes:
      "running_supervisord"      expression => returnszero("/usr/bin/pgrep -f '/usr/bin/python .*/supervisord -c.*' >/dev/null", "useshell");

always returns true (zero), even on machines where no such process is running (e.g. not even any other python process running).  From verbose output:

2015-05-22T12:34:53+0200  verbose: /default/configure_supervisord/classes: Evaluating promise 'running_supervisord'
2015-05-22T12:34:53+0200  verbose: /default/configure_supervisord/classes: returnszero ran '/usr/bin/pgrep -f "/usr/bin/python .*/supervisord -c.*" >/dev/null' successfully and it returned zero

Copy-n-pasting the exact same command to commandline (sh/bash) properly sets exit code to 1 (false).

I've changed the ' chars to \" in the command, same result.

Obviously I'm doing something wrong, or the way returnszero() passes the string to the shell is bad, but I cannot see what.  As it is this gives me a false positive at the moment, and since I have these types of checks in quite a few places that's a bit worrisome.   About all I can think of is CFengine creating a shortlived child process that itself matches the pgrep expression, but that seems far-fetched (hopefully).

Any pointers?   This is 3.6.4 on RHEL6 & 7, btw.

Thanks,
  Håkan.

[Bas van der Vlies]

sorry this was accidentally send.

> On 22 mei 2015, at 13:22, Bas van der Vlies <bas.van...@surfsara.nl> wrote:
>
> I just tested it and you have to leave >/dev/null

>> --
>> You received this message because you are subscribed to the Google Groups "help-cfengine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
>> To post to this group, send email to help-c...@googlegroups.com.
>> Visit this group at http://groups.google.com/group/help-cfengine.
>> For more options, visit https://groups.google.com/d/optout.
>
> ---
> Bas van der Vlies
> | Operations, Support & Development | SURFsara | Science Park 140 | 1098 XG Amsterdam
> | T +31 (0) 20 800 1300 | bas.van...@surfsara.nl | www.surfsara.nl |
>
>
>

---
Bas van der Vlies
| Operations, Support & Development | SURFsara | Science Park 140 | 1098 XG Amsterdam
| T +31 (0) 20 800 1300 | bas.van...@surfsara.nl | www.surfsara.nl |

[Bas van der Vlies]

So if you leave out it works, but the side effect is that the matches are displayed:
{{{
root# cf-agent -KI -f ./test.cf
22570
22570
R: running_supervisord
}}}

I tested it with 3.6.5. For me this is a bug.

[Neil Watson]

Use a processes promise and process_select attribute instead. It's
cheaper and more portable than resturnszero

...
vars:
"regex"
comment => "Regex that should never match",
string => "/usr/bin/pgrep -f '/usr/bin/perl .*/nevermatched -c.*' > /dev/null";

processes:
"nevermatched"
process_select => by_name( "${regex}" ),
restart_class => "command_not_found";
}

body process_select by_name(x)
{
command => "${x}";
process_result => "command";
}

--
Neil H Watson
Sr. Partner, Architecture and Infrastructure
CFEngine reporting: https://github.com/evolvethinking/delta_reporting
CFEngine policy: https://github.com/evolvethinking/evolve_cfengine_freelib
CFEngine and vim: https://github.com/neilhwatson/vim_cf3
CFEngine support: http://evolvethinking.com

[Neil Watson]

On Fri, May 22, 2015 at 08:00:08AM -0400, Neil Watson wrote:
>Use a processes promise and process_select attribute instead. It's
>cheaper and more portable than resturnszero
>
>...
> vars:
> "regex" comment => "Regex that should never match",
> string => "/usr/bin/pgrep -f '/usr/bin/perl .*/nevermatched -c.*' > /dev/null";

Actually, this should be:
string => "/usr/bin/perl .*/nevermatched -c.*";

[Håkan Olsson]

Thanks for the tip, after some experimentation this is what appears to do what I want (set the class if the process is running):

 processes:
      "/usr/bin/pyhon .*/supervisord -c.*'"
                                         classes => if_ok("myclass"),
                                         comment => "Check if supervisord process is running.";

reports:
   myclass::
      "supervisord proc match on host $(sys.uqhost)";

I had expected something like 'process_count => any_count("myclass")' to work, but from what I can tell it was never even triggered here.

[Håkan Olsson]

Ah, no. :(

If there is no process running on the host, the log emits this:
2015-05-25T12:49:22+0200  verbose: /default/main_jeppesen/methods/'supervisord_svc'/default/configure_supervisord: Observe process table with /bin/ps -eo user,pid,ppid,pgid,pcpu,pmem,vsz,ni,rss:9,nlwp,stime,etime,time,args
2015-05-25T12:49:22+0200  verbose: /default/main_jeppesen/methods/'supervisord_svc'/default/configure_supervisord/processes: Evaluating promise '/usr/bin/pyhon .*/supervisord -c.*''
2015-05-25T12:49:22+0200  verbose: /default/main_jeppesen/methods/'supervisord_svc'/default/configure_supervisord/processes/'/usr/bin/pyhon .:/supervisord -c.:''[0]: No restart promised for /usr/bin/pyhon .*/supervisord -c.*'
2015-05-25T12:49:22+0200  verbose: /default/main_jeppesen/methods/'supervisord_svc'/default/configure_supervisord/processes/'/usr/bin/pyhon .:/supervisord -c.:''[0]: Defining promise result class 'myclass'

Ok, more experimentation...

[Neil Watson]

On Mon, May 25, 2015 at 03:34:07AM -0700, Håkan Olsson wrote:
>
>
> Thanks for the tip, after some experimentation this is what appears to
> do what I want (set the class if the process is running):
>
>  processes:
>       "/usr/bin/pyhon .*/supervisord -c.*'"
>                                          classes => if_ok("myclass"),
>                                          comment => "Check if
> supervisord process is running.";

Look at my example again. I use the attributes restart_class and
process_select, not classes => if_ok. That will not work.

[Håkan Olsson]

It may be I don't understand the details of your example, but it only checks for the *absence* of a process, right?  "set class X if process Y is not found". 

Here I'm interested in the opposite; "set class X if process Y is found", i.e. the presence of it. As far as I can tell restart_class is not useful here.  (also: with restart_class => "X" and e.g. using "!X" (negated) is likely to be problematic considering how relatively late processes: promises are evaluated)

I've tried some "likely candidates", such as

  processes:
      "/usr/bin/pyhon .*/supervisord -c.*"
                               process_count => any_count("proc_is_running"),
                                     comment => "Check if our supervisord process is running.";

and e.g.

  processes:
      "nevermatch"
                              process_select => by_name("/usr/bin/pyhon .*/supervisord -c.*'"),
                               process_count => any_count("proc_is_running"),
                                        comment => "Check if our supervisord process is running.";

but none of them so far appear capable to set the class when a process matching that regex exists in the system.

[Neil Watson]

Then try negating the process select:

process_result => "!command";

[Håkan Olsson]

Unfortunately that gives me false positives on machines where the process does not exist (possibly due to restart_class, regardless of process_select).

To me it seems the processes: promisetype is missing a simple way to set a class when a process _is_ running.

I'll go back to the returnszero() method in the meantime, something without redirection should still work until https://dev.cfengine.com/issues/7217 is solved.

[Aleksey Tsalolikhin]

On Tue, May 26, 2015 at 5:38 AM, Håkan Olsson <hakan.o...@gmail.com> wrote:

To me it seems the processes: promisetype is missing a simple way to set a class when a process _is_ running.

Indeed it is. I asked for this feature in https://dev.cfengine.com/issues/1940  

However I did not have a real-world use case to back up my request.  It seems that you do, perhaps you would care to read the ticket and if you feel like it, detail your situation and need?

We *can* set a class today when a process is running, but as you can see it is not simple.   I'd like to see a simple way to do it.

Best,

Aleksey

[Håkan Olsson]

I'll check that bugreport.

For now I opted to go with a 'classic' approach (unfortunately):

classes:
  "myclass" expression => returnszero("[ 1 == `ps auxw | grep '/usr/bin/python .*/supervisord -c.*' | grep -v grep | wc -l` ]", "useshell");

(As it turns out, using returnszero("somecommand", "useshell"); causes cf-agent for fork off a "sh -c 'somecommand'" subprocess, so using pgrep always generated one extra match due to this extra 'sh' process.)