Defining a class with returnszero containing grep [f]oo

[tentaculartentacle]

Hi,

If I define a class like so:
  "yes" expression => returnszero("ps aux | grep [f]oo","useshell");
It works when running cf-agent manually. It doesn't work during scheduled runs ("yes" is always defined regardless of whether process foo is running or not).
If I use the traditional "grep foo | grep -v grep" instead, it works as expected.

Seems very odd. What am I missing? This is version 3.6.1.

[Neil Watson]

What's your end goal? CFEngine processes promises will match classes for
your. Also, short cut your command by using pgrep instead.


--
Neil H Watson
CFEngine reporting: https://github.com/neilhwatson/delta_reporting
CFEngine policy: https://github.com/neilhwatson/evolve_cfengine_freelib
CFEngine and vim: https://github.com/neilhwatson/vim_cf3

[Aleksey Tsalolikhin]

 You could try:

 "yes" expression => returnszero("/bin/ps aux | /bin/grep [f]oo","useshell");

But it would use less system resources and be more CFEngine-y to set the class using a processes promise using the any_count body from the Standard Library:

bundle agent main {

 processes:

 

       "vim"

 

         process_count   => any_count("got_a_match");

reports:

  got_a_match::

   "got a match!  there is a 'vim' process running";

}

body common control { inputs => { "$(sys.libdir)/../stdlib.cf"}; }

You might have to change how you load the Standard Library for your version of CFEngine (3.6.1).  I don't remember at what point we split off the 3.6 and 3.7 policies into separate subdirectories, but it looks like we are re-unifying them (see my other post, not really relevant to your question, just letting you know if you copy and paste my example it might now work due to path to standard library.)

The added advantage of doing it all in CFEngine is that makes convergence more likely.  Who knows what will happen when you run an external command... I mean, things go out of CFEngine's control at that point, so we can't guarantee convergence.

More complex examples of process detection using built-in facilities of CFEngine:

Listing 13.31: 520-070-Process_Selection-0610-Select_by_several_things.cf

 1 ########################################################
 2 #
 3 # Simple test processes
 4 #
 5 ########################################################
 6 
 7 
 8 bundle agent main
 9 
10 {
11   processes:
12 
13       ".*"
14 
15         process_count   => anyprocs,
16         process_select  => proc_finder;
17 
18 
19   reports:
20 
21     any_procs::
22 
23       "Found processes in range";
24 
25     in_range::
26       "Found no processes in range";
27 
28 }
29 
30 ########################################################
31 
32 body process_select proc_finder
33 
34 {
35 
36         command => "vim .*";
37         # (Anchored) regular expression matching the CMD field
38 
39         stime_range => irange(ago(1,0,0,0,0,0),ago(0,0,0,0,0,10));
40         # Processes started between 1 year and 10 seconds ago
41 
42         process_owner => { "root" };
43         # List of regexes matching the user of a process
44 
45         process_result => "stime&command&process_owner";
46 
47 }
48 
49 ########################################################
50 
51 body process_count anyprocs
52 
53 {
54         match_range => "0,0";
55       # Integer range for acceptable number of matches for this process
56       # (In this case, one or more processes
57 
58         out_of_range_define => { "any_procs" };
59       # List of classes to define if the matches are out of range
60 
61         in_range_define => { "in_range" };
62       # List of classes to define if the matches are in range.
63       # We should never have a process that has a count of 0.
64 
65 }

The above is from www.cfenginetutorial.org

Here is another example, from https://docs.cfengine.com/docs/master/reference-promise-types-processes.html

bundle agent main { processes: "init" process_count => any_count("booted_over_1_day_ago"), process_select => days_older_than(1), comment => "Define a class indicating we found an init process running for more than 1 day."; reports: booted_over_1_day_ago:: "This system was booted over 1 days ago since there is an init process that is older than 1 day."; !booted_over_1_day_ago:: "This system has been rebooted recently as the init process has been running for less than a day"; } body process_count any_count(cl) { match_range => "0,0"; out_of_range_define => { "$(cl)" }; } body process_select days_older_than(d) { stime_range => irange(ago(0,0,"$(d)",0,0,0),now); process_result => "!stime"; }

I should also mention, though it won't help you now, that starting with CFEngine 3.9 there is a processexists() function: https://docs.cfengine.com/docs/master/reference-functions-processexists.html

Best,

Aleksey Tsalolikhin

-- 

Looking for CFEngine training?   Email trai...@verticalsysadmin.com

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

ᐧ

ᐧ

[tentaculartentacle]

Thanks.
A processes promise works nicely in this case, I'll use that.
But I was actually more concerned with how returnszero could return true while running a command that definitely returns 1. Because there isn't always a built-in promise I can use instead (e.g. version_is_55 =  /some/command --version | grep 5.5). I could really break stuff if I can't be really confident on the returnzero behaving as expected.

[Aleksey Tsalolikhin]

You are welcome.

You had said,  It doesn't work during scheduled runs ("yes" is always defined regardless of whether process foo is running or not).

Did you try putting in the full path?  I would be surprised if your code worked at all, as I recall CFEngine refuses to guess about path to executables -- your infrastructure is too important to be guessing about it.  You normally have to specify the full path to binaries.  That's a security point as well.

Best,

Aleksey

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

--

Aleksey Tsalolikhin

Founder and Chief Trainer

Vertical Sysadmin, Inc.

+1-323-393-0779

ᐧ

[Nick Anderson]

On 07/13/2016 08:57 AM, Aleksey Tsalolikhin wrote:
> Did you try putting in the full path? I would be surprised if your code
> worked at all, as I recall CFEngine refuses to guess about path to
> executables -- your infrastructure is too important to be guessing about
> it. You normally have to specify the full path to binaries. That's a
> security point as well.

There is a feature request open to implement PATH support for the agent.


https://tracker.mender.io/browse/CFE-1804

[Nick Anderson]

On 07/12/2016 12:15 PM, Aleksey Tsalolikhin wrote:
> But it would use less system resources and be more CFEngine-y to set the
> class using a processes promise using the any_count body from the
> Standard Library

In 3.9+ you can also enjoy findprocesses() and processexists()


https://docs.cfengine.com/docs/master/reference-functions-findprocesses.html
https://docs.cfengine.com/docs/master/reference-functions-processexists.html

[tentaculartentacle]

Same result with full paths. It would be fine for it to fail without full paths, but then I would expect "returnszero" to not return zero!
What does work as expected is changing the grep [f]oo to "grep foo | grep -v grep". That works with or without paths (which I presume is due to "useshell").

I'm quite worried why 1) it behaves differently when running cf-agent on the command line vs scheduled and 2) why something that definitely returns false is considered as returning zero.

[Aleksey Tsalolikhin]

Right! Thanks for getting to the bottom of it.  

I might suggest looking at verbose or even debug output from command line vs scheduled runs to understand the difference; I suspect this would shed light on why returnszero() is behaving the way it does.

ᐧ

[Erik S]

On Wednesday, July 13, 2016 at 10:35:07 AM UTC-5, tentaculartentacle wrote:

Same result with full paths. It would be fine for it to fail without full paths, but then I would expect "returnszero" to not return zero!
What does work as expected is changing the grep [f]oo to "grep foo | grep -v grep". That works with or without paths (which I presume is due to "useshell").

I'm quite worried why 1) it behaves differently when running cf-agent on the command line vs scheduled and 2) why something that definitely returns false is considered as returning zero.

Speculating here: it's possible your grep [f]oo is being submitted to the CFEngine subshell as grep foo when run via the normal mechanism (as opposed to ad-hoc from the command line). If so, it would match based on the presence of grep foo in the process table.

Try single-quoting the expression you hand to grep(1), as in: "grep -Eq '[f]oo'"

For instance, I just did some testing and this behaves as expected:

###############################################################################
# Contrived returnszero() test
###############################################################################

bundle agent b_test
{
meta:
    "tags" slist => { "autorun" };

classes:
    "got_zero_exit" expression =>
        returnszero(
            "/bin/ps -ef | grep -Eq '\bxxrsyslogd\b'",
            "useshell"
        );

reports:
    got_zero_exit::
    "Pipeline reported zero exit";

    !got_zero_exit::
    "Pipeline reported NON zero exit";
}

Note my quoting of the expression (using single quotes), and bear in mind that xxrsyslogd does not exist in the process table.

(All that said, you're going the right direction by using the "proccesses" promise. And, as mentioned, pgrep(1) would be cleaner in this particular case because no pipeline would be necessary. But I still see your point about being able to evaluate arbitrary commands using returnszero().)

[Aleksey Tsalolikhin]

Nice, Erik!   This is the sort of thing I was suspecting was going on.  Thanks, mate.

ᐧ

[Gregory Matthews]

On 13/07/16 18:33, Erik S wrote:
> (All that said, you're going the right direction by using the
> "proccesses" promise. And, as mentioned, pgrep(1) would be cleaner in
> this particular case because no pipeline would be necessary. But I still
> see your point about being able to evaluate arbitrary commands using
> returnszero().)

or "ps -fC rsyslogd"

GREG

--
Greg Matthews 01235 778658
Scientific Computing Group Leader
Diamond Light Source Ltd. OXON UK

--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

[mike.w...@verticalsysadmin.com]

As another possibility: You are stuffing the unquoted (from the shell's perspective) string "[f]oo" into the arguments to grep.  Before grep sees that, the shell will expand it as a shell glob.  If you have a file called "foo" in your current directory, it will be expanded to "foo" rather than "[f]oo" and that's what "grep" will see.  If you don't have such a file, then the result will depend on your shell's default setting for "nullglob": if nullglob is set, then the grep command will receive no arguments at all (as the glob couldn't be expanded to a filename); if nullglob is not set, and there is no file called "foo" in your current directory, only then will "grep" receive the exact argument "[f]oo".

Quoting is needed a lot in shells.  :)  Also see http://mywiki.wooledge.org/BashPitfalls

But of course, the right action here is a processes promise anyway.

--Mike Weilgart