set_line_based(...): bug or feature?
David Lee
Historically we have maintained entries in "/etc/services" with an slist of entries:
"svcs" slist => {
"hpss_d_mvr1 65501/tcp",
"hpss_d_mvr2 65503/tcp",
...
};
then:
edit_line => insert_lines("$(svcs)")
That was OK, although it left open the possibility of things going wrong if the spacing within one of those "slist" lines changed.
So I have been switching over to use "set_line_based(...)", thus:
"svc[hpss_d_mvr1]" string => "65501/tcp";
"svc[hpss_d_mvr2]" string => "65503/tcp";
...
then:
# edit_line => set_line_based("$(this.bundle).svc", " ", "\s+", ".*", "\s*#\s*"),
edit_line => set_line_based("$(this.bundle).svc", " ", "\s+", ".*", "\s*#\s*")
The first attempt (commented out) has a multi-space separator; it is nice because it tries to imitate the prettiness of "/etc/services". BUT... it gives loud warnings every time about non-convergence.
If I reduce that argument to a single-space separator, everything works well and quietly, although the entries in "/etc/services" are less pretty.
Basic question: is that multi-space warning a bug or a feature?
Next question: is there a nice way to get that multi-space behaviour.
-- David Lee
-- ECMWF
Nick Anderson
David Lee <davi...@ecmwf.int> writes:
> Basic question: is that multi-space warning a bug or a feature?
>
> Next question: is there a nice way to get that multi-space behaviour.
Can you paste the WARNING?
FWIW I don't seem to get errors from this on 3.11
bundle agent main { vars: "map[hpss_d_mvr1]" string => "65501/tcp"; # Include padding in the value itself "map[hpss_d_mvr2]" string => " 65503/tcp"; files: "/tmp/services" create => "true"; # Use more than one space in the second field for the minimum number of # spaces between columns "/tmp/services" edit_line => set_line_based( "$(this.bundle).map", " ", "\s+", ".*", "" ); reports: "$(sys.cf_version)"; "/tmp/services" printfile => cat("/tmp/services"); }
R: 3.11.0 R: /tmp/services R: hpss_d_mvr2 65503/tcp R: hpss_d_mvr1 65501/tcp
–
Nick Anderson
Doer of things, CFEngine
David Lee
2. That inclusion of padding in the value looks like a useful possibility, although somewhat non-obvious!
3. A sample set of warnings:
info: Promised replacement 'hpss_d_mvr3 65505/tcp' for pattern '^\s*(hpss_d_mvr3\s+(?!65505\/tcp$).*|hpss_d_mvr3)$' is not properly convergent while editing '/etc/services'
info: Because the regular expression '^\s*(hpss_d_mvr3\s+(?!65505\/tcp$).*|hpss_d_mvr3)$' still matches the end-state replacement string 'hpss_d_mvr3 65505/tcp'
info: Promise belongs to bundle 'set_line_based' in file '/var/cfengine/inputs/lib/files.cf' near line 804
info: Comment is 'Correct the value 'hpss_d_mvr3''
All the best.
-- David
Nick Anderson
David Lee <davi...@ecmwf.int> writes:
> 1. Remind me: what's the simplest, easiest, no-fuss way to wrap up a
> standalone, test bundle such as your "bundle main {…}" above into
> something runnable? So that I can do something like "cf-agent -KI …" on
> that bundle, and still have it include the library that defines
> "setlinebased"?
I use [[https://github.com/nickanderson/ob-cfengine3][ob-cfengine3]] with org-mode inside of spacemacs for most of my self
contained examples and when I start prototyping new policy.
When I have a cfengine3 SRC block and I execute it body file control is
magically inserted to load the stdlib.
bundle agent main { reports: "$(this.promise_filename):" printfile => cat( $(this.promise_filename) ); }
You can see in the output here the body file control that was automatically
inserted.
R: /home/nickanderson/cfengine3-30374PIb:
R: body file control{ inputs => { '$(sys.libdir)/stdlib.cf' };}
R: bundle agent main
R: {
R: reports:
R: "$(this.promise_filename):" printfile => cat( $(this.promise_filename) );
R: }
In my opinion using body file control is the easiest way to include another
policy file.
> 2. That inclusion of padding in the value looks like a useful possibility,
> although somewhat non-obvious!
:)
> 3. A sample set of warnings:
>
> info: Promised replacement 'hpssdmvr3 65505/tcp' for pattern
>> '^(hpssdmvr3+(?!65505\/tcp
' is not properly
>> convergent while editing '/etc/services'
>> info: Because the regular expression
>> '^(hpssdmvr3+(?!65505\/tcp
' still matches the
>> end-state replacement string 'hpssdmvr3 65505/tcp'
>> info: Promise belongs to bundle 'setlinebased' in file
>> '/var/cfengine/inputs/lib/files.cf' near line 804
>> info: Comment is 'Correct the value 'hpssdmvr3''
Oh my snippet ran without --inform that's why I didn't see those messages.
Probably we could avoid that annoying warning by tweaking the regular expression
for the replacement.
This change seems to do it.
modified lib/files.cf @@ -818,7 +818,7 @@ bundle edit_line set_line_based(v, sep, bp, kp, cp) # If the line is there with the wrong value, replace with # the correct value - "^\s*($(i)$(bp)(?!$(ev[$(i)])$).*|$(i))$" + "^\s*(?!$(i)$(bp)$(ev[$(i)])$.*|$(i))$" comment => "Correct the value '$(i)'", replace_with => value("$(i)$(sep)$($(v)[$(i)])"), classes => results("bundle", "replace_attempted_$(ci[$(i)])");
Want to give it a try? Perhaps open a Pull Request to masterfiles with the
change if it works well for you.
Mike Weilgart
—Mike Weilgart
Vertical Sysadmin, Inc.
>> '^(hpssdmvr3+(?!65505\/tcp<latex30374Dxz_a589ffda70233b59f1ebc9435c70dd84d0c1e4ab.png>' is not properly
>> convergent while editing '/etc/services'
>> info: Because the regular expression
>> '^(hpssdmvr3+(?!65505\/tcp<latex30374cZV_a589ffda70233b59f1ebc9435c70dd84d0c1e4ab.png>' still matches the
>> end-state replacement string 'hpssdmvr3 65505/tcp'
>> info: Promise belongs to bundle 'setlinebased' in file
>> '/var/cfengine/inputs/lib/files.cf' near line 804
>> info: Comment is 'Correct the value 'hpssdmvr3''
Oh my snippet ran without --inform that's why I didn't see those messages.
Probably we could avoid that annoying warning by tweaking the regular expression
for the replacement.
This change seems to do it.
modified lib/files.cf @@ -818,7 +818,7 @@ bundle edit_line set_line_based(v, sep, bp, kp, cp) # If the line is there with the wrong value, replace with # the correct value - "^\s*($(i)$(bp)(?!$(ev[$(i)])$).*|$(i))$" + "^\s*(?!$(i)$(bp)$(ev[$(i)])$.*|$(i))$" comment => "Correct the value '$(i)'", replace_with => value("$(i)$(sep)$($(v)[$(i)])"), classes => results("bundle", "replace_attempted_$(ci[$(i)])");
Want to give it a try? Perhaps open a Pull Request to masterfiles with the
change if it works well for you.
–
Nick Anderson
Doer of things, CFEngine
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.
Nick Anderson
R: /tmp/services
R: hpss_d_mvr2 65503/tcp
R: hpss_d_mvr1 65501/tcp
> $ cat /tmp/services
hpss_d_mvr2 65503/tcp
hpss_d_mvr1 65501/tcp
nickanderson@nickanderson-ThinkPad-W550s /tmp [7:34:27]
> $ vi /tmp/services
nickanderson@nickanderson-ThinkPad-W550s /tmp [7:34:58]
> $ cat /tmp/services
hpss_d_mvr2 65503/tcp
hpss_d_mvr1 65501/tcp
nickanderson@nickanderson-ThinkPad-W550s /tmp [7:35:00]
> $ cf-agent -KIf ./t.cf
R: /tmp/services
R: hpss_d_mvr2 65503/tcp
R: hpss_d_mvr1 65501/tcp
> $ cat /tmp/services
hpss_d_mvr2 65503/tcp
hpss_d_mvr1 65501/tcp
Nick, if the spaces are included in the value itself, won't it edit the file if the spaces are absent even if the value is correct?
Best,
—Mike Weilgart
Vertical Sysadmin, Inc.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengine+unsubscribe@googlegroups.com.
David Lee
Firstly, I should point out that this particular use-case is simply for "prettiness" rather than something functional. The entries in "/etc/services" are of the form "key space value", where 'space' is technically one or more spaces, and conventionally is implemented as several spaces (not one) in order to keep it looking nice and pretty. So this use-case is not high priority.
Normally I would love to pursue the suggestions. Indeed, in the past I have done so, and the results of that actually led to the development of the "set_line_based()" itself.
Normally, I would then have considered trying additionally to provide test cases for the test suite. Again, in the past I have done so, for instance with some strange behaviour when combining lists, way back in the days of "cf_null".
But currently, work, health and other things are somewhat stressful. So I may not get around to this, as the workaround ("just use a single space") technically works.
What I would add, for anyone wishing to pursue this is that the items in play are:
- a multi-space second argument to "set_line_based()"; try at least one, two, three and four
- possible leading spaces in the value part of:"arr[key1]" string => "value1-with-leading-spaces"
- pre-existence of "keyn" in the file
- pre-absence of "keyn" from the file
- combinations of the above
- finally, encapsulating this in regression tests
Thanks again for your help!
-- David