CFEngine slist problem

[Todd Erwin]

Im working on capturing all of the ssh keys in a particular directory and then putting them all in a Slist, however my slist seems to always contain an EMPTY.

Here is the very simplified code and the output..

   vars:

      any::

         "dot_ssh"  string => "/root/.ssh";

         "pubkeys_dir"  string => "$(dot_ssh)/pubkeys";

          "pubkeys_list" slist => splitstring( execresult("/bin/ls $(pubkeys_dir)", "noshell"), "\s", "99999999999" );

  reports:

      "$(pubkeys_list)";

Here is the output..

R: $(pubkeys_list)

R: pubkey1.pub

R: pubkey2.pub

Output of ls /root/.ssh/pubkeys

pubkey1.pub pubkey2.pub

Im not sure how it keeps getting the expanded variable as the first element in the list, as you can imagine this causes errors when then trying to work with the files becuase file $pubkeys_list doesn't exsist.

Is there a better way to get an array of filenames from a Directory?  Why does this code break?

Thanks..

[Aleksey Tsalolikhin]

Try using findfiles() instead of /bin/ls and splitstring()

https://docs.cfengine.com/docs/3.10/reference-functions-findfiles.html

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengine+unsubscribe@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

[Aleksey Tsalolikhin]

P.S. you are seeing some spillover from the current implementation of the CFE promise language. :-/  it's supposed to "just work" but as you can see...

[Todd Erwin]

Yes unfortunately I tried with findfiles also with the same result..

                        "pubkeys_list" slist => findfiles("$(pubkeys_dir)/*");

R: $(pubkeys_list)

R: /root/.ssh/pubkeys/pubkey1.pub

R: /root/.ssh/pubkeys/pubkey2.pub

And then in my edit files output errors it complains like this.. (I got to thinking maybe the issue was with reports and not down in files"

2017-03-09T11:16:10-0700    error: /default/global_root_authorized_keys/methods/'any'/default/update_common_root_ssh_files/files/'/root/.ssh/authorized_keys'/default/insert_file_if_no_line_matching/insert_lines/'/root/.ssh/pubkeys/$(pubkeys_list)'[0]: Could not read file '/root/.ssh/pubkeys/$(pubkeys_list)'. (fopen: No such file or directory)

To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.

[Aleksey Tsalolikhin]

Ok. I'm on smartphone on a layover so can't share source code but I actually solved distributing ssh keys recently with a recursive files copyfrom promise.

Is that what you are working on?

To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengine+unsubscribe@googlegroups.com.

[Todd Erwin]

Yes and what i have is working, minus the fact that the SLIST contains the VARNAME which causes errors in every run because of nofile exsisting.  Also I have a central repository for keys that get's copied locally to each box so hardcoding the slist is not an option.  The code that Im using is actually documented with some slight modifications..

https://docs.cfengine.com/docs/3.5/examples-policy-distribute-ssh-keys.html

[Beto]

findfiles works fine:

bundle agent test
{
  vars:
      any::
         "dot_ssh"  string => "/tmp";
         "pubkeys_dir"  string => "$(dot_ssh)/masterfiles-3.10.0/masterfiles/controls";
          "pubkeys_list" slist => findfiles( "$(pubkeys_dir)/*.cf" );

  reports:
      "$(pubkeys_list)";
}

beto@corsair [inputs]$ cf-agent -V
CFEngine Core 3.9.2
beto@corsair [inputs]$ cf-agent -KIf ./findfiles.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/cf_agent.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/cf_execd.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/cf_hub.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/cf_monitord.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/cf_runagent.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/cf_serverd.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/def.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/def_inputs.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/reports.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/update_def.cf
R: /tmp/masterfiles-3.10.0/masterfiles/controls/update_def_inputs.cf

[Sean Johnson]

For what it's worth, I don't get your result. Any chance there's some other config that's producing the extra output?

~ ❯❯❯ cat foo.cf
body common control
{
bundlesequence => { foo };
}

bundle agent foo
{
vars:
"dot_ssh" string => "/tmp/foodir";
"pubkeys_dir" string => "$(dot_ssh)/pubkeys";
"pubkeys_list" slist => findfiles("$(pubkeys_dir)/*");

reports:
"$(pubkeys_list)";

}

~ ❯❯❯ ls /tmp/foodir/pubkeys
file1 file2 file3 file4 file5

~ ❯❯❯ cf-agent -f ./foo.cf
R: /tmp/foodir/pubkeys/file1
R: /tmp/foodir/pubkeys/file2
R: /tmp/foodir/pubkeys/file3
R: /tmp/foodir/pubkeys/file4
R: /tmp/foodir/pubkeys/file5

~ ❯❯❯

On 9 Mar 2017, at 12:35, Todd Erwin wrote:

Yes and what i have is working, minus the fact that the SLIST contains the VARNAME which causes errors in every run because of nofile exsisting.  Also I have a central repository for keys that get's copied locally to each box so hardcoding the slist is not an option.  The code that Im using is actually documented with some slight modifications..

https://docs.cfengine.com/docs/3.5/examples-policy-distribute-ssh-keys.html

[Todd Erwin]

So yes after looking at this code I over simplified..

Im using a method call, and then inside of the method Im setting a conditional for AFTER I copy the files.  The first run that conditional is not true because im using if_ok to set the class.  So thus the empty value the first iteration, next iteration it then populates the slist.

I fixed this by creating another bundle and calling it first.

more simplified code..

  "any" usebundle => setup_common_root_ssh_files;

  "any" usebundle => update_root_ssh_authorized_keys;

So now in setup_common_root_ssh_files it copies all the files and sets the class.

Then i update the authorized_keys file and first time thru it has all ready "CONVERGED" on the copying and set the class for the copy being Okay.

This workaround worked.

Unfortunately as it turns out my logic for stuffing into authorized_keys is not accounting for ROGUE entries..  Im not sure re-creating the file every run is smart to make sure nothing but keys from my repo are there? 

[Aleksey Tsalolikhin]

There is a copyfrom attribute to purge files that are not present on source. 

I'd use it with care in a WAN environment liable to network issues. 

--

You received this message because you are subscribed to the Google Groups "help-cfengine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengine+unsubscribe@googlegroups.com.

[Nick Anderson]

Hey Todd,

I was just skimming the thread and thought you might find this example
interesting.

If I understand correctly you have some data (public keys) that you want to
ensure exist in a file. These should be the ONLY entries in the file, but you
don't want to re-write the file each time the agent runs.

I really like to use a collection pattern.

bundle agent main
{
  vars:

    "string1" string => "key1", meta => { "mytag" };
    "string2" string => "key2", meta => { "mytag" };

    "collected" data => variablesmatching_as_data(".*", "mytag");

    "i" slist => sort( getvalues(collected), lex);

  reports:
    "$(i)";

}

R: key1
R: key2

In the above policy you have 2 strings (string1 and string2). Their values
could be populated by reading in data from each of your individual files. We tag
them with mytag so that we can easily find them. This allows you to have more
data sources outside this bundle that might be contributing to your full data
set.

We collect all the variables tagged with mytag into a data structure and then
extract the values into a sorted list. The sort would be important so that the
order of the list doesn't change and cause unnecessary writes.

From there you could use a template (I recommend mustache templates) or an
edit_line promise to ensure that only that canonical list of content exists in
the file.

You could also use a classic array as a generator like this:

/tmp/iterate-top.mustache:

{{#-top-}}
{{.}}
{{/-top-}}

Policy:

bundle agent main
{
  vars:

    "keys" slist => lsdir("/etc/ssh/", ".*.pub", "false");
    "key[$(keys)]"
     string => readfile( "/etc/ssh/$(keys)", inf);

    "i" slist => sort( getvalues(key), lex);

  files:
    "/tmp/example.txt"
      create => "true",
      edit_template => "/tmp/iterate-top.mustache",
      template_method => "mustache",
      template_data => @(i); #TIL template_data can take lists just like data.


  reports:
    "$(key$(range))";
    "$(range)";
    "/tmp/example.txt:"
      printfile => cat("/tmp/example.txt");

}

R: /tmp/example.txt:
R: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB7BkQAUSfVHfbsswmCcK43ES3rvprk+s33c4XTslhWOdlEzUf/zfQpHX/psjFswkyQSil9TtI7rXfqGhYcsQGU= root@nickanderson-ThinkPad-W550s
R: ssh-dss AAAAB3NzaC1kc3MAAACBAObSHMPdeRlXA7Qlzpdlsxg1Sdgts3U+o3a4o0vK/ta0iYJGWWyvw/5Yq8iWwDlRTb4T12zZ0P71tAmZgGrjQnkQCv+aFr6KBjJAqxOUTDfogAB3tacKL23zXgFzj5Rs5h11UHQTggmDUKHBm98CV4/1cpeuPMSP/i+o0qhrJtJ1AAAAFQD+I4hCTCbaUTi93TM1zox9KiuVGwAAAIEAlNhUEqEUHoh0S3iOyJqA+ULM4Y6PHUIPlBzYrIWXdDzi8EQJ2fKssDerfRw4GWldc1ZZvEkKK0iaDGPuWEDlZAP09YMb82kbXlDqIsgH79sH4JVSj1ew0H+aJvoizEjszRxLy77lT7VF8UgI8l8vhUuqNYJ0rMEZWUSO7PcNlKoAAACBALphnIeqVB44tHyNmflZZURCnGgNAQj3qEQkCILynZEYc9fr8f4Hqso3p2qJBGV2nWp8ADx89+C83iKw11wyujN73fow3JpMkzO7pwSP7/az6i/DCeSG4CxQTHJdt8c9uEk0rX7bPhQONHHK3Aap3T7wlysecNa/2IiJFGs1CpX/ root@nickanderson-ThinkPad-W550s
R: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJlAr3MIXec9zlwSicQPTdWwiwNSAgt/a9he5nMFlX2P root@nickanderson-ThinkPad-W550s
R: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIWriCUQAisvcsq+51/pCV8aHrihMbfpmVNff/M4hZwS8dChGPQbmvhtfxL/LcX3iNVoD1mH6lRKCeQJjjY2u9n6EhMgx6yW6l0oIlnGuJ0qYArBB6zoCCfapK/9HcoQRbso/QFDwMc4hqNFxot6iBkxUd1ib6eKIEUBt8FJRMAQNXCfVQ/IrQ3q0Ksx2NVNpdrWo57eUGIu6qrGFUsYZCH+MRCdhvjSB7A8kPH7X/pgNAUcIkIza5omarz3hWRckTL+62Wz83mzqLIOsC/xB9Ir0BEKBwb/95b6XKzOgXByOtutEzCOmZxEedcT7SoA4JQfBjloHpKyCftiTcjG8H root@nickanderson-ThinkPad-W550s

You could even mix them and merge the results together with mergedata.

–
Nick Anderson
Doer of things, CFEngine