classfiltercsv: filtering only works on bundle level

[Xander Cage]

seems that the classes filter only works if the data and filter classes are in the same bundle. i want to use the data container as a somewhat "global" variable without repeating it  in any bundle that uses it.

for example:

bundle agent parent_bundle
{
  classes:

      #"weirdos";
      #"twats";
      #"zombie";
      #"amlinz";

  vars:
      "data_file" string => "/root/cfe_testbed/users.csv";
      "d" data => classfiltercsv($(data_file), "true", 0);

      "keys_unsorted" slist => getindices("d");
      "keys" slist => sort(keys_unsorted, "lex");

      #"users_$(keys)" data => mergedata("d[$(keys)]");


  methods:

   "call_child"     usebundle => child_bundle(@(d), $(keys)),
                           handle => "remove_it";

   
}

bundle agent child_bundle (info, idx) {

    classes:
        

         # filtering class to get only the relevant data for this bundle, seems the natural way          #to do it,

         # but does not work 

         "zombie";
             
    vars:

        "names" slist => getvalues("info[$(idx)][name]");
        "names_str" string => format("%S", names);

    methods:

        "call_check_user"    usebundle => check_user("$(names_str)");
}

[Nick Anderson]

Why is this not in my inbox?

[Nick Anderson]

Hi,

Hopefully this attaches itself to the correct thread, no email in my inbox to respond to for this one. ¯\_(ツ)_/¯

Can you provide a completely stand-alone policy so that we can easily run and re-produce the problem you have?

Proceeding only with my internal parser :)….

Regarding "only works if the data and filter classes are in the same bundle", I suspect that you are just seeing class scope at work. In the documentation, Classes and Decisions in Language Concepts of the Reference manual has a section on Class scope which talks about this a bit. It looks like you are using classes that are defined with a bundle scope (only visible within that bundle) and when you call your child_bundle you did not have it inherit the classes, so inside of child_bundle, any bundle scoped classes that were defined in parent_bundle won't be visible.

bundle agent __main__
{
  methods:
      "parent";

}
bundle agent parent
{
  classes:
      "my_bundle_scoped_class_defined_in_parent";

  methods:
      "Write our child into our will"
        usebundle => child,
        inherit => "true";

      "Step children get the shaft by default"
        usebundle => step_child;

}
bundle agent child
{
  reports:
      "$(this.bundle): I can see my_bundle_scoped_class_defined_in_parent since my parent wrote me into the will."
        if => "my_bundle_scoped_class_defined_in_parent";
      "$(this.bundle): I wasn't written into the will, at least I am not red-headed."
        unless => "my_bundle_scoped_class_defined_in_parent";
}
bundle agent step_child
{
  reports:
      "$(this.bundle): I can see my_bundle_scoped_class_defined_in_parent since my parent wrote me into the will."
        if => "my_bundle_scoped_class_defined_in_parent";
      "$(this.bundle): I wasn't written into the will, at least I am not red-headed."
        unless => "my_bundle_scoped_class_defined_in_parent";
}

R: child: I can see my_bundle_scoped_class_defined_in_parent since my parent wrote me into the will.
R: step_child: I wasn't written into the will, at least I am not red-headed.

Regarding "i want to use the data container as a somewhat "global" variable without repeating it" , I don't think that you have to repeat the data container, you can pass it whole like you do, or pass it's name. Either way, you don't get a variable registered in the state (see --show-evaluated-vars).

Listing 1: Example Policy

bundle agent __main__
{
   vars:
      "my_data" data => '{ "data": "here" }';

  methods:
      "Pass data container"
        usebundle => example_1( @(my_data) );

      "Pass NAME of data container"
        usebundle => example_2( "$(this.namespace):$(this.bundle).my_data" );
}

bundle agent example_1(my_data)
{
      vars: "my_data_$(this.bundle)" data => '{ "bundle": "$(this.bundle)" }';
      reports: "$(this.bundle):$(with)" with => storejson( @(my_data) );
}
bundle agent example_2(my_data)
{
      vars: "my_data_$(this.bundle)" data => '{ "bundle": "$(this.bundle)" }';
      reports: "$(this.bundle):$(with)" with => storejson( $(my_data) );
}

# cf-agent --no-lock --log-level info --show-evaluated-vars=my_data --file /tmp/example.cf
R: example_1:{
  "data": "here"
}
R: example_2:{
  "data": "here"
}
Variable name                            Variable value                                               Meta tags                                Comment                                 
default:example_1.my_data_example_1      {"bundle":"example_1"}                                       source=promise                                                                   
default:example_2.my_data_example_2      {"bundle":"example_2"}                                       source=promise                                                                   
default:main.my_data                     {"data":"here"}                                              source=promise                                                                   

Hope this helps …

[Xander Cage]

ah...yes, just a matter of reorganization, thank you. helped a lot.

while we are at it...i am just doing all this because of this phrase in -> https://docs.cfengine.com/docs/3.18/reference-language-concepts-variables.html#associative-arrays

Note that associative arrays are being deprecated in favor of the data variable type.

this sent me on the hunt for arrays in my policy jungle. or is this statement just informational?

[Nick Anderson]

Deprecated is probably too strong of a word there. They aren't going anywhere, at least not in CFEngine 3. But there is more focus on data type variables and generally I would tend to use functions that return data vs returning a classic array.

So, I probably would not spend time hunting down classic arrays unless I had nothing better to do.

[Xander Cage]

ah good to know, so there is no hurry in eliminating arrays.

btw...i tried you suggestion  and passed the data container to the child function...no fun. variable undefined and a strange error thrown.

/var/cfengine/bin/cf-agent -KI -f ./data_container_test.cf
   error: The value of variable 'default:remove_in_files.user' contains a reference to itself, '$(user)'
R: $(user)
R: Time: Mon Oct 18 10:07:14 2021 -  Bundle: check_user - Message: uid $(user) completely removed.

bundle agent parent_bundle
{

 
  vars:
      "data_file" string => "/root/cfe_testbed/users.csv";
      "d" data => classfiltercsv($(data_file), "true", 0);

     

  methods:

   "call_check_user"     usebundle => check_user( @(d) ),
                         handle => "remove_it";

   
}

bundle agent check_user (info) {

    vars:

          "keys_unsorted" slist => getindices("info");

          "keys" slist => sort(keys_unsorted, "lex");

          "user" string => "$(info[$(keys)][uid])";

    classes:

               classes:

                 "zombie";


               "EXISTS_PASSWD"
                     comment    => "check if user exists as local user (in /etc/passwd)",
                     expression => regline("^$(user):.*", "/etc/passwd");

               "EXISTS_LDAP"
                     comment    => "check if user has an entry in the /etc/security/user file",
                     expression => regline("^$(user):", "/etc/security/user");

               "EXISTS_HOMEDIR"
                    comment    => "user has a homedir..",
                    expression => fileexists("/home/$(user)/.");

    methods:

        "remove_by_aix"       usebundle  => remove_by_aix($(user)),
                              comment    => "if its an OS-user let AIX do a rmuser",
                              ifvarclass => "EXISTS_PASSWD";

        "remove_from_files"   usebundle  => remove_in_files($(user)),
                              comment    => "remove in stanza-files and cronjob-dir";

        "remove_home_dir"     usebundle  => remove_home_dir($(user)),
                              comment    => "will assume /home/$USER is the homedir",
                              ifvarclass => "EXISTS_HOMEDIR";

    reports:

       "$(user)";

        EXISTS_PASSWD.!EXISTS_LDAP::

            "Time: $(sys.date) -  Bundle: $(this.bundle) - Message: uid $(user) exists in /etc/passwd only (OS user).";

        !EXISTS_PASSWD.EXISTS_LDAP::

            "Time: $(sys.date) -  Bundle: $(this.bundle) - Message: uid $(user) exists only in security-file (LDAP user).";

        EXISTS_HOMEDIR.!EXISTS_LDAP.!EXISTS_PASSWD::

            "Time: $(sys.date) -  Bundle: $(this.bundle) - Message: uid $(user) leftover /home/dir.";

        !EXISTS_HOMEDIR.!EXISTS_LDAP.!EXISTS_PASSWD::

            "Time: $(sys.date) -  Bundle: $(this.bundle) - Message: uid $(user) completely removed.";

}

bundle agent remove_by_aix(user) {


    commands:

        "/usr/sbin/rmuser -p  "
            args    => "$(user)",
            comment => "remove with OS means ..";

}

bundle agent remove_in_files(user) {


    vars:

        "stanza_files" slist => { "/etc/security/limits",
                                  "/etc/security/user",
                                  "/etc/security/lastlog",
                                  "/etc/security/passwd",
                                  "/etc/security/environ",
                                  "/etc/security/user.roles",
                                };

    files:

        "/var/spool/cron/crontabs/$(user)"
                comment => "delete leftover cronjob file",
                delete  => tidy;

    methods:

        "call_remove_stanza" usebundle  => delete_stanza($(stanza_files), $(user)),
                             comment    => "delete user in each file";
}

bundle agent remove_home_dir(user) {


    methods:

        "remove_home_dir" usebundle => rm_rf("/home/$(user)"),
                          comment   => "delete leftover home-dir";
}

[Xander Cage]

simplified version...

csv data used:

ClassExpression,uid,name,group,ssh_key
weirdos,mscott,Michael Scott,pathetic,ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg
weirdos,jkras,John Krasinski,pathetic,ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg
twats,tflen,Toby Flenderson,tedious,ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg
zombie,wimm,,dbaldap,ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg

bundle agent parent_bundle
{

  vars:

      "data_file" string => "/root/cfe_testbed/users.csv";

      "my_data" data => classfiltercsv($(data_file), "true", 0);

      "keys_unsorted" slist => getindices(my_data);

      "keys" slist => sort(keys_unsorted, "lex");

      "user" string => "$(my_data[$(keys)][uid])";



  methods:

   "call_check_user"     usebundle => check_user( @(my_data) ),
                         handle => "remove_it";

    "call_create_user"   usebundle => create_users_new( @(my_data) ),
                         depends_on => { "remove_it" };

  reports:

   "parent bundle: $(user)";

}


bundle agent check_user (info) {

    vars:

          "keys_unsorted" slist => getindices(info);
          "keys" slist => sort(keys_unsorted, "lex");
          "user" string => "$(info[$(keys)][uid])";

    classes:

          "zombie";


     reports:

       "check_user reached: $(user)";


}


bundle agent create_users_new(info) {


    vars:
           "keys_unsorted" slist => getindices(info);

           "keys" slist => sort(keys_unsorted, "lex");

           "user" string => "$(info[$(keys)][uid])";

           "group" string => "$(info[$(keys)][group])";
           "ssh_key" string => "$(info[$(keys)][ssh_key])";
           "name" string => "$(info[$(keys)][name])";

    classes:

             "weirdos";
             "twats";

    reports:

        "create_users reached: $(user)";

}



bundle agent __main__
{
  methods:
      "parent_bundle";
}

output:

root@aixtest01: /root/cfe_testbed # /var/cfengine/bin/cf-agent -KI -f ./data_container_test_simplified.cf
R: check_user reached: $(user)
R: create_users reached: $(user)
R: parent bundle: $(user)

verbose:

verbose: Using bundlesequence =>  {"main"}
 verbose: B: *****************************************************************
 verbose: B: BEGIN bundle main
 verbose: B: *****************************************************************
 verbose: P: .........................................................
 verbose: P: BEGIN promise 'promise_data_container_test_simplified_cf_88' of type "methods" (pass 1)
 verbose: P:    Promiser/affected object: 'parent_bundle'
 verbose: P:    Part of bundle: main
 verbose: P:    Base context class: any
 verbose: P:    Stack path: /default/main/methods/'parent_bundle'[1]
 verbose: B: *****************************************************************
 verbose: B: BEGIN bundle parent_bundle
 verbose: B: *****************************************************************
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 1)
 verbose: V:     Computing value of 'data_file'
 verbose: V:     Computing value of 'my_data'
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: P: .........................................................
 verbose: P: BEGIN promise 'remove_it' of type "methods" (pass 1)
 verbose: P:    Promiser/affected object: 'call_check_user'
 verbose: P:    Part of bundle: parent_bundle
 verbose: P:    Base context class: any
 verbose: P:    Stack path: /default/main/methods/'parent_bundle'/default/parent_bundle/methods/'call_check_user'[1]
 verbose: B: *****************************************************************
 verbose: B: BEGIN bundle check_user( {"@(my_data)"})
 verbose: B: *****************************************************************
 verbose: V:     +  Private parameter: 'info' in scope 'check_user' (type: s) in pass 1
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 1)
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: C: .........................................................
 verbose: C: BEGIN classes / conditions (pass 1)
 verbose: C:     +  Private class: zombie
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: P: .........................................................
 verbose: P: BEGIN promise 'promise_data_container_test_simplified_cf_54' of type "reports" (pass 1)
 verbose: P:    Promiser/affected object: 'check_user reached: $(user)'
 verbose: P:    From parameterized bundle: check_user( {"@(my_data)"})
 verbose: P:    Base context class: any
 verbose: P:    Stack path: /default/main/methods/'parent_bundle'/default/parent_bundle/methods/'call_check_user'/default/check_user/reports/'check_user reached: $(user)'[1]
R: check_user reached: $(user)
 verbose: A: Promise was KEPT
 verbose: P: END reports promise (check_user reached: $(user))
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 2)
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: C: .........................................................
 verbose: C: BEGIN classes / conditions (pass 2)
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 3)
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: C: .........................................................
 verbose: C: BEGIN classes / conditions (pass 3)
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: A: ...................................................
 verbose: A: Bundle Accounting Summary for 'check_user' in namespace default
 verbose: A: Promises kept in 'check_user' = 1
 verbose: A: Promises not kept in 'check_user' = 0
 verbose: A: Promises repaired in 'check_user' = 0
 verbose: A: Aggregate compliance (promises kept/repaired) for bundle 'check_user' = 100.0%
 verbose: A: ...................................................
 verbose: Additional promise info: handle 'remove_it' source path './data_container_test_simplified.cf' at line 26
 verbose: Method 'check_user' verified
 verbose: B: *****************************************************************
 verbose: B: END bundle check_user
 verbose: B: *****************************************************************
 verbose: A: Promise was KEPT
 verbose: P: END methods promise (call_check_user)
 verbose: P: .........................................................
 verbose: P: BEGIN promise 'promise_data_container_test_simplified_cf_29' of type "methods" (pass 1)
 verbose: P:    Promiser/affected object: 'call_create_user'
 verbose: P:    Part of bundle: parent_bundle
 verbose: P:    Base context class: any
 verbose: P:    Stack path: /default/main/methods/'parent_bundle'/default/parent_bundle/methods/'call_create_user'[1]
 verbose: B: *****************************************************************
 verbose: B: BEGIN bundle create_users_new( {"@(my_data)"})
 verbose: B: *****************************************************************
 verbose: V:     +  Private parameter: 'info' in scope 'create_users_new' (type: s) in pass 1
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 1)
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: C: .........................................................
 verbose: C: BEGIN classes / conditions (pass 1)
 verbose: C:     +  Private class: weirdos
 verbose: C:     +  Private class: twats
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: P: .........................................................
 verbose: P: BEGIN promise 'promise_data_container_test_simplified_cf_79' of type "reports" (pass 1)
 verbose: P:    Promiser/affected object: 'create_users reached: $(user)'
 verbose: P:    From parameterized bundle: create_users_new( {"@(my_data)"})
 verbose: P:    Base context class: any
 verbose: P:    Stack path: /default/main/methods/'parent_bundle'/default/parent_bundle/methods/'call_create_user'/default/create_users_new/reports/'create_users reached: $(user)'[1]
R: create_users reached: $(user)
 verbose: A: Promise was KEPT
 verbose: P: END reports promise (create_users reached: $(user))
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 2)
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: C: .........................................................
 verbose: C: BEGIN classes / conditions (pass 2)
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 3)
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: C: .........................................................
 verbose: C: BEGIN classes / conditions (pass 3)
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: A: ...................................................
 verbose: A: Bundle Accounting Summary for 'create_users_new' in namespace default
 verbose: A: Promises kept in 'create_users_new' = 1
 verbose: A: Promises not kept in 'create_users_new' = 0
 verbose: A: Promises repaired in 'create_users_new' = 0
 verbose: A: Aggregate compliance (promises kept/repaired) for bundle 'create_users_new' = 100.0%
 verbose: A: ...................................................
 verbose: Additional promise info: source path './data_container_test_simplified.cf' at line 29
 verbose: Method 'create_users_new' verified
 verbose: B: *****************************************************************
 verbose: B: END bundle create_users_new
 verbose: B: *****************************************************************
 verbose: A: Promise was KEPT
 verbose: P: END methods promise (call_create_user)
 verbose: P: .........................................................
 verbose: P: BEGIN promise 'promise_data_container_test_simplified_cf_34' of type "reports" (pass 1)
 verbose: P:    Promiser/affected object: 'parent bundle: $(user)'
 verbose: P:    Part of bundle: parent_bundle
 verbose: P:    Base context class: any
 verbose: P:    Stack path: /default/main/methods/'parent_bundle'/default/parent_bundle/reports/'parent bundle: $(user)'[1]
R: parent bundle: $(user)
 verbose: A: Promise was KEPT
 verbose: P: END reports promise (parent bundle: $(user))
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 2)
 verbose: V:     Computing value of 'data_file'
 verbose: V:     Computing value of 'my_data'
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: V: .........................................................
 verbose: V: BEGIN variables (pass 3)
 verbose: V:     Computing value of 'data_file'
 verbose: V:     Computing value of 'my_data'
 verbose: V:     Computing value of 'keys_unsorted'
 verbose: V:     Computing value of 'keys'
 verbose: Skipping iteration since variable 'keys' resolves to an empty list
 verbose: A: ...................................................
 verbose: A: Bundle Accounting Summary for 'parent_bundle' in namespace default
 verbose: A: Promises kept in 'parent_bundle' = 5
 verbose: A: Promises not kept in 'parent_bundle' = 0
 verbose: A: Promises repaired in 'parent_bundle' = 0
 verbose: A: Aggregate compliance (promises kept/repaired) for bundle 'parent_bundle' = 100.0%
 verbose: A: ...................................................
 verbose: Additional promise info: source path './data_container_test_simplified.cf' at line 88
 verbose: Method 'parent_bundle' verified
 verbose: B: *****************************************************************
 verbose: B: END bundle parent_bundle
 verbose: B: *****************************************************************
 verbose: A: Promise was KEPT
 verbose: P: END methods promise (parent_bundle)
 verbose: A: ...................................................
 verbose: A: Bundle Accounting Summary for 'main' in namespace default
 verbose: A: Promises kept in 'main' = 6
 verbose: A: Promises not kept in 'main' = 0
 verbose: A: Promises repaired in 'main' = 0
 verbose: A: Aggregate compliance (promises kept/repaired) for bundle 'main' = 100.0%
 verbose: A: ...................................................
 verbose: B: *****************************************************************
 verbose: B: END bundle main
 verbose: B: *****************************************************************
 verbose: Waiting for background processes
 verbose: No more background processes to wait for
 verbose: No lock purging scheduled
 verbose: Logging total compliance, total 'Outcome of version (not specified) (agent-0): Promises observed to be kept 100.00%, Promises repaired 0.00%, Promises not repaired 0.00%'

[Xander Cage]

i forgot to mention that the above shows hat the "filter classes"  in the classesfiltercsv function not "filtering" when used in various child bundles if the definition of the datacontainer or the call to the classfiltercsv function is not present in the bundle. whats the deal if i have to define all filtering classes in the parent bundle? seems useless to me.

this function is at least somewhat inconvenient , or i am doing something completely wrong. any insights?

[Xander Cage]

ping ;-)

[Nick Anderson]

All the pings in the world don't help when these darn things land in SPAM  ;'/

[Nick Anderson]

Xander Cage <christia...@itsv.at> writes:

i forgot to mention that the above shows hat the "filter classes" in the classesfiltercsv function not "filtering" when used in various child bundles if the definition of the datacontainer or the call to the classfiltercsv function is not present in the bundle.

Reading the policy …, you are using classfiltercsv() inside of bundle agent parent_bundle. Running with your data set I get no data container defined, for example:

# cf-agent --no-lock --log-level info --show-evaluated-vars=parent_bundle --file /tmp/example.cf
R: check_user reached: $(user)
R: create_users reached: $(user)
R: parent bundle: $(user)
Variable name                            Variable value                                               Meta tags                                Comment                                 
default:parent_bundle.data_file          /tmp/my.csv                                                  source=promise                                                                   
default:parent_bundle.keys                                                                            source=promise                                                                   
default:parent_bundle.keys_unsorted                                                                   source=promise                                                                   
default:parent_bundle.my_data            []                                                           source=promise                                                                   

This is because none of the class expressions in your first (zeroth) column evaluate to true on my system in that bundle. So, I added a row that would (any,nickanderson,Nick Anderson,neither-us-nor-them,ssh-bs) and I modified the policy to point to the path I am using for the CSV (/tmp/my.csv)

Listing 1: CSV Data (/tmp/my.csv)

ClassExpression,uid,name,group,ssh_key
weirdos,mscott,Michael Scott,pathetic,ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg
weirdos,jkras,John Krasinski,pathetic,ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg
twats,tflen,Toby Flenderson,tedious,ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg
zombie,wimm,,dbaldap,

ssh-someSupERsecretKey287483438dsdjhdsdsdjahhsgdsg
any,nickanderson,Nick Anderson,neither-us-nor-them,ssh-bs

Listing 2: Example Policy

bundle agent parent_bundle
{

  vars:

      "data_file" string => "/tmp/my.csv";
      "my_data" data => classfiltercsv( $(data_file), "true", 0);

      

With that change, the rows with class expressions that evaluate to true become part of the data container that result from parsing.

# cf-agent --no-lock --log-level info --show-evaluated-vars=parent_bundle --file /tmp/example.cf
R: check_user reached: nickanderson
R: create_users reached: nickanderson
R: parent bundle: nickanderson
Variable name                            Variable value                                               Meta tags                                Comment                                 
default:parent_bundle.data_file          /tmp/my.csv                                                  source=promise                                                                   
default:parent_bundle.keys                {"0"}                                                       source=promise                                                                   
default:parent_bundle.keys_unsorted       {"0"}                                                       source=promise                                                                   
default:parent_bundle.my_data            [{"group":"neither-us-nor-them","name":"Nick Anderson","ssh_key":"ssh-bs","uid":"nickanderson"}] source=promise                                                                   
default:parent_bundle.user               nickanderson                                                 source=promise                                                                   

Reading the policy further …., some bundles (bundle agent check_user, and bundle agent create_user_new) are promised as methods. Inside these bundles there are some bundle scope classes defined and some variables are created by pulling the data off of the previously defined data container that is passed in to the bundle as an argument.

whats the deal if i have to define all filtering classes in the parent bundle? seems useless to me. this function is at least somewhat inconvenient , or i am doing something completely wrong. any insights?

Yes, sorry to say, I think you have a completely different expectation of what the behavior should be than what it provides.

To me, it looks like you expect classfiltercsv() to filter your data set when you access the data set. That might be a nifty function in it's own right, but that is not how classfiltercsv() behaves. classfiltercsv() filters the data as it is read in based on the context at that point in time.

So, if you want one data set of the rows that match weirdos and twats and another data set consisting of the rows that match zombie as I believe your example policy illustrates you would need to make 2 calls to classfiltercsv().

[Xander Cage]

nick, thank you for the clarification, at least i know which route to go now.

[Nick Anderson]