accessing data from json array of json arrays

[Xander Cage]

Hi,

i am reading in a inventory file in csv format with readdcsv() and get what i believe is a array of arrays.

R: From aix inventory, got data [["xlsmp.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.en_US.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.EN_US.rte","5.1.0.0","default","lslpp"],["xlsmp.aix61.rte","5.1.0.0","default","lslpp"],["xlmass.lib","6.1.0.4","default","lslpp"],["xlmass.aix53.lib","6.1.0.4","default","lslpp"],["xlmass.adt.include","6.1.0.4","default","lslpp"],["xlmass.8.1.3","8.1.3.1","default","lslpp"],["xlmass.10.1.0","10.1.0.0","default","lslpp"],["xlfrte.aix61","15.1.3.5","default","lslpp"],["xlfrte","15.1.3.5","default","lslpp"]]

now i need to extract the version number from a package (by name).  tried filter() and grep() but both returned nothing.

example:

["xlfrte.aix61","15.1.3.5","default","lslpp"]

      |                         |    

  package name   version

test promise:

bundle agent eval_numbers {


    vars:

        "my_data" data => readcsv("$(sys.workdir)/state/software_packages.csv");
        "aix_inventory_str" string => format("%S", my_data);

        "test_filtergrep" slist => grep("^ xlfrte .*$", my_data);

        #"keys_unsorted" slist => getindices("my_data");
        #"keys"          slist => sort(keys_unsorted, "lex");

    #methods:

        #"call_to_get_stuff"   usebundle => get_stuff ( @(my_data), $(keys) );

    reports:

        "From aix inventory, got data $(aix_inventory_str)";

        "Filtered result $(test_filtergrep)";

        #"Keys sorted $(keys)";

}

so how to do it?

wbr

chris

[Nick Anderson]

Hi, Chris

Hi,

i am reading in a inventory file in csv format with readdcsv() and get what i believe is a array of arrays.

R: From aix inventory, got data [["xlsmp.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.en_US.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.EN_US.rte","5.1.0.0","default","lslpp"], ["xlsmp.aix61.rte","5.1.0.0","default","lslpp"],["xlmass.lib","6.1.0.4","default","lslpp"],["xlmass.aix53.lib","6.1.0.4","default","lslpp"], ["xlmass.adt.include","6.1.0.4","default","lslpp"],["xlmass.8.1.3","8.1.3.1","default","lslpp"],["xlmass.10.1.0","10.1.0.0","default","lslpp"], ["xlfrte.aix61","15.1.3.5","default","lslpp"],["xlfrte","15.1.3.5","default","lslpp"]]

Yep, that's what it looks like to me.

By the way, have you seen classfiltercsv()? One of it's nice features is named columns.

now i need to extract the version number from a package (by name). tried filter() and grep() but both returned nothing.

example:

["xlfrte.aix61","15.1.3.5","default","lslpp"]

package name version

so how to do it?

What is it you want to do with the version number?

I often like to use associative arrays for to re-construct data structures.

I might access it like this:

bundle agent __main__
{
  methods: "eval_numbers";
}
bundle agent eval_numbers {

  vars:

      #"my_data" data => readcsv("$(sys.workdir)/state/software_packages.csv");
      "my_data" data => '[["xlsmp.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.en_US.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.EN_US.rte","5.1.0.0","default","lslpp"],

       ["xlsmp.aix61.rte","5.1.0.0","default","lslpp"],["xlmass.lib","6.1.0.4","default","lslpp"],["xlmass.aix53.lib","6.1.0.4","default","lslpp"],
       ["xlmass.adt.include","6.1.0.4","default","lslpp"],["xlmass.8.1.3","8.1.3.1","default","lslpp"],["xlmass.10.1.0","10.1.0.0","default","lslpp"],

       ["xlfrte.aix61","15.1.3.5","default","lslpp"],["xlfrte","15.1.3.5","default","lslpp"]]';

      # here we get the positional indices of each array which we use to iterate over the data
      "l_indices_my_data" slist => getindices( my_data );

      # We might build an associative array which can make the data easier to
      # work with. You may or may not want to use all the fields. You can even
      # do filtering here if you only need to use part of the data, e.g. by
      # adding if => regcmp( "^xlfrte.*$", "$(my_data[$(l_indices_my_data)][0])" )

      "arr_my_data[$(my_data[$(l_indices_my_data)][0])][version]"
        string =>"$(my_data[$(l_indices_my_data)][1])";
      "arr_my_data[$(my_data[$(l_indices_my_data)][0])][field3]" # Perhaps useless
        string =>"$(my_data[$(l_indices_my_data)][2])";
      "arr_my_data[$(my_data[$(l_indices_my_data)][0])][command]" # Perhaps useless
        string =>"$(my_data[$(l_indices_my_data)][3])";


      # From here you can get a consolidated list of package names, which could be filtered
      "l_my_data_packages" slist => getindices( arr_my_data );
      "l_my_data_filtered" slist => filter( "^xlfrte.*$",
                                            getindices( arr_my_data ), # Could have used l_my_data_packages instead of getindices again
                                            true,
                                            false,
                                            inf);
  reports:

      # Print out a list of the filtered package names
      "Filtered result: $(with)" with => join( ", ", l_my_data_filtered );

      # Print out a list of the filtered package names and their versions
      "Package $(l_my_data_filtered): $(arr_my_data[$(l_my_data_filtered)][version])";
}

# cf-agent --no-lock --log-level info --show-evaluated-vars=eval_numbers --file /tmp/example.cf
R: Filtered result: xlfrte, xlfrte.aix61
R: Package xlfrte: 15.1.3.5
R: Package xlfrte.aix61: 15.1.3.5
Variable name                            Variable value                                               Meta tags                                Comment
default:eval_numbers.arr_my_data[xlfrte.aix61][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlfrte.aix61][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlfrte.aix61][version] 15.1.3.5                                                     source=promise
default:eval_numbers.arr_my_data[xlfrte][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlfrte][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlfrte][version] 15.1.3.5                                                     source=promise
default:eval_numbers.arr_my_data[xlmass.10.1.0][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlmass.10.1.0][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.10.1.0][version] 10.1.0.0                                                     source=promise
default:eval_numbers.arr_my_data[xlmass.8.1.3][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlmass.8.1.3][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.8.1.3][version] 8.1.3.1                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.adt.include][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlmass.adt.include][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.adt.include][version] 6.1.0.4                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.aix53.lib][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlmass.aix53.lib][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.aix53.lib][version] 6.1.0.4                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.lib][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlmass.lib][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlmass.lib][version] 6.1.0.4                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.aix61.rte][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlsmp.aix61.rte][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.aix61.rte][version] 5.1.0.0                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.msg.EN_US.rte][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlsmp.msg.EN_US.rte][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.msg.EN_US.rte][version] 5.1.0.0                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.msg.en_US.rte][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlsmp.msg.en_US.rte][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.msg.en_US.rte][version] 5.1.0.0                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.rte][command] lslpp                                                        source=promise
default:eval_numbers.arr_my_data[xlsmp.rte][field3] default                                                      source=promise
default:eval_numbers.arr_my_data[xlsmp.rte][version] 5.1.0.0                                                      source=promise
default:eval_numbers.l_indices_my_data    {"0","1","2","3","4","5","6","7","8","9","10"}              source=promise
default:eval_numbers.l_my_data_filtered   {"xlfrte","xlfrte.aix61"}                                   source=promise
default:eval_numbers.l_my_data_packages   {"xlfrte","xlmass.adt.include","xlmass.8.1.3","xlfrte.aix61","xlmass.10.1.0","xlsmp.aix61.rte","xlmass.lib","xlmass.aix53.lib","xlsmp.rte","xlsmp.msg.EN_US.rte","xlsmp.msg.en_US.rte"} source=promise
default:eval_numbers.my_data             [["xlsmp.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.en_US.rte","5.1.0.0","default","lslpp"],["xlsmp.msg.EN_US.rte","5.1.0.0","default","lslpp"],["xlsmp.aix61.rte","5.1.0.0","default","lslpp"],["xlmass.lib","6.1.0.4","default","lslpp"],["xlmass.aix53.lib","6.1.0.4","default","lslpp"],["xlmass.adt.include","6.1.0.4","default","lslpp"],["xlmass.8.1.3","8.1.3.1","default","lslpp"],["xlmass.10.1.0","10.1.0.0","default","lslpp"],["xlfrte.aix61","15.1.3.5","default","lslpp"],["xlfrte","15.1.3.5","default","lslpp"]] source=promise

[Xander Cage]

Hi Nick,

<< What is it you want to do with the version number?>>

in the end, i need the version number to upgrade aix filesets if downlevel. until now, i did this by exuting plain old system commands, but i thought it would be cool to use the already existing inventory file.

anyway...your example helped a lot.... as always ;-)

thank you for that

chris

[Nick Anderson]

in the end, i need the version number to upgrade aix filesets if downlevel. until now, i did this by exuting plain old system commands, but i thought it would be cool to use the already existing inventory file. anyway…your example helped a lot…. as always ;-) thank you for that

Cool, what specifically was enlightening in the example I posted?

[Xander Cage]

My initial thougt was that i can do the data filtering directly in the json array and ommit the intermediate legacy array. my existing promise splits the  output of the lslpp command into an array. as this command is somewhat cpu intensive i contemplated with the inventory file solution. so i learned data filtering is not possible in an json array.

bundle agent get_lpp_info(fileset)
{
vars:
"lslpp_info" string => execresult ("/usr/bin/lslpp -O u -cl $(fileset) | /usr/bin/grep -v '^#'", "useshell"),
comment => "get info from lslpp fileset";
"info_array" slist => splitstring("$(lslpp_info)", ":", 8),
comment => "get each field from lslpp output";
"lpp_version" string => nth("info_array", 2);
"lpp_status" string => nth("info_array", 4);
"lpp_long_desc" string => nth("info_array", 6);

reports:

"Time: $(sys.date) - Bundle: $(this.bundle) - Message: [$(lslpp_info)] reported by lslpp."
comment => "do not remove this report .. won't call lslpp otherwise ..";
"$(lslpp_info)" bundle_return_value_index => "0",
comment => "give output back as index[0] (makes grepping for 'not installed' possible";
"$(lpp_version)" bundle_return_value_index => "1";
"$(lpp_status)" bundle_return_value_index => "2";
"$(lpp_long_desc)" bundle_return_value_index => "3";
}