url_get retrieving raw cfengine content

[ruisse...@gmail.com]

I'm using 

"url" string => "http://servername/hostinfo.php?hostname=$(hostname)&exportType=cfengine3";

"res" data => url_get($(url), options);

# "list" slist => getvalues( "res[content]" );
"out" string => format("%S", res);

reports:

 "$(out)";

 "$(output[content])";

 "$(res)"

     report_to_file => "$(targetdir)/$(hostname).cf"

and I get the following content

R: $(list) {"content":"realcontent","headers":"HTTP/1.1 200 OK\r\nContent-Type: application/cfengine3\r\nServer: Microsoft-IIS/7.5\r\nX-Powered-By: PHP/5.3.8\r\nX-Powered-By: ASP.NET\r\nDate: date GMT\r\nContent-Length: 1588\r\n\r\n","rc":0,"returncode":200,"success":true}

realcontent is really 300 lines of host specific cfengine code which I want to write to $(hostname).cf

Can anybody help me achieve this.? 

It should also be canonified  and the format needs to be maintained. (newlines etc).

[Ted Zlatanov]

On Wed, 20 Jul 2016 08:53:30 -0700 (PDT) ruisse...@gmail.com wrote:

r> I'm using
r> "url" string =>
r> "http://servername/hostinfo.php?hostname=$(hostname)&exportType=cfengine3";
r> "res" data => url_get($(url), options);
r> # "list" slist => getvalues( "res[content]" );
r> "out" string => format("%S", res);

r> reports:
r> "$(out)";
r> "$(output[content])";
r> "$(res)"
r> report_to_file => "$(targetdir)/$(hostname).cf"

r> and I get the following content
r> R: $(list) {"content":"realcontent","headers":"HTTP/1.1 200
r> OK\r\nContent-Type: application/cfengine3\r\nServer:
r> Microsoft-IIS/7.5\r\nX-Powered-By: PHP/5.3.8\r\nX-Powered-By:
r> ASP.NET\r\nDate: date GMT\r\nContent-Length:
r> 1588\r\n\r\n","rc":0,"returncode":200,"success":true}

r> realcontent is really 300 lines of host specific cfengine code which I want
r> to write to $(hostname).cf

r> Can anybody help me achieve this.?

Please post runnable self-contained examples. Trying to figure out your
actual code; I would check that "success" is true. Then:

vars: "mydata" string => nth("content", res); # and then write mydata to a file as usual
... or ...

reports:
"$(res[content])"
report_to_file => "$(targetdir)/$(hostname).cf"

That has a limit of 4Kb I think, test to be sure it works for you.

r> I should also be canonified and I want to maintain the format (newlines
r> etc).

Sorry, I don't know what this means.

Ted

[ruisse...@gmail.com]

Sorry for my english translation. I've made corrections and the following self contained solution.

#!/var/cfengine/bin/cf-agent -Kf

body common control {

    bundlesequence => { "get_hosts"};

    inputs => { "/var/cfengine/masterfiles/lib/3.6/stdlib.cf" };

}

bundle agent get_hosts {

vars:

 "options_str" string => ' {

    "url.verbose": 0,

    "url.timeout": 20,

  }';

  "options" data => parsejson($(options_str));

  "seenhosts" slist => { hostsseen("inf","lastseen","name") };

methods:

  "any" usebundle => store_host("$(seenhosts)","/home/user/.cfagent/inputs");

reports:

  "$(seenhosts)";

}

bundle agent store_host(hostname,targetdir) {

  vars:

   "options_str" string => ' {

      "url.verbose": 0,

      "url.timeout": 20,

    }';

    "options" data => parsejson($(options_str));

    "url" string => "http://server/hostinfo.php?hostname=$(hostname)&exportType=cfengine3";

    "res" data => url_get($(url), options);

    "output" data => mergedata("res[content]");

  files:

  "$(targetdir)/${hostname}.cf"

    comment                   => "Write host data",

    create                    => "true",

    perms                     => mo("644","root"),

    action                    => u_immediate,

    edit_line                 => fill_data(${output}),

    classes                   => if_repaired("host_inputs_copied");

}

bundle edit_line fill_data(data) {

  delete_lines:

    ".*";

  insert_lines:

    "$(data)";

}

body action u_immediate

{

      ifelapsed => "0";

}