Reading a JSON file, adding to it and then saving it

[Mark Gavalda]

Hi,

I'm trying to add lines to the Logstash Forwarder config file, which looks like this right now:

{

  "network": {

    "servers": [ "1.2.3.4:5000" ],

    "timeout": 15,

    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"

  },

  "files": [

    {

      "paths": [

        "/logs/access.log"

       ],

      "fields": { "type": "web_nginx_access" }

    }

   ]

}

And I'd like to add other logs in that files.paths array, then save it. The problem is that starting from the second line I will have to add the commas to not screw up the JSON syntax... how can I do that with Ansible.

I've tried so many solutions, just can't figure this one out.

I was thinking about using a "command cat logstash-forwarder.conf" plus "register: result" then somehow manipulating the object with {{ result.stdout|from_json }} but that's as far as I got. I'd really appreciate any comments.

Thanks,

Mark

[Mark McCoy]

Rather than doing this in a single logstash config file, I would suggest using a templated version of this file. For every log file that you want logstash to monitor, you can drop a separate file into /etc/logstash/conf.d/ and it will pick them all up (if you are using the default setting for logstash).

----
Mark McCoy

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/c14f6f62-0f63-4afa-b18d-f935fb233dba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joe Stewart]

I don't believe  logstash forwarder supports conf.d type configuration yet - https://github.com/elasticsearch/logstash-forwarder/issues/244

I'm curious how others have assembled a config file when a seerver is a member of multiple groups 

thanks,

Joe

[Mark McCoy]

My bad. I was assuming that logstash-forwarder had the same feature that regular logstash did.

It's kludgy, but how about having each group drop a list into a file on the target system, and then have a handler that would assemble the files into the final json configuration?

Also, according to the logstash-forwarder documentation, you can use a file glob, so could you put your log files into a single directory and use *.log or something like that?

----
Mark McCoy

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/7bf36575-1bbe-4788-9ff9-144f66465996%40googlegroups.com.

[James Martin]

Mark G.,

The to_json (or to_nice_json) filter and a template will do the magic you want.  Given this playbook:

- hosts: localhost
  connection: local
  vars:
    log_paths:
    - "/logs/access.log"
    - "/logs/error.log"
  tasks:
  - name: create config file
    template: src=logstash.json.j2 dest=/tmp/logstash.json

and this template:

{
  "network": {
    "servers": [ "1.2.3.4:5000" ],
    "timeout": 15,
    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"
  },
  "files": [
    {

      "paths": {{ log_paths | to_json }},
      "fields": { "type": "web_nginx_access" }
    }
   ]
}

Will render:

{
  "network": {
    "servers": [ "1.2.3.4:5000" ],
    "timeout": 15,
    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"
  },
  "files": [
    {

      "paths": ["/logs/access.log", "/logs/error.log"],
      "fields": { "type": "web_nginx_access" }
    }
   ]
}

- James

[Mark Gavalda]

Thanks James and everyone, the other night I ended up doing almost exactly like what you suggested and it works great!

Cheers,

Mark