File beat installation and configuration with Ansible

[Omri]

Hello,

I'm writing an Ansible script that install and configure filebeat (agent of logstash).

I've finished the installation part. Now i need to add the configuration part to the script. I pretty new with Ansible and i need some help.

The user that suppose to run the script will have to select a few groups from a list, and each group from the list contains a few logs paths, which need to be added to the filebeat configuration file.

For example:

Groups to select foe example: redis, nginx, php-fpm

Redis logs:

/var/log/redis/redis.log

/var/log/redis/sentinel.log

Nginx logs:

/var/log/nginx/access.log

/var/log/nginx/error.log

php-fpm logs:

/var/log/php-fpm/error.log

If the user will select redis and php-fpm, these logs will be added to filebeat configuration file (on the remote host), which located on /etc/filebeat/filebeat.yml, under paths section:

filebeat:
  prospectors:
    -
      paths:
        - /var/log/redis/redis.log
        - /var/log/redis/sentinel.log
        - /var/log/php-fpm/error.log

I will really appreciate  some help over here. Which of the modules are most recommended in this case? Code examples? etc. 

Here is what i've done so far, which contain the installation part only.

- name: Install filebeat

  gather_facts: False

  remote_user: myUser

  become: yes

  vars:

    elasticsearch_repo_base_url: "https://packages.elastic.co/beats/yum/el/$basearch"

    elasticsearch_repo_gpg_key_url: "http://packages.elastic.co/GPG-KEY-elasticsearch"

  tasks:

  - name: Importing Elasticsearch public GPG key

    rpm_key:

      key: "{{ elasticsearch_repo_url }}"

      state: present

  - name: Add repository for filebeat

    yum_repository:

      name: Elastic Beats Repository

      description: Elastic Beats Repository

      baseurl: "{{ elasticsearch_repo_base_url }}"

      gpgkey: "{{ elasticsearch_repo_gpg_key_url }}"

      gpgcheck: yes

  - name: Install filebeat

    yum:

      name: filebeat

      state: latest

  - name: Enabling filebeat service on boot and starting

    service:

      name: filebeat

      state: restarted

      enabled: yes

[Sergey Baranov]

Here is my example. I am using hash_behaviour = merge.

Layout:

├── group_vars
│   ├── php-fpm
│   └── nginx
├── roles
│   └── filebeat
│       ├── defaults
│       │   └── main.yml
│       ├── handlers
│       │   └── main.yml
│       ├── meta
│       │   └── main.yml
│       ├── README.md
│       ├── tasks
│       │   ├── configure.yml
│       │   ├── install.yml
│       │   ├── main.yml
│       │   ├── prepare.yml
│       │   ├── redhat
│       │   │   ├── install.yml
│       │   │   └── prepare.yml
│       │   └── debian
│       │       ├── install.yml
│       │       └── prepare.yml
│       ├── templates
│       │   └── filebeat.yml.j2
│       └── vars
|           ├── debian.yml
│           └── redhat.yml
├── filebeat.yml
└── inventory

templates/filebeat.yml.j2:

{{ ansible_managed}}
{{ filebeat.options | to_nice_yaml }}

tasks/configure.yml:

---

- name: Deliver Filebeat config
  template:
    src: filebeat.yml.j2
    dest: "{{ filebeat.config_path }}/filebeat.yml"
    mode: 0644
    backup: yes
    validate: 'filebeat.sh -configtest -strict.perms=false -c %s'
  notify:
    - restart filebeat

defaults/main.yml:

---

filebeat_version: "{{ version | default('5.3.0') }}"

filebeat:

  # Version of the filebeat server
  version: "{{ filebeat_version }}"

  config_path: '/etc/filebeat'

  options:
    filebeat:
      config_dir: /etc/filebeat/conf.d/
      prospectors:

    shipper:

    logging:
      to_syslog: true
      to_files: true
      files:
        path: /var/log/filebeat/
        name: filebeat.log
        rotateeverybytes: 10485760
      level: info

    output:
      logstash:
        hosts:
        - logstash.example.org:5044
        insecure: true
        index: filebeat

group_vars/php-fpm

---

filebeat:
  options:
    filebeat:
      prospectors:

        - document_type: syslog
          paths:
            - /var/log/cron
            - /var/log/secure
            - /var/log/messages
            - /var/log/yum.log
          input_type: log

        - document_type: php-fpm
          paths:
            - /var/log/php-fpm/error.log
          input_type: log

group_vars/nginx

---

filebeat:
  options:
    filebeat:
      prospectors:

        - document_type: syslog
          paths:
            - /var/log/cron
            - /var/log/secure
            - /var/log/messages
            - /var/log/yum.log
          input_type: log

        - document_type: nginx-access
          paths:
            - /var/log/nginx/access.log
          input_type: log

        - document_type: nginx-error
          paths:
            - /var/log/nginx/error.log
          input_type: log

        - document_type: php-fpm
          paths:
            - /var/log/php-fpm/error.log
          input_type: log

inventory:

[nginx]
nginx_server1
nginx_server2
nginx_server3

[php-fpm]
php_server1
php_server2
php_server3

четверг, 6 июля 2017 г., 1:54:11 UTC+3 пользователь Omri написал: