Filebeat on wazuh not success and always failed

[khai]

Hello Team,

Hope every one is doing good.

I am new to this Wazuh Tool.

We already install wazuh on Linux machine with following the tutorial at the offficial website, and all is well.

But, We always failed at install filebeat for geting log from others apps ( apache, databases, etc). 

Please help me to guide step-by-step how to install filebeat and make it posiblenfor wazuh server. 

I saw the documentation, but still i am getting confused.

# filebeat.yml

output.elasticsearch:

  hosts: ["127.0.0.1:9200"]

  protocol: https

  username: "admin"

  password: "admin"

  ssl.certificate_authorities:

    - /etc/filebeat/certs/root-ca.pem

  ssl.certificate: "/etc/filebeat/certs/filebeat.pem"

  ssl.key: "/etc/filebeat/certs/filebeat.key"

setup.template.json.enabled: true

setup.template.json.path: '/etc/filebeat/wazuh-template.json'

setup.template.json.name: 'wazuh'

setup.ilm.overwrite: true

setup.ilm.enabled: true

filebeat.modules:

  - module: wazuh

    alerts:

      enabled: true

    archives:

      enabled: false

#filebeat test output

elasticsearch: https://127.0.0.1:9200...

  parse url... OK

  connection...

    parse host... OK

    dns lookup... OK

    addresses: 127.0.0.1

    dial up... OK

  TLS...

    security: server's certificate chain verification is enabled

    handshake... OK

    TLS version: TLSv1.3

    dial up... OK

  talk to server... OK

  version: 7.9.1

#Elasticsearch test

 curl -XGET https://localhost:9200 -u admin:admin -k

{

  "name" : "node-1",

  "cluster_name" : "elasticsearch",

  "cluster_uuid" : "q2_dYsklQEyFmCorzyy2Sg",

  "version" : {

    "number" : "7.9.1",

    "build_flavor" : "oss",

    "build_type" : "deb",

    "build_hash" : "083627f112ba94dffc1232e8b42b73492789ef91",

    "build_date" : "2020-09-01T21:22:21.964974Z",

    "build_snapshot" : false,

    "lucene_version" : "8.6.2",

    "minimum_wire_compatibility_version" : "6.8.0",

    "minimum_index_compatibility_version" : "6.0.0-beta1"

  },

  "tagline" : "You Know, for Search"

}

[Juan Pablo Saez]

Hi khai,

Since the command filebeat test output does not show any errors, I might think you installed correctly filebeat. Did you install Kibana and Wazuh APP? Are you able to see any alerts in the Wazuh APP? Also, let me check everything is going right and you are looking in the correct file for logs. Please refer to /var/ossec/logs/alerts/alerts.(log|json) to check whether the wazuh manager is writing alerts.

Log files from services as apache or mysql should be monitored using the localfile option. In case you already configured it but the logs are not showing up, I might assume that you do not have appropiate rules to detect logs from the services. We can create you custom rules for that.

To help you in a better way, let us know your use of case: which service are you trying to monitorize, the OS distribution you are using and the deployment, if you did a single node installation or distributed.

Regards,
Sergio

​

[khai]

Hi Sergio,

I'm happy for your feedback  and thanks

we already install : 

1. Server Wazuh : wazuh manager, kibana, elasticsearch,  and filebeat according to the instructions from https://documentation.wazuh.com/4.0/installation-guide/open-distro/all-in-one-deployment/unattended-installation.html

2. Wazuh Agent : wazuh agent already install, and all is well, But for filebeat we got error at run command filebeat setup such as at below :

Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://x.x.x.x:9200: Get https://x.x.x.x:9200: x509: certificate is valid for 127.0.0.1, not x.x.x.x]

as your recommendation we check file /var/ossec/logs/alerts/alerts.log, and the result is 

For your information, we don't create anything rule ( not yet ) , because we still try to get log from machine agent to store log on wazuh server .

I hope Wazuh can provide results such as sample data on kibana. But I'm still confused to make it happen 

 

I would be happy if you could provide a sample rule to get access logs from our web server and database server  ( Linux OS : Ubuntu 20 )

I am ready to listen to your instructions ... 

Thanks

Khai

[khai]

Hi Sergio,

I'm happy for your feedback  and thanks

we already install : 

1. Server Wazuh : wazuh manager, kibana, elasticsearch,  and filebeat according to the instructions from https://documentation.wazuh.com/4.0/installation-guide/open-distro/all-in-one-deployment/unattended-installation.html

2. Wazuh Agent : wazuh agent already install, and all is well, But for filebeat we got error at run command filebeat setup such as at below :

Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at https://x.x.x.x:9200: Get https://x.x.x.x:9200: x509: certificate is valid for 127.0.0.1, not x.x.x.x]

as your recommendation we check file /var/ossec/logs/alerts/alerts.log, and the result is 

** Alert 1609261209.0: - ossec,pci_dss_10.5.2,pci_dss_10.5.5,gpg13_10.1,gdpr_II_5.1.f,gdpr_IV_35.7.d,hipaa_164.312.b,nist_800_53_AU.9,tsc_CC6.1,tsc_CC7.2,tsc_CC7.3,tsc>

2020 Dec 30 00:00:09 (dev-eform) any->ossec-logcollector

Rule: 591 (level 3) -> 'Log file rotated.'

ossec: File rotated (inode changed): '/var/log/apache2/error.log'.

** Alert 1609261209.373: - ossec,pci_dss_10.5.2,pci_dss_10.5.5,gpg13_10.1,gdpr_II_5.1.f,gdpr_IV_35.7.d,hipaa_164.312.b,nist_800_53_AU.9,tsc_CC6.1,tsc_CC7.2,tsc_CC7.3,t>

2020 Dec 30 00:00:09 (dev-eform) any->ossec-logcollector

Rule: 591 (level 3) -> 'Log file rotated.'

ossec: File rotated (inode changed): '/var/log/syslog'.

** Alert 1609261290.737: - ossec,pci_dss_10.5.2,pci_dss_10.5.5,gpg13_10.1,gdpr_II_5.1.f,gdpr_IV_35.7.d,hipaa_164.312.b,nist_800_53_AU.9,tsc_CC6.1,tsc_CC7.2,tsc_CC7.3,t>

2020 Dec 30 00:01:30 dev-siem->ossec-logcollector

Rule: 591 (level 3) -> 'Log file rotated.'

ossec: File rotated (inode changed): '/var/log/syslog'.

** Alert 1609273617.1094: - ossec,syscheck,pci_dss_11.5,gpg13_4.11,gdpr_II_5.1.f,hipaa_164.312.c.1,hipaa_164.312.c.2,nist_800_53_SI.7,tsc_PI1.4,tsc_PI1.5,tsc_CC6.1,tsc>

2020 Dec 30 03:26:57 dev-siem->syscheck

Rule: 550 (level 7) -> 'Integrity checksum changed.'

File '/etc/filebeat/filebeat.yml' modified

Mode: scheduled

Changed attributes: size,mtime,md5,sha1,sha256

Size changed from '600' to '630'

Old modification time was: '1609144070', now it is '1609231941'

Old md5sum was: '56b8e1b425acc9107313514795fa1d54'

New md5sum is : '1ce03cfeb7ba22bb8af1c74e41742157'

Old sha1sum was: 'c46619dab696dbcf045bce2e2fa80deeb118ac7b'

New sha1sum is : '15bfd9af9e5c4fb82e6e16ca57b3bfdcfe24f207'

Old sha256sum was: 'ed460417299c497c28b9c907b8b3f0625702716745d15fad91d3b3a1558d4070'

New sha256sum is : 'd433037eb7898aff2f4d607602f00821eb3162b2fb5523729ad44a0daed09f05'

For your information, we don't create anything rule ( not yet ) , because we still try to get log from machine agent to store log on wazuh server .

I hope Wazuh can provide results such as sample data on kibana. But I'm still confused to make it happen 

 

I would be happy if you could provide a sample rule to get access logs from our web server and database server  ( Linux OS : Ubuntu 20 )

I am ready to listen to your instructions ... 

Thanks

Khai



Reply all

Reply to author

Forward

On Tuesday, December 29, 2020 at 5:26:13 PM UTC+7 Juan Pablo Saez wrote:

[Juan Pablo Saez]

Hello again khai,

The output from your /var/ossec/logs/alerts/alerts.log file shows that your Wazuh server is working properly. 

On the other hand, seems like Filebeat is trying to connect with elasticsearch but it is not correctly configured. In single-node environments, the output.elasticsearch.hosts on filebeat.yml file should be pointing always to localhost(127.0.0.1) since the certificates are created for that IP address (related configuration in this file). 

Once filebeat it is correctly running again, and filebeat test output is all green, please, check out if you can acces Kibana and see alerts there.

After this we could help you monitoring the desired files in your servers and, if needed, creating custom rules.

Regards,
Sergio

​