Troubleshooting Filebeat

[Hich]

Hi all, 

Few days ago me Kibana was empty so i've tried to restart my server. 

Now, doing "systemctl" command showing that filebeat.service is "failed failed". What command can i use to have more information on the error?

Thanks a lot

[Javier Escobar]

Hi Hich, 

To help you in the best way possible it would be necessary further information. Could you execute the following commands and paste here the output? 

systemctl status filebeat -l

filebeat test config

tail -n 20 /var/log/filebeat/filebeat

Can you share your Filebeat configuration file which is stored at /etc/filebeat/filebeat.yml?

Let me know if you have any questions.

Regards, 

Javier

[Hich]

Other commands are saying: no modules or inputs enabled and configuration reloading disabled. What files do you want me to watch?

[Javier Escobar]

Hi again Hich, 

Are you using Logstash with Elasticsearch? And what version of Wazuh/Elastic Stack are you using?

I noticed that your Filebeat configuration file (/etc/filebeat/filebeat.yml) doesn't have configured the IP and port for Logstash and that may be what is causing the issue. 

Edit the file with your Logstash IP and port and restart the service with the following command:

systemctl restart filebeat

If you keep having issues let us know.

Regards, 

Javier

[Hich]

Thanks a lot for your answer, my logstash is on the same server, should i use someting like ["localhost:5000"] ? If so, it doesn't change anything but maybe i'm wrong with the port, it should be the default one 

[Hich]

My Architecture was working perfectly, i was just having not enough disk space, but then my kibana wasn't showing anything and by restarting my server, filebeat crashed... i've seen that no modules are enabled also

[Javier Escobar]

Hi Hich, 

I would recommend using the IP of the machine and port 5000 rather than localhost (example: 172.16.1.2:5000). Edit the file /etc/filebeat/filebeat.yml and replace YOUR_ELASTIC_SERVER_IP with the IP address of Logstash.

And restart filebeat with this command:

systemctl restart filebeat

Also, could you execute the following commands and paste here the output? 

Elastic version:

curl ELASTIC_IP:9200?pretty

Kibana version:

/usr/share/kibana/bin/kibana --version

Filebeat version:

/usr/share/filebeat/bin/filebeat version

Logstash version:

/usr/share/logstash/bin/logstash --version

Wazuh version:

cat /var/ossec/etc/ossec-init.conf

Wazuh API version:

cat /var/ossec/api/package.json

Wazuh APP version:

cat /usr/share/kibana/plugins/wazuh/package.json

Regards, 

Javier

[Hich]

Hi Javier,

Pretty much everything is in version 6.3, and wazuh 3.9. Except, filebeat in version 7.3, do you think that an update of services can fix it?

[Javier Escobar]

Hi Hich, and sorry for the late response.

When using Elastic Stack, all components (Filebeat, Elasticsearch, Kibana ...) should have a similar version to work properly. Moreover, you are using an old version of Elastic which is not compatible with Wazuh 3.9 (https://documentation.wazuh.com/3.9/installation-guide/compatibility_matrix/index.html#api-and-kibana-app). 

I think the best solution should be to upgrade Elastic Stack. If you decide to upgrade we have several guides to do so. 

First, you will need to upgrade from 6.3 to 6.8 following this guide: 

https://documentation.wazuh.com/3.9/upgrade-guide/upgrading-elastic-stack/elastic_server_hard_upgrade.html

And then, upgrade from 6.8 to 7.3.0: 

https://documentation.wazuh.com/3.9/upgrade-guide/upgrading-elastic-stack/elastic_server_rolling_upgrade.html

If you decide to upgrade, you must do the same with Wazuh. Here you can see how to do it:

https://documentation.wazuh.com/3.9/upgrade-guide/upgrading/latest_wazuh3_minor.html#upgrading-latest-minor

Starting from 3.9.x Wazuh does not require having Logstash by default and Filebeat forward the alerts directly to Elasticsearch. If you have a requirement to keep Logstash and as you are using a single host architecture (Wazuh & ELK stack in same box) Filebeat won't be need in this use case.

Let me know if you need anything.

Regards,

Javier