Dashboard not displaying any information since the 22nd.

[Dennis Hidecker]

I went to log into our Wazuh server this morning and noticed that nothing is being displayed in the dashboard since the 22nd of this month.  But when you look at the folders where /var/ossec/logs/alerts/2023/Mar/ossec-alerts-##.log & /var/ossec/logs/archives/2023/Mar/ossec-archive-##.log & .json are stored they contain logs from our servers, firewalls, & switches.  Is there any way to get the dashboard to repopulate the missing information into it?  Also, as I've done some research, I noticed that the index patterns in the dashboard only go up to the 22nd of the month an there is nothing since then.  I'm not sure if this is the cause or a symptom of the problem.

Thanks in advance for any help and for making a great & awesome software!!!!

[Javier Castro]

Hello,

The fact that Wazuh alerts are generated on the Wazuh manager side implies there must be an issue either in:

• Filebeat (it sends alerts to the Wazuh indexer or Elasticsearch, depending on what you are using).
• Wazuh indexer / Elasticsearch. It is the database used to visualize the information.

I recommend following the data flow and running these checks:

• On every Wazuh manager, run filebeat test output. This will show if there's some issue during the shipping process.
• Check disk usage both in your Wazuh manager and Wazuh indexer / Elasticsearch components by doing df -h. It shouldn't be above 85% usage in your Wazuh indexer / Elasticsearch as you may reach the watermark for disk usage.
• Check your Wazuh indexer / Elasticsearch logs, which should be located under /var/log/elasticsearch or /var/log/wazuh-indexer, as they may point you in the right direction.

I also recommend you read through the architecture section of the Wazuh documentation to understand the data flow: https://documentation.wazuh.com/current/getting-started/architecture.html

I hope this helps!

[Javier Castro]

It looks like indexing-wise, everything should be ok.

I would check again in the alerts.json file on your manager to ensure alerts are coming in real-time.

You can do this with tail -f /var/ossec/logs/alerts/alerts.json

Once this is confirmed, I would check that the Filebeat configuration file fetches the alerts.json file. This file is located in /etc/filebeat/filebeat.yml

It should start with something similar to this: 

# Wazuh - Filebeat configuration file
filebeat.modules:
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: false

setup.template.json.enabled: true
setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.template.json.name: 'wazuh'
setup.template.overwrite: true
setup.ilm.enabled: false

As you can see, the Wazuh module is enabled for the alerts.

Hope that helps!

Javier Castro Fernández

Director of Operations | Wazuh Inc.

www.wazuh.com

On Thu, Mar 30, 2023 at 10:25 AM Dennis Hidecker <dhid...@knoxcountyarc.com> wrote:

Good Morning,

It looks like my reply from yesterday didn't go through so I'm posting again.  :)

I ran the recommended commands and below are the results:

root@kc-wazuh-2:~# filebeat test output
elasticsearch: https://127.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 127.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
root@kc-wazuh-2:~# df -h
Filesystem                         Size  Used Avail Use% Mounted on
udev                                16G     0   16G   0% /dev
tmpfs                              3.2G  1.4M  3.2G   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv  2.0T  1.1T  829G  58% /
tmpfs                               16G  9.3M   16G   1% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
tmpfs                               16G     0   16G   0% /sys/fs/cgroup
/dev/loop0                         128K  128K     0 100% /snap/bare/5
/dev/loop2                          64M   64M     0 100% /snap/core20/1828
/dev/loop3                          56M   56M     0 100% /snap/core18/2714
/dev/loop4                          64M   64M     0 100% /snap/core20/1852
/dev/loop5                         165M  165M     0 100% /snap/gnome-3-28-1804/161
/dev/loop7                          82M   82M     0 100% /snap/gtk-common-themes/1534
/dev/loop6                          92M   92M     0 100% /snap/gtk-common-themes/1535
/dev/loop8                          50M   50M     0 100% /snap/snapd/18596
/dev/loop11                        347M  347M     0 100% /snap/wine-platform-runtime/340
/dev/loop10                         50M   50M     0 100% /snap/snapd/18357
/dev/loop12                        323M  323M     0 100% /snap/wine-platform-6-stable/19
/dev/sda2                          2.0G  209M  1.6G  12% /boot
/dev/loop14                         92M   92M     0 100% /snap/lxd/23991
/dev/loop13                         92M   92M     0 100% /snap/lxd/24061
tmpfs                              3.2G   32K  3.2G   1% /run/user/1000
/dev/loop15                         56M   56M     0 100% /snap/core18/2721
/dev/loop1                         347M  347M     0 100% /snap/wine-platform-runtime/341
root@kc-wazuh-2:~#

Also, I looked into the Wazuh-Indexer logs as you suggested and it looks like the problem did start on the 22nd.  Also, as I'm still receiving alert e-mails from the system even though I'm not seeing anything in the dashboard.  I've included the logs for the 22nd & 23rd for review.  Also, I realized that I didn't include any information on our system's setup:

* Wazuh 4.3.10

* App revision: 4311

* Virtual Machine

* All-In-One installation

* 8 Virtual Processors (Host has dual Xeon Gold 5218's)

* 45G RAM (Originally has 32G RAM but upped to 45G has I was troubleshooting to rule out ram issues.)

* Virtual Disk is currently 2TB but can expand as needed.

* Ubuntu 20.04.6 LTS

Thank-You,

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/ff43dd7a-f74d-4665-b380-8206918be9b2n%40googlegroups.com.