Re: Archive Logs not visible in dashboard

[Subash Ponnuswamy]

--

Regards,

SUBASH P

| www.mafiree.com

On Fri, Oct 17, 2025 at 4:14 PM Subash Ponnuswamy <suba...@mafiree.com> wrote:

Hi Team,

In the wazuh dashboard, after a restart, I'm not able to view the archive logs. The count is increasing in wazuh-archives, but the logs are not visible after a system restart.

The wazuh-alerts index is working fine. Any help is appreciated.

I have tried restarting wazuh-manager, wazuh-indexer and filebeat.

Filebeat logs

2025-10-17T16:02:21.575+0530    ERROR   [elasticsearch] elasticsearch/client.go:224     failed to perform any bulk index operations: Post "https://127.0.0.1:9200/_bulk": EOF
2025-10-17T16:02:21.578+0530    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2025-10-17T16:02:21.578+0530    INFO    [publisher]     pipeline/retry.go:223     done
2025-10-17T16:02:22.952+0530    ERROR   [publisher_pipeline_output]     pipeline/output.go:180  failed to publish events: Post "https://127.0.0.1:9200/_bulk": EOF
2025-10-17T16:02:22.952+0530    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2025-10-17T16:02:22.952+0530    INFO    [publisher]     pipeline/retry.go:223     done
2025-10-17T16:02:22.952+0530    INFO    [publisher_pipeline_output]     pipeline/output.go:143  Connecting to backoff(elasticsearch(https://127.0.0.1:9200))
2025-10-17T16:02:25.723+0530    ERROR   [publisher_pipeline_output]     pipeline/output.go:154  Failed to connect to backoff(elasticsearch(https://127.0.0.1:9200)): Get "https://127.0.0.1:9200": dial tcp 127.0.0.1:9200: connect: connection refused
2025-10-17T16:02:25.725+0530    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2025-10-17T16:02:25.725+0530    INFO    [publisher]     pipeline/retry.go:223     done
2025-10-17T16:02:25.728+0530    INFO    [publisher_pipeline_output]     pipeline/output.go:145  Attempting to reconnect to backoff(elasticsearch(https://127.0.0.1:9200)) with 1 reconnect attempt(s)
2025-10-17T16:02:33.166+0530    ERROR   [publisher_pipeline_output]     pipeline/output.go:154  Failed to connect to backoff(elasticsearch(https://127.0.0.1:9200)): Get "https://127.0.0.1:9200": dial tcp 127.0.0.1:9200: connect: connection refused
2025-10-17T16:02:33.166+0530    INFO    [publisher_pipeline_output]     pipeline/output.go:145  Attempting to reconnect to backoff(elasticsearch(https://127.0.0.1:9200)) with 2 reconnect attempt(s)
2025-10-17T16:02:33.166+0530    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2025-10-17T16:02:33.166+0530    INFO    [publisher]     pipeline/retry.go:223     done
2025-10-17T16:02:44.871+0530    ERROR   [publisher_pipeline_output]     pipeline/output.go:154  Failed to connect to backoff(elasticsearch(https://127.0.0.1:9200)): Get "https://127.0.0.1:9200": dial tcp 127.0.0.1:9200: connect: connection refused
2025-10-17T16:02:44.871+0530    INFO    [publisher_pipeline_output]     pipeline/output.go:145  Attempting to reconnect to backoff(elasticsearch(https://127.0.0.1:9200)) with 3 reconnect attempt(s)
2025-10-17T16:02:44.871+0530    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2025-10-17T16:02:44.871+0530    INFO    [publisher]     pipeline/retry.go:223     done
2025-10-17T16:02:44.917+0530    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.10.2
2025-10-17T16:02:44.922+0530    INFO    template/load.go:97     Template wazuh already exists and will not be overwritten.
2025-10-17T16:02:44.922+0530    INFO    [index-management]      idxmgmt/std.go:298      Loaded index template.
2025-10-17T16:02:44.929+0530    INFO    [publisher_pipeline_output]     pipeline/output.go:151  Connection to backoff(elasticsearch(https://127.0.0.1:9200)) established
^C
admin@wazuh:~$ sudo systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
     Loaded: loaded (/usr/lib/systemd/system/filebeat.service; enabled; preset: enabled)
     Active: active (running) since Fri 2025-10-17 15:57:07 IST; 12min ago
       Docs: https://www.elastic.co/products/beats/filebeat
   Main PID: 6901 (filebeat)
      Tasks: 14 (limit: 18619)
     Memory: 1.0G (peak: 1.0G swap: 2.0M swap peak: 2.2M)
        CPU: 2min 54.066s
     CGroup: /system.slice/filebeat.service
             └─6901 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --p>

Oct 17 15:57:07 wazuh systemd[1]: Started filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch..

Wazuh indexer logs

[2025-10-17T16:02:37,459][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-archives-4.x-2025.10.01/-HfBGxquQvuRv5owr8SCdw]
[2025-10-17T16:02:37,501][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,523][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,534][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,543][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,555][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.30/B7al-uJuRzSwLAmd2PkXIg]
[2025-10-17T16:02:37,559][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.opendistro-alerting-alert-history-2025.09.30-1/9PbVr0iaRamgdknwhr43PA]
[2025-10-17T16:02:37,562][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.opendistro-alerting-alerts/cDxphB4lTUKfrD0JH_XmIA]
[2025-10-17T16:02:37,570][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,584][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,595][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,606][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,648][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-archives-4.x-2025.09.30/6-pvYqmOQoKhKpmbZh9hMA]
[2025-10-17T16:02:37,659][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,693][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,712][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.29/uVDNxo72RA27F0HHWjxVzA]
[2025-10-17T16:02:37,719][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,740][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,752][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,765][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-archives-4.x-2025.09.29/Mv88kPf3Ts6F83D40gjnJg]
[2025-10-17T16:02:37,775][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,839][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,851][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,864][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,877][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-statistics-2025.40w/Pqq42KfITy2m2YlmqhIcEw]
[2025-10-17T16:02:37,880][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.28/SfMRV3e_RIqUrenvLJ431w]
[2025-10-17T16:02:37,884][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-monitoring-2025.40w/FiaO59byT1-s20pZScdaOQ]
[2025-10-17T16:02:37,891][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,920][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,931][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,945][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-archives-4.x-2025.09.28/gJu2ctCdSWyNM6y5gmHubw]
[2025-10-17T16:02:37,954][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:37,995][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,013][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,024][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,034][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,046][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.27/QbZDEWBIT8OU1T0kFdJp2w]
[2025-10-17T16:02:38,057][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,088][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,099][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,146][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-archives-4.x-2025.09.27/yHotjy73T4qC2AYHCzxqZA]
[2025-10-17T16:02:38,157][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,190][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,199][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,209][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.26/nywe6gGISQqgnbM1v1yZoA]
[2025-10-17T16:02:38,213][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-archives-4.x-2025.09.26/xq4aH_giSpyDjCMq-a038g]
[2025-10-17T16:02:38,223][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,282][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,292][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,302][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-archives-4.x-2025.09.25/LVR9gNq8S3ewQ2OkKGh9uw]
[2025-10-17T16:02:38,312][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,356][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,366][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,379][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.25/Xe5ZuUHpQxiABZOqZhDtMQ]
[2025-10-17T16:02:38,389][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,498][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,511][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,523][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.opendistro-reports-definitions/o5On7XPURg-0qZnHGgMbzA]
[2025-10-17T16:02:38,523][INFO ][o.o.j.JobSchedulerPlugin ] [node-1] JobSweeper started listening to operations on index .opendistro-reports-definitions
[2025-10-17T16:02:38,525][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-000002/-v7UiW2wTrmvS7QIRNMNRg]
[2025-10-17T16:02:38,526][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-000001/HRIvGvTFTR2zwaFGmvs72w]
[2025-10-17T16:02:38,527][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.opendistro-reports-instances/Jx4YXJANS1q0H5gT8H9Bew]
[2025-10-17T16:02:38,535][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,546][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,557][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,571][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.opensearch-notifications-config/-STACjDPS36v8u1kEdQBVw]
[2025-10-17T16:02:38,575][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.opendistro-ism-config/ZBQdIISTSd2g1lPOamxMQA]
[2025-10-17T16:02:38,575][INFO ][o.o.j.JobSchedulerPlugin ] [node-1] JobSweeper started listening to operations on index .opendistro-ism-config
[2025-10-17T16:02:38,582][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.24/AZBO9TZfTty233FZUXDCCA]
[2025-10-17T16:02:38,589][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-statistics-2025.39w/YqXTtKTvRaqfgJI3r7G38A]
[2025-10-17T16:02:38,600][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,652][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,692][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,709][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-monitoring-2025.39w/mf3HGLnQQ7KPbcxu_7RFoA]
[2025-10-17T16:02:38,711][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-statistics-2025.37w/d2ikQyRCSbmNQUdrHKEQLw]
[2025-10-17T16:02:38,719][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,736][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,747][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,757][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-monitoring-2025.37w/BJIXc8jFQC6KaUB6p0JNMg]
[2025-10-17T16:02:38,759][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[wazuh-alerts-4.x-2025.09.14/zzhDy2lqRRuvhubHOOnCpA]
[2025-10-17T16:02:38,801][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,814][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,824][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,834][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.opensearch-observability/DPsn3LoyTSKy6THwJVJb9w]
[2025-10-17T16:02:38,836][INFO ][o.o.p.PluginsService     ] [node-1] PluginService:onIndexModule index:[.kibana_1/QVHbdOaOQMmGrKTQfr2K_w]
[2025-10-17T16:02:38,841][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,853][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:02:38,861][INFO ][o.o.c.r.a.AllocationService] [node-1] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.kibana_1][0]]]).
[2025-10-17T16:02:38,870][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [node-1] Detected cluster change event for destination migration
[2025-10-17T16:07:33,641][INFO ][o.o.j.s.JobSweeper       ] [node-1] Running full sweep
[2025-10-17T16:07:33,795][INFO ][o.o.i.i.PluginVersionSweepCoordinator] [node-1] Canceling sweep ism plugin version job

[Subash Ponnuswamy]

[Dennis Ariel Gamboa Veliz]

Hi Subash,

Before replicating the issue on my side, I would like to gather a few details about your environment to better understand the situation and confirm the conditions where the problem occurs.

Could you please provide the following information?

1. Is your environment configured as a cluster(manager + workers) or a single-node setup?
2. What Wazuh version are you currently running?
   

Based on the initial analysis of your logs, the issue seems related to a temporary indexing inconsistency after the system reboot, possibly involving the wazuh-archives.

After a reboot, the alias might not correctly point to the active index, preventing the dashboard from displaying the archive logs, even though the index count continues increasing.

To address this, please try the following steps directly from the Wazuh Dashboard:

1. Remove the existing alias wazuh-archives
2. Create a new alias pointing to the latest wazuh-archives-* index

Once the new alias is created, please wait a few minutes and check if the archived logs appear correctly in the dashboard.

If the issue persists after performing these steps, please send me the following command outputs so I can analyze them in detail:

1. curl -k -u admin:admin https://127.0.0.1:9200/_cat/indices?v | grep wazuh-archives
2. curl -k -u admin:admin https://127.0.0.1:9200/_cat/aliases?v | grep wazuh-archives
3. curl -k -u admin:admin https://127.0.0.1:9200/_cluster/health?pretty

In the meantime, I'll be working on replicating your issue in a test environment to verify if the same behavior occurs after a reboot.
Please feel free to reply if you face any difficulties. I'll assist you as soon as possible

Best regards,
Dennis Gamboa

[Subash Ponnuswamy]

Hi Dennis,

This seems odd. One of the agents running in a Mac Studio has sent a large number of logs to the server, and once I stop the agent on the Mac Studio automatically after 15-30 minutes, the logs are visible in the dashboard.

I have started the agent again, but the issue didn't recur until now.