Wazuh not creating indices

[Albert Waweru]

Hello all, i noticed my setup is not wrtiting indices for a few days now and i have checked through the forum for potential solutions but i havent found one thats working yet. I have confirmed that the index is in healthy state, the filebeat module is able to communicate to the indexer and alerts are coming through the slack channel but not showing on the dashboard. Please help

[hasitha.u...@wazuh.com]

Hi Albert,

To verify and find the root cause of this issue, please share these command outputs to check further.

If you can access to Wazuh dashboard, then try to navigate to Index Management> Dev Tools

Use this command:

GET _cluster/health

If you want to check in CLI try this command.

curl -XGET -k -u admin:pass "https://localhost:9200/_cluster/health"

Please share the cluster health command output to check further.

Also, share the filebeat test output command output.

filebeat test output

 
Additionally, check the disk usage and the other resource usage of your Indexer node.

df -h

free -h

Top

Share the results with us to check further.

If the disk space is low try clearing some old unwanted data on the system.

Additionally, share the indexer log.

cat /var/log/wazuh-indexer/wazuh-cluster.log | grep -i -E "error|warn|crit"

Also, share the filbeat logs to check further.
cat /var/log/wazuh-indexer/wazuh-cluster.log | grep -i -E "error|warn"

cat /var/log/filebeat/filebeat | grep -i -E "error|warn"

Additionally, try restarting all services if any issues or errors you could not find.
systemctl restart wazuh-manager
systemctl restart wazuh-indexer

systemctl restart filebeat
systemctl restart wazuh-dashboard

Let me know the update on this.

Regards,
Hasitha Upekshitha

[Albert Waweru]

Hello Hasitha, please see the attached

cat /var/log/wazuh-indexer/wazuh-indexer-cluster.log | grep -i -E "error|warn|crit"

[2025-01-27T10:57:16,876][INFO ][o.o.n.Node               ] [node-1] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.security.manager=allow, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/var/log/wazuh-indexer/tmp, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/wazuh-indexer, -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.security.manager=allow, -Djava.util.concurrent.ForkJoinPool.common.threadFactory=org.opensearch.secure_sm.SecuredForkJoinWorkerThreadFactory, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=file:///etc/wazuh-indexer/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/usr/share/wazuh-indexer, -Dopensearch.path.conf=/etc/wazuh-indexer, -Dopensearch.distribution.type=deb, -Dopensearch.bundled_jdk=true]
[2025-01-27T10:57:17,216][WARN ][o.a.l.i.v.VectorizationProvider] [node-1] Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.
[2025-01-27T10:57:20,511][WARN ][o.o.s.OpenSearchSecurityPlugin] [node-1] File /etc/wazuh-indexer/opensearch-security/roles_mapping.yml has insecure file permissions (should be 0600)
[2025-01-27T10:57:20,512][WARN ][o.o.s.OpenSearchSecurityPlugin] [node-1] File /etc/wazuh-indexer/opensearch-security/internal_users.yml has insecure file permissions (should be 0600)
[2025-01-27T10:57:20,512][WARN ][o.o.s.OpenSearchSecurityPlugin] [node-1] File /etc/wazuh-indexer/opensearch-security/internal_users.yml.dpkg-old has insecure file permissions (should be 0600)
[2025-01-27T10:57:20,512][WARN ][o.o.s.OpenSearchSecurityPlugin] [node-1] File /etc/wazuh-indexer/opensearch-security/roles.yml has insecure file permissions (should be 0600)
[2025-01-27T10:57:23,314][WARN ][stderr                   ] [node-1] WARNING: A restricted method in java.lang.foreign.Linker has been called
[2025-01-27T10:57:23,315][WARN ][stderr                   ] [node-1] WARNING: java.lang.foreign.Linker::downcallHandle has been called by the unnamed module
[2025-01-27T10:57:23,315][WARN ][stderr                   ] [node-1] WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for this module
[2025-01-27T10:57:27,270][ERROR][o.o.p.c.j.GCMetrics      ] [node-1] MX bean missing: G1 Concurrent GC
[2025-01-27T10:57:28,824][WARN ][o.o.s.c.Salt             ] [node-1] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes
[2025-01-27T10:57:28,883][ERROR][o.o.s.a.s.SinkProvider   ] [node-1] Default endpoint could not be created, auditlog will not work properly.
[2025-01-27T10:57:28,885][WARN ][o.o.s.a.r.AuditMessageRouter] [node-1] No default storage available, audit log may not work properly. Please check configuration.
[2025-01-27T10:57:30,731][WARN ][o.o.s.p.SQLPlugin        ] [node-1] Master key is a required config for using create and update datasource APIs. Please set plugins.query.datasources.encryption.masterkey config in opensearch.yml in all the cluster nodes. More details can be found here: https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/datasources.rst#master-key-config-for-encrypting-credential-information
[2025-01-27T10:57:32,297][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,301][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,307][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,309][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,309][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,310][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,310][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,310][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,310][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,311][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,311][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,311][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,311][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,311][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,312][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,312][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,312][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,321][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,321][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,322][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,322][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,323][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,323][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,323][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,324][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,324][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,324][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,325][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,347][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,351][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,352][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,353][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,355][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,359][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,360][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,360][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,360][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,361][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,361][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,361][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,362][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:32,362][WARN ][o.o.p.c.ThreadPoolMetricsCollector] [node-1] Fail to read queue capacity via reflection
[2025-01-27T10:57:33,747][WARN ][o.o.g.DanglingIndicesState] [node-1] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2025-01-27T10:57:36,472][ERROR][o.o.s.l.BuiltinLogTypeLoader] [node-1] Failed loading builtin log types from disk!
        at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) [opensearch-cli-2.16.0.jar:2.16.0]
[2025-01-27T10:57:37,961][WARN ][o.o.p.c.s.h.ConfigOverridesClusterSettingHandler] [node-1] Config override setting update called with empty string. Ignoring.
[2025-01-27T10:57:37,983][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:38,074][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:38,855][WARN ][o.o.o.i.ObservabilityIndex] [node-1] message: index [.opensearch-observability/eFK4eKT4S2uq0l7AMI93wQ] already exists
[2025-01-27T10:57:38,960][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:38,984][WARN ][o.o.s.SecurityAnalyticsPlugin] [node-1] Failed to initialize LogType config index and builtin log types
[2025-01-27T10:57:39,068][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,071][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,071][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,071][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,071][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,072][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,072][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,077][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,078][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,079][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure No shard available for [org.opensearch.action.get.MultiGetShardRequest@2569e909] retrieving configuration for [ACTIONGROUPS, ALLOWLIST, AUDIT, CONFIG, INTERNALUSERS, NODESDN, ROLES, ROLESMAPPING, TENANTS, WHITELIST] (index=.opendistro_security)
[2025-01-27T10:57:39,550][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:39,709][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:39,716][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:39,724][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:39,901][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:40,286][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:40,901][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:41,473][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:41,667][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:41,732][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:41,737][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:41,742][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:41,750][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:42,153][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:42,384][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:42,742][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:43,005][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:43,201][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:43,505][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:43,639][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:44,232][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:44,237][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:44,240][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:44,244][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:44,401][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:44,862][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:45,117][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:45,335][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:45,606][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:45,733][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:45,852][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:46,108][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:46,279][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:46,514][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:46,732][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:46,736][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:46,739][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:46,743][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:46,928][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:47,227][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:47,434][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:47,620][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:47,752][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:48,150][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:48,253][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:48,566][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:48,851][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:49,019][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:49,182][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:49,274][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:49,290][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:49,295][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:49,299][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:49,447][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:49,679][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:49,870][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:50,116][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:50,548][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:50,728][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:51,199][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:51,731][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:51,735][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:51,738][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:51,742][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:51,745][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)
[2025-01-27T10:57:51,945][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:52,116][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:52,357][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:52,846][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:53,030][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:53,363][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:53,758][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:54,430][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:54,626][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:54,925][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:55,231][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:55,557][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T10:57:55,730][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T11:00:05,605][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T11:00:05,816][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T11:00:06,028][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set
[2025-01-27T11:00:06,426][WARN ][o.o.c.r.a.AllocationService] [node-1] Falling back to single shard assignment since batch mode disable or multiple custom allocators set

cat /var/log/filebeat/filebeat
2025-01-27T14:04:13.387+0300    INFO    instance/beat.go:645    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2025-01-27T14:04:13.402+0300    INFO    instance/beat.go:653    Beat ID: 7d6e96b3-277b-4369-aa77-6152b4ba2364
2025-01-27T14:04:13.412+0300    INFO    [index-management]      idxmgmt/std.go:184      Set output.elasticsearch.index to 'filebeat-7.10.2' as ILM is enabled.
2025-01-27T14:04:13.414+0300    INFO    eslegclient/connection.go:99    elasticsearch url: https://159.223.132.53:9200
2025-01-27T14:04:13.428+0300    INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:93     add_cloud_metadata: hosting provider type detected as digitalocean, metadata={"instance":{"id":"385783609"},"provider":"digitalocean","region":"nyc1"}
2025-01-27T14:04:13.452+0300    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.10.2

[Albert Waweru]

Can someone help me here..i have not had data showing up on the dashboard for two weeks now and i have tried to create indices manually and reindex the last index but still its not showing up on the dashboard.

[Albert Waweru]

Hello, can someone help me?

[hasitha.u...@wazuh.com]

Hi Albert,

First, we need to make sure Filebeat is reading the alert files and sending them to the indexer. To do so, you may use the lsof command to verify Filebeat is reading the file and check the Filebeat connection. Please, share the output of the following commands:
lsof /var/ossec/logs/alerts/alerts.json

Aslo share the full filebeat log filebeat log file to check further.

This one indicates that [ca-wazuh-indexer-1] has not been initialized.

[2025-01-27T10:57:49,299][ERROR][o.o.s.a.BackendRegistry  ] [node-1] Not yet initialized (you may need to run securityadmin)

To address this, you could try running the following command:
/usr/share/wazuh-indexer/bin/indexer-security-init.sh

Also let me know the your Wazuh version.
/var/ossec/bin/wazuh-control info

Let me know the update on this.

Regards,
Hasitha Upekshitha

[Albert Waweru]

Hello Hasitha,

I managed to fix the problem by reverting to an older version of the filebeat.yml file

[satish senapathi]

hi Albert and Hasitha,
iam having the same issue how to resolve it?