System inventory could be disabled or has a problem

[Lukas Fritzsche]

Hello there,

so apparently I forgot to disable auto-updates on my wazuh-machines and now have 4.13.

Well, I was on vacation for 4 weeks and am now back to seeing " System inventory could be disabled or has a problem" whenever viewing the inventory of an agent.

I use three seperate servers (one for dashboard, one for indexer, one for manager/server)

I have already checked:

- Certs on wazuh-server are all good (valid till 2034).

- Cluster health: YELLOW (already have deleted older indexes and moved the retention policy from 90d to 60d)

- configuration of ossec.conf (everything in order there)

Using the following command: 

curl -u admin:PASSWORD --cacert /etc/filebeat/certs/root-ca.pem --cert /etc/filebeat/certs/wazuh-server.pem --key /etc/filebeat/certs/wazuh-server-key.pem -X GET "https://INDEXER_IP/_cluster/health?pretty"

i got:

{
  "cluster_name" : "wazuh-indexer-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 321,
  "active_shards" : 321,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 32,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 90.93484419263456
}

Viewing the ossec.log using this command:

cat /var/ossec/logs/ossec.log | grep -iE "inventory|indexer|syscollector"

i got:

2025/09/24 00:02:55 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2025/09/24 00:03:06 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2025/09/24 01:03:07 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2025/09/24 01:03:18 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2025/09/24 02:03:18 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2025/09/24 02:03:30 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2025/09/24 03:03:31 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2025/09/24 03:03:43 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2025/09/24 04:03:44 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2025/09/24 04:03:56 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2025/09/24 05:03:57 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2025/09/24 05:04:07 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2025/09/24 05:29:28 wazuh-modulesd:syscollector: INFO: Stop received for Syscollector.
2025/09/24 05:29:28 wazuh-modulesd:syscollector: INFO: Module finished.
2025/09/24 05:29:32 indexer-connector: WARNING: Failed to sync agent '002' with the indexer.
[...] basically every agent listed here
2025/09/24 05:29:32 indexer-connector: WARNING: Failed to sync agent '040' with the indexer.
2025/09/24 05:29:32 wazuh-modulesd:inventory-harvester: INFO: Stopping inventory_harvester module.
2025/09/24 07:30:03 wazuh-modulesd:inventory-harvester: INFO: Loaded Inventory harvester module.
2025/09/24 07:30:13 wazuh-modulesd:inventory-harvester: INFO: Loaded Inventory harvester module.
2025/09/24 07:30:13 wazuh-modulesd:inventory-harvester: INFO: Starting inventory_harvester module.
2025/09/24 07:30:14 wazuh-modulesd:syscollector: INFO: Module started.
2025/09/24 07:30:14 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2025/09/24 07:30:14 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2025/09/24 07:30:14 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-packages-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:15 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-system-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:20 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-processes-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:21 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-ports-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:22 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-hotfixes-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:22 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-hardware-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:23 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-protocols-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:24 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-interfaces-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:24 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventor-networks-wazuh-server', retrying until the connection is successful.
2025/09/24 07:30:24 logger-helper: INFO: InventoryHarvesterFacade module started.
2025/09/24 07:31:30 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-vulnerabilities-wazuh-server', retrying until the connection is successful.

So here i am now, back from vacation with zero clue what's wrong. I know nobody of my colleagues worked with wazuh in the meantime.

I have also seen on Reddit I'm not the only one with this issue after upgrading to 4.13 but this (https://github.com/wazuh/wazuh/issues/32044#issuecomment-3312554521) also has not helped me.

I appreciate any help!

[Javier Sanchez Gil]

Hi Lukas Fritzsche,

It looks like the inventory itself is working (agents are collecting data), but the issue is that the manager cannot initialize the inventory indices in the indexer. This often happens when there’s a mismatch in the <indexer> block configuration inside ossec.conf, usually related to the certificate paths.

I’d suggest double-checking that the certificate paths under <indexer> match the ones Filebeat is using on your indexer node. You can use the example here as a reference: https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/indexer.html#configuration-example

Also, you can run the following command to verify which cert files are present and confirm your paths:

ll /etc/filebeat/certs/

That should help confirm whether the <indexer> block is pointing to the correct certificates.

[Lukas Fritzsche]

Sorry, apparently I only answered to you and not this thread, so here I go again:

Hi Javier Sanchez Gil,

thanks for your answer, I have checked:

<indexer> block in ossec.conf:

  <indexer>
    <enabled>yes</enabled>
    <hosts>
      <host>INDEXER_IP</host>
    </hosts>
    <ssl>
      <certificate_authorities>
        <ca>/etc/filebeat/certs/root-ca.pem</ca>
      </certificate_authorities>
      <certificate>/etc/filebeat/certs/wazuh-server.pem</certificate>
      <key>/etc/filebeat/certs/wazuh-server-key.pem</key>
    </ssl>
  </indexer>

using the command ll /etc/filebeat/certs/ :

total 20
dr-x------ 2 root root 4096 Sep 24 07:55 ./
drwxr-xr-x 4 root root 4096 Mai  8 11:36 ../
-r-------- 1 root root 1204 Nov 21  2024 root-ca.pem
-r-------- 1 root root 1704 Nov 21  2024 wazuh-server-key.pem
-r-------- 1 root root 1285 Nov 21  2024 wazuh-server.pem

So I am assuming everythings in order, right?

Again, this is very weird, no changes were made except the update to Wazuh 4.13 and now the inventory "does not work" properly.

[Javier Sanchez Gil]

Hi Lukas,

Just to confirm, after checking the certificate paths, is it still not working correctly?

If so, could you double-check whether your <indexer> block has the host defined like this:

<host>https://x.x.x.x:9200</host>

If it’s missing the https:// prefix and only appears as <host>x.x.x.x:9200</host>, that can cause connection errors with the Wazuh indexer.

It would also be helpful if you could run:

cat /var/ossec/logs/ossec.log | grep -iE "error|warn|crit|fatal"

to see if there are any additional details in the logs explaining why the IndexerConnector initialization is failing.

[Lukas Fritzsche]

Hi Javier,

so i have double checked.

The indexer does have the https:// prefix.

after running cat /var/ossec/logs/ossec.log | grep -iE "error|warn|crit|fatal" :

2025/10/01 02:32:20 wazuh-authd: WARNING: Duplicate name 'REDACTED', rejecting enrollment. Agent '048' key already exists on the manager.
2025/10/01 00:35:24 wazuh-remoted: WARNING: Agent key already in use: agent ID '048'
2025/10/01 00:35:35 wazuh-remoted: WARNING: Agent key already in use: agent ID '048'
2025/10/01 00:36:00 wazuh-remoted: WARNING: Agent key already in use: agent ID '048'
2025/10/01 01:07:01 wazuh-remoted: WARNING: Agent key already in use: agent ID '024'
2025/10/01 01:07:02 wazuh-remoted: WARNING: Agent key already in use: agent ID '023'
2025/10/01 01:07:11 wazuh-remoted: WARNING: Agent key already in use: agent ID '024'
2025/10/01 01:07:12 wazuh-remoted: WARNING: Agent key already in use: agent ID '023'
2025/10/01 01:07:19 wazuh-remoted: WARNING: Agent key already in use: agent ID '042'
2025/10/01 01:07:21 wazuh-remoted: WARNING: Agent key already in use: agent ID '024'
2025/10/01 01:07:22 wazuh-remoted: WARNING: Agent key already in use: agent ID '023'
2025/10/01 01:07:29 wazuh-remoted: WARNING: Agent key already in use: agent ID '042'
2025/10/01 01:07:31 wazuh-remoted: WARNING: Agent key already in use: agent ID '024'
2025/10/01 01:07:32 wazuh-remoted: WARNING: Agent key already in use: agent ID '023'
2025/10/01 01:07:39 wazuh-remoted: WARNING: Agent key already in use: agent ID '042'
2025/10/01 01:07:49 wazuh-remoted: WARNING: Agent key already in use: agent ID '042'
2025/10/01 07:14:08 wazuh-authd: WARNING: Duplicate name 'REDACTED, rejecting enrollment. Agent '001' key already exists on the manager.
2025/10/01 05:24:48 wazuh-remoted: WARNING: Agent key already in use: agent ID '169'
2025/10/01 05:27:33 wazuh-remoted: WARNING: Agent key already in use: agent ID '169'

So nothing there about IndexerConnector or anything :/

[Javier Sanchez Gil]

Hi Lukas,

It doesn’t seem to show anything new. As mentioned in this section of the Wazuh documentation about these errors: https://documentation.wazuh.com/current/upgrade-guide/troubleshooting.html#indexerconnector-initialization-failed please enable wazuh_modules.debug=2 inside /var/ossec/etc/local_internal_options.conf to see if we get more information and check for anything related to the indexer-connector.

On the other hand, please review that your /etc/filebeat/filebeat.yml configuration is correct: https://documentation.wazuh.com/current/user-manual/manager/indexer-integration.html#wazuh-indexer

[Lukas Fritzsche]

Hi Javier,

after enabling wazuh_modules.debug=2

using  cat /var/ossec/logs/ossec.log | grep -iE "error|warn|crit|fatal" i get a LOT of output, but the "interesting" one is probably:

2025/10/01 10:16:13 indexer-connector[13956] indexerConnector.cpp:847 at operator()(): WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventory-processes-wazuh-server', retrying until the connection is successful.
2025/10/01 10:16:14 indexer-connector[13956] indexerConnector.cpp:847 at operator()(): WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventory-ports-wazuh-server', retrying until the connection is successful.
2025/10/01 10:16:14 indexer-connector[13956] indexerConnector.cpp:847 at operator()(): WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventory-hotfixes-wazuh-server', retrying until the connection is successful.
2025/10/01 10:16:15 indexer-connector[13956] indexerConnector.cpp:847 at operator()(): WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventory-hardware-wazuh-server', retrying until the connection is successful.
2025/10/01 10:16:15 indexer-connector[13956] indexerConnector.cpp:847 at operator()(): WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventory-protocols-wazuh-server', retrying until the connection is successful.
2025/10/01 10:16:16 indexer-connector[13956] indexerConnector.cpp:847 at operator()(): WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventory-interfaces-wazuh-server', retrying until the connection is successful.
2025/10/01 10:16:16 indexer-connector[13956] indexerConnector.cpp:847 at operator()(): WARNING: IndexerConnector initialization failed for index 'wazuh-states-inventory-networks-wazuh-server', retrying until the connection is successful.

using  cat /var/ossec/logs/ossec.log | grep -iE "error|warn|crit|fatal|IndexerConnector" i get:

2025/10/01 10:21:07 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-system-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:21:15 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-processes-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:21:16 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-ports-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:21:16 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-hotfixes-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:21:17 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-hardware-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:21:17 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-protocols-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:21:18 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-interfaces-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:21:18 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-networks-wazuh-server': No available server. Retrying in 60 seconds.
2025/10/01 10:22:06 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-vulnerabilities-wazuh-server': No available server. Retrying in 60 seconds.

2025/10/01 10:22:06 indexer-connector[13956] indexerConnector.cpp:839 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-inventory-packages-wazuh-server': No available server. Retrying in 60 seconds.

and many of these:

2025/10/01 10:24:07 wazuh-modulesd:vulnerability-scanner[13956] scanOrchestrator.hpp:153 at operator()(): DEBUG: AgentReScanListException. Reason: Error executing rescan for multiple agents.
2025/10/01 10:24:07 wazuh-modulesd:vulnerability-scanner[13956] scanAgentList.hpp:243 at handleRequest(): DEBUG: Error executing query to fetch agent data for agents. Reason: DB query not synced.
2025/10/01 10:24:07 wazuh-modulesd:vulnerability-scanner[13956] scanOrchestrator.hpp:153 at operator()(): DEBUG: AgentReScanListException. Reason: Error executing rescan for multiple agents.
2025/10/01 10:24:07 wazuh-modulesd:vulnerability-scanner[13956] scanAgentList.hpp:243 at handleRequest(): DEBUG: Error executing query to fetch agent data for agents. Reason: DB query not synced.
2025/10/01 10:24:07 wazuh-modulesd:vulnerability-scanner[13956] scanOrchestrator.hpp:153 at operator()(): DEBUG: AgentReScanListException. Reason: Error executing rescan for multiple agents.

unfortunately i cant paste the entire output, as i cant redact all the private information that would be exposed.

I'm still curious how this happened after not changing any configuration and updating the wazuh servers.

The "IT Hygiene" was not there before updating and that is the error im getting now. 

The same error is shown if I go to inspect an agent in the overview.

The "Inventory" menu that was available when inspecting an agent is gone so I'm assuming that all has now moved to "IT Hygiene" with the update.

A configuration issue is basically impossible as ZERO changes where made before and after the update, assuming wazuh itself has not changed the way it expects configuration with the newest update.

But that contradicts the warning-logs, so i am clueless here.

[Lukas Fritzsche]

Hi Javier,

i also tested:

filebeat test output :

elasticsearch: https://REDACTED...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: REDACTED
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.2
    dial up... OK
  talk to server... OK
  version: 7.10.2

[Lukas Fritzsche]

Hi Javier, 

I've just confirmed in 4.13 Wazuh added "IT Hygiene" (https://documentation.wazuh.com/current/release-notes/release-4-13-0.html)

When comparing the App Settings from the github changes and my dashboard (github.com/wazuh/wazuh-dashboard-plugins/pull/7368)

I can see that there is no index for the inventory, obviously, but also the settings for IT Hygiene do not exist at all in the Dashboard Management -> App Settings.

Might this be a faulty Wazuh update?

[Lukas Fritzsche]

Hi Javier,

sorry for the spam! :D

I have also tried:

GET /syscollector/001/hardware (in the Dashboard -> Server management -> Dev Tools)

and that yields UP TO DATE (just from an hour ago) results, so the syscollector scans happen and are successfully forwarded to the indexer.

Apparently the dashboard just can not display them i'm assuming?

Maybe because there is no index created for the inventory?

Could that be because there are only 32 remaining shards? (Why is it only 333 total available shards tho, i thought it was 1000 per indexer? I have also deleted 30 days of the total 90 days of indexes we keep and updated the policies but the available shards have barely gone up)

I'm just assuming here tho and really trying to get this fixed.

Sorry again for the spam, I should have just gathered all that and put it into one answer.

I really appreciate your time and help!

Greetings 

Lukas

[Javier Sanchez Gil]

Hi Lukas,

Just to confirm, could you verify whether all central components (manager, indexer, and dashboard) have been updated to 4.13, not only the manager? A version mismatch between components can cause issues like the one you’re seeing with the inventory indices.

Also, please note that the upgrade process includes some manual steps (not handled automatically by the package upgrade). These steps involve things like:

- Backing up and reapplying indexer security configuration.

- Updating the <indexer> block in ossec.conf with the correct hosts, certificates, and keystore credentials.

- Downloading and applying the new Filebeat module and template.

- Reapplying dashboard configuration and re-importing saved objects.

I’d recommend carefully reviewing the official upgrade guide and checking each section to ensure all manual tasks were completed:
https://documentation.wazuh.com/current/upgrade-guide/upgrading-central-components.html

This will help rule out any skipped steps and make sure all components are aligned with version 4.13.

[Lukas Fritzsche]

Hi Javier,

sorry for the late reply.

I have now done the upgrade manually to 4.13.1.

Now the IT-Hygiene tab is atleast "loading". But a lot of the fields show as "error". (see the picture) [update after some minutes: they now seem to be loading in step by step]

see in "wazuh-picture1.png"

Also when clicking the "Software" Tab i get an internal server and then the dashboard breaks: 

see in "wazuh-picture2.png"

It's also prompting me to "Set up API credentials", the wazuh-wui API Connection is successfull tho in wazuh-dashboard -> Dashboard Management -> Server APIs.

Was there something new added here for IT-Hygiene?

I did everything as described in the update manual you linked me.

The longer I wait and retry it seems to be getting better, but after reviewing every single tab in IT-Hygiene i get: (It feels like a rate-limit like "too many requests")

see in "wazuh-picture3.png"

[Javier Sanchez Gil]

Hi Lukas Fritzsche,

During the manual upgrade, you didn’t encounter any errors, right?

I’m going to ask you to run the following commands again to look for possible causes of the issue:

Wazuh indexer:
cat /var/log/wazuh-indexer/wazuh-cluster.log | grep -i -E "error|warn"

Wazuh manager:
cat /var/log/filebeat/filebeat | grep -i -E "error|warn"
cat /var/ossec/logs/ossec.log | grep -i -E "error|warn"

Wazuh dashboard:
journalctl -u wazuh-dashboard | grep -i -E "error|warn"

Also, please follow the Dashboard troubleshooting steps described here to see if we can identify the root cause: https://documentation.wazuh.com/current/user-manual/wazuh-dashboard/troubleshooting.html#troubleshooting

[Lukas Fritzsche]

Hi Javier,

so I've used wazuh for the last days now and the errors seamingly have fixed themselves. Honestly no clue why. But no during the manual upgrade I haven't experienced any errors. Your provided  commands yield A LOT of results on my console soo i probably can't post them here (since they also contain agent logs).

But well, it seems, the problem solved itself with your help in upgrading all the systems manually.

I really appreciate the help and time!