Vulnerability detection seems to be disabled or has a problem

[Juliano Nascimento]

Hello everyone! I hope you're all doing well!

After upgrading my single-node server from version 4.4.5 to version 4.8.2, we're receiving the warning ‘Vulnerability detection seems to be disabled or has a problem.’

We’ve tried the following actions without success:

• Checking the /var/ossec/etc/ossec.conf file to confirm the addition of the <vulnerability-detection> parameters.

• Disabling <vulnerability-detection> by setting <enable> to ‘no’. The same for <indexer>, followed by a reboot, re-enabling, and another reboot.

• Adjusting the indexer to match the address specified in the /etc/filebeat/filebeat.yml file.

• Evaluating the certificates, and they appear to be correctly configured.

• Forcing credentials again using the following commands:

/var/ossec/bin/wazuh-keystore -f indexer -k username -v admin

/var/ossec/bin/wazuh-keystore -f indexer -k password -v <password>

Output from the command: curl -u admin:tEfOZiksFuDBTR09ipkf80zUvqJ6SXjY --cacert /etc/filebeat/certs/root-ca.pem --cert /etc/filebeat/certs/filebeat.pem --key /etc/filebeat/certs/filebeat-key.pem -X GET "https://127.0.0.1:9200/_cluster/health"

Output from the command:  curl -u "admin:tEfOZiksFuDBTR09ipkf80zUvqJ6SXjY" --cacert /etc/filebeat/certs/root-ca.pem --cert /etc/filebeat/certs/filebeat.pem --key /etc/filebeat/certs/filebeat-key.pem -X GET "https://127.0.0.1:9200"

Output from the command: openssl verify -verbose -CAfile /etc/filebeat/certs/root-ca.pem /etc/filebeat/certs/filebeat.pem

Output from the command: openssl rsa -check -noout -in /etc/filebeat/certs/filebeat-key.pem

ossec.log file attached

Server resources:

Can someone give us some guidance? We’re at a loss on what to do next =/

[Stuti Gupta]

Hi Jullian 

Can you please provide the ossec.log 
Also, your cluster health is red that message indicates that there are unassigned shards in your cluster. The module won't make any request to the Wazuh Indexer cluster unless this clsuter health is green. So to do that please delete the unassigned shards
1. Check all Indexer Unassigned Shards: You can check the name of the unassigned shards and its current state by using the command  curl -XGET -k -u admin:admin https://localhost:9200/_cat/shards?h=index,shards,state,prirep,unassigned.reason | grep UNASSIGNED
2. . Delete Unassigned Shards: You can use the command: curl -k -XGET -u user:pass "https://<elasticsearxch>:9200/_cat/shards" | grep UNASSIGNED | awk '{print $1}' | xargs -i curl -k -XDELETE -u user:pass "https://<indexer_ip>:9200/{}"  

You can also refer to https://documentation.wazuh.com/current/upgrade-guide/troubleshooting.html#vulnerability-detection-seems-to-be-disabled-or-has-a-problem

Hope to hear from you soon 

[Juliano Nascimento]

Hi Stuti,

I sincerely thank you in advance for your guidance!

I have attached my ossec.log file.

Here is the output of the command: curl -XGET -k -u admin:admin https://localhost:9200/_cat/shards?

I tried to proceed with the removal, but I'm getting the following response:

Here is the output of this command: curl -XGET -k 'https://127.0.0.1:9200/_cluster/allocation/explain?pretty' -u admin:password


Any idea what it could be?  

[Juliano Nascimento]

  I also receive errors from wazuh-modulesd:vulnerability-scanner indicating that the database query is not synchronized.

  

[Stuti Gupta]

Hi 

The issue might have been caused by the cluster previously being in yellow state. It might have also been caused by the DBs getting messed up during the upgrade.
Now that it is in green state lets remove the old DBs and restart the services so fresh ones are created:
Remove the DB files rm -rf /var/ossec/queue/db/*
Restart the wazuh manager and indexer
systemctl restart wazuh-manager
systemctl restart wazuh-indexer
Once you have restarted, give it a few mins then observe if the issue persists

[Juliano Nascimento]

Hello!

Thank you very much for your response. I removed the database files and restarted the services as suggested. However, it didn't work; I still receive the warning 'Vulnerability detection seems to be disabled or has a problem.' :'(

I would like to point out that my cluster has never entered the green status and remains in the yellow status. There are 9 unassigned_shards that I cannot proceed with removing, and I receive the following message when I try to delete them:

  This is recorded in the logs of the wazuh-cluster.

  

  Any idea how I can remove these shards or give my admin user permission to proceed with their removal?  

[Stuti Gupta]

Hi Juliano

Can you please share the following logs and information?

Please share the ossec.log related wazuh-states-vulnerabilities that you can run command like: cat /var/ossec/logs/ossec.log | grep wazuh-states-vulnerabilities
Also, share the filebeat logs 

So we can get more information related to this error and help you accordingly.

Hope to hear from you soon

[Juliano Nascimento]

Hello!

Sure, I have attached the logs collected from the following destinations below.

cat /var/ossec/logs/ossec.log and /var/log/filebeat

I'm available if you need anything.  

[Stuti Gupta]

There is an error in ossec.log:
 indexerConnector.cpp:474 at operator()(): DEBUG: Unable to initialize IndexerConnector for index 'wazuh-states-vulnerabilities-socmelnick': No available server. Retrying in 60 seconds.

Please make sure the credentials you are using in the following commands:

/var/ossec/bin/wazuh-keystore -f indexer -k username -v admin
/var/ossec/bin/wazuh-keystore -f indexer -k password -v <password>

You can use this command to verify the certificate paths, names, and indexer ip: curl -u <user>:<pass> --cacert <path.pem> --cert <path-client.pem> --key <path-client-key.pem> -X GET "https://<IP>:9200/_cluster/health" After that, save the configuration and restart the manager/cluster using the command: systemctl restart wazuh-manager 

Hope this helps

[Juliano Nascimento]

Hello Stuti,

I forced the credentials again using the commands you mentioned and restarted the Wazuh Manager, but I’m still receiving the indexerConnector error.

I also noticed this warning: ‘The SSL peer certificate or remote SSH key was not OK.’

Do you know if this is related?

I validated the certificate paths, and they are correctly configured in the filebeat.yml and ossec.conf files.

I'm available if you need anything.

[Juliano Nascimento]

Hello,

I took the opportunity to update my Wazuh from version 4.8 to version 4.9, hoping that my Vulnerability Detection module would resume normal operation. However, I am still receiving the error 'Vulnerability detection seems to be disabled or has a problem.'

My cluster is still in yellow status:

I have UNASSIGNED shards:

And I am still unable to delete them, receiving the following permission error:

Could some kind soul help me?
Any information would be greatly appreciated!
Thank you very much!  

[Stuti Gupta]

Hi  Juliano

The ssl error is related to the certificate Please check if the wazuh0-indexer and filebeat certs are generated correctly. to verify the certification authority, please run the commands below and ensure the Issuer is Wazuh. For the key we need to ensure that the public key contained in the private key and the certificate must be the same:

Wazuh Indexer:
openssl x509 -in /etc/wazuh-indexer/certs/admin.pem -text -noout
openssl x509 -in /etc/wazuh-indexer/certs/root-ca.pem -text -noout
openssl x509 -in /etc/wazuh-indexer/certs/indexer.pem -text -noout
openssl rsa -in /etc/wazuh-indexer/certs/indexer-key.pem -pubout

Filebeat:
openssl rsa -in /etc/filebeat/certs/filebeat-key.pem -pubout
openssl x509 -in /etc/filebeat/certs/root-ca.pem -text -noout
openssl x509 -in /etc/filebeat/certs/filebeat.pem -text -noout

In case it is not correct and the output is not ok. then you need to regenerate and configured the certs again.https://documentation.wazuh.com/current/installation-guide/wazuh-indexer/step-by-step.html#certificates-creation

Hope this helps

[Sebastian Falcone]

Hi, I should intervene because the answers given by Sluti are not okay

This error is just related to the cluster's health, the Vulnerability Module is only able to index alerts when its status is green. Yellow or red status will prevent it from indexing

Given that you already identified the cause of the yellow state (unassigned shards):


I would suggest you follow the Elastiseach guide on how to fix it 

[Juliano Nascimento]

Hello, Sebastian Pleasure to meet you! Thank you very much for your contribution.

I did some checks based on the link you shared, but I wasn't very successful in resolving the status of my cluster.

When I try to delete these unassigned shards, I receive a warning that my admin user does not have permission to perform this action.  

Have you ever experienced this at any point? Would you know how to guide me in removing these shards? I don't mind losing them.

Thank you very much for your help and attention.

[Juliano Nascimento]

  Adding to the post above, I am providing additional evidence to help resolve my issue, as outlined below:

  

[Stuti Gupta]

Hi  Juliano

Thank you  Sebastian pointing this out. This pattern corresponds to system indices, which are protected in this case you can remove the replica by changing the number of replicas to 0. To do you can do the following solutions:

Solutions1:
You need to change the plugins.security.system_indices.enabled: true to false in /etc/wazuh-indexer/opensearch.yml
Restart the wazuh-indexer using the command: systemctl restart wazuh-indexer.
Then from the wazuh interface UI. You need to change the replica no. For that Go to Index management >> Index Management > Indexes > click on the unassigned indices. 
Finally, change the Number of replicas to 0 and click on save.

Solution2: Through CLI 
Create index templates for all opensearch system indices and set the number of replicas to 0 (https://opensearch.org/docs/1.2/im-plugin/ism/settings/#audit-history-indices):
Then go to /etc/wazuh-indexer/certs, then run the following command change the name <opendistro-ism-managed-index-history-*"> to unassigned shards:
# curl --key admin-key.pem --cert admin.pem  --insecure -XPUT https://127.0.0.1:9200/_index_template/ism_history_indices -H 'Content-Type: application/json' -d'
{
  "index_patterns": [
".opendistro-ism-managed-index-history-*"
  ],
  "template": {
"settings": {
  "number_of_shards": 1,
  "number_of_replicas": 0
}
  }
}'

To more about this, you can follow this Google community: https://groups.google.com/g/wazuh/c/tgaabFMiAL8
Note : Make sure to change plugins.security.system_indices.enabled:  to true again

Hope this helps with unassigned shards. Once its solved you can see the vulnerabilities 

[Stuti Gupta]

For your convenience I am adding these

[Juliano Nascimento]

  Hello everyone!
Just stopping by to thank you all for the support. My vulnerability dashboard is now working correctly.  

I followed the steps you provided, Stuti, and managed to adjust the replicas of some shards, while I had to delete the others. I had to set the replica value to 1 for my vulnerability template. I'm not sure if that enables the template, but it was previously set to 0.

I'm very grateful for all the help you’ve provided!