Hi all,
I have noticed that after updating my Ubuntu’s agents to release 4.0.4 config are not sync’ed in a cluster deployment. Example:
root@inverness:/var/ossec/etc/shared/rhel# agent_control -i 013
Wazuh agent_control. Agent information:
Agent ID: 013
Agent Name: edinburgh.lab.uxdom.org
IP address: 172.22.55.10
Status: Active
Operating system: Linux |edinburgh |5.8.0-40-generic |#45~20.04.1-Ubuntu SMP Fri Jan 15 11:35:04 UTC 2021 |x86_64
Client version: Wazuh v4.0.4
Configuration hash: 13eb20c2f8717608cb9e08ade77a2abf
Shared file hash: e65d07f9743cbf61fad1132498ccaead
Last keep alive: 1611508761
Syscheck last started at: Unknown
Syscheck last ended at: Unknown
Rootcheck last started at: Unknown
root@inverness:/var/ossec/etc/shared/ubuntu# md5sum merged.mg
28d9b410ff4d9f31c6d663841239283f merged.mg
But this is not occurs with RHEL based agents ….Any idea?
Regards,
C. L. Martinez
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/99B42740-CE79-4AD0-801E-83DD34830AFD%40outlook.com.
Good morning Selu.
Regarding your questions:
Ubu01.mydomain.org is the agent with problems …
Output of grep -iE "err|warn" /var/ossec/logs/ossec.log
2021/01/25 07:03:28 ossec-logcollector: ERROR: (1103): Could not open file '/tmp/fw.log' due to [(2)-(No such file or directory)].
2021/01/25 07:03:28 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/nginx/error.log'.
2021/01/25 07:03:45 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 07:18:46 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 07:33:47 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 07:48:48 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 08:03:52 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 08:18:53 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 08:33:54 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 08:48:55 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 09:03:56 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 09:18:57 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 09:33:58 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 09:48:59 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 10:04:02 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
2021/01/25 10:19:03 wazuh-modulesd:vulnerability-detector: ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.
Output of /var/ossec/bin/cluster_control -a
ID NAME IP STATUS VERSION NODE NAME
000 wazuhmaster.mydomain.org 127.0.0.1 active Wazuh v4.0.4 wazuhmaster.mydomain.org
001 obsd01.mydomain.org 172.22.55.28 disconnected Wazuh v4.0.3 wazuhwork01.mydomain.org
002 obsd02.mydomain.org 172.22.55.29 disconnected Wazuh v4.0.3 wazuhwork01.mydomain.org
003 obsd03.mydomain.org 172.22.56.6 disconnected Wazuh v4.0.3 wazuhwork01.mydomain.org
010 win2k19dc.msft.uxdom.org 172.22.61.6 disconnected Wazuh v4.0.4 wazuhwork01.mydomain.org
012 rhelsrv01.mydomain.org 172.22.55.5 active Wazuh v4.0.4 wazuhwork01.mydomain.org
013 ubu01.mydomain.org 172.22.55.10 active Wazuh v4.0.4 wazuhwork01.mydomain.org
014 rhelsrv02.mydomain.org 172.22.59.5 disconnected Wazuh v4.0.4 wazuhmaster.mydomain.org
Output of tail -n100 /var/ossec/logs/cluster.log
021/01/25 10:49:06 INFO: [Client wazuhwork01.mydomain.org] [Main] Finished sending information to wazuh-db (1 chunks sent).
2021/01/25 10:49:07 INFO: [Worker wazuhwork01.mydomain.org] [Main] Permission to synchronize granted
2021/01/25 10:49:07 INFO: [Worker wazuhwork01.mydomain.org] [Main] Compressing files
2021/01/25 10:49:07 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sending compressed file to master
2021/01/25 10:49:07 INFO: [Worker wazuhwork01.mydomain.org] [Main] Worker files sent to master
2021/01/25 10:49:07 INFO: [Worker wazuhwork01.mydomain.org] [Main] The master has verified that the integrity is right.
2021/01/25 10:49:16 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting agent-info sync process.
2021/01/25 10:49:16 INFO: [Client wazuhwork01.mydomain.org] [Main] Obtaining data to be sent to master's wazuh-db.
2021/01/25 10:49:16 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting to send information to wazuh-db.
2021/01/25 10:49:16 INFO: [Client wazuhwork01.mydomain.org] [Main] Finished sending information to wazuh-db (0 chunks sent).
2021/01/25 10:49:16 INFO: [Worker wazuhwork01.mydomain.org] [Main] Permission to synchronize granted
2021/01/25 10:49:16 INFO: [Worker wazuhwork01.mydomain.org] [Main] Compressing files
2021/01/25 10:49:16 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sending compressed file to master
2021/01/25 10:49:16 INFO: [Worker wazuhwork01.mydomain.org] [Main] Worker files sent to master
2021/01/25 10:49:16 INFO: [Worker wazuhwork01.mydomain.org] [Main] The master has verified that the integrity is right.
2021/01/25 10:49:25 INFO: [Worker wazuhwork01.mydomain.org] [Main] Permission to synchronize granted
2021/01/25 10:49:25 INFO: [Worker wazuhwork01.mydomain.org] [Main] Compressing files
2021/01/25 10:49:25 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sending compressed file to master
2021/01/25 10:49:25 INFO: [Worker wazuhwork01.mydomain.org] [Main] Worker files sent to master
2021/01/25 10:49:25 INFO: [Worker wazuhwork01.mydomain.org] [Main] The master has verified that the integrity is right.
2021/01/25 10:49:26 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting agent-info sync process.
2021/01/25 10:49:26 INFO: [Client wazuhwork01.mydomain.org] [Main] Obtaining data to be sent to master's wazuh-db.
2021/01/25 10:49:26 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting to send information to wazuh-db.
2021/01/25 10:49:26 INFO: [Client wazuhwork01.mydomain.org] [Main] Finished sending information to wazuh-db (0 chunks sent).
2021/01/25 10:49:34 INFO: [Worker wazuhwork01.mydomain.org] [Main] Permission to synchronize granted
2021/01/25 10:49:34 INFO: [Worker wazuhwork01.mydomain.org] [Main] Compressing files
2021/01/25 10:49:34 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sending compressed file to master
2021/01/25 10:49:34 INFO: [Worker wazuhwork01.mydomain.org] [Main] Worker files sent to master
2021/01/25 10:49:34 INFO: [Worker wazuhwork01.mydomain.org] [Main] The master has verified that the integrity is right.
2021/01/25 10:49:36 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting agent-info sync process.
2021/01/25 10:49:36 INFO: [Client wazuhwork01.mydomain.org] [Main] Obtaining data to be sent to master's wazuh-db.
2021/01/25 10:49:36 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting to send information to wazuh-db.
2021/01/25 10:49:36 INFO: [Client wazuhwork01.mydomain.org] [Main] Finished sending information to wazuh-db (0 chunks sent).
2021/01/25 10:49:43 INFO: [Worker wazuhwork01.mydomain.org] [Main] Permission to synchronize granted
2021/01/25 10:49:43 INFO: [Worker wazuhwork01.mydomain.org] [Main] Compressing files
2021/01/25 10:49:43 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sending compressed file to master
2021/01/25 10:49:43 INFO: [Worker wazuhwork01.mydomain.org] [Main] Worker files sent to master
2021/01/25 10:49:43 INFO: [Worker wazuhwork01.mydomain.org] [Main] The master has verified that the integrity is right.
2021/01/25 10:49:46 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting agent-info sync process.
2021/01/25 10:49:46 INFO: [Client wazuhwork01.mydomain.org] [Main] Obtaining data to be sent to master's wazuh-db.
2021/01/25 10:49:46 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting to send information to wazuh-db.
2021/01/25 10:49:46 INFO: [Client wazuhwork01.mydomain.org] [Main] Finished sending information to wazuh-db (1 chunks sent).
2021/01/25 10:49:50 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sucessful response from master: keepalive
2021/01/25 10:49:52 INFO: [Worker wazuhwork01.mydomain.org] [Main] Permission to synchronize granted
2021/01/25 10:49:52 INFO: [Worker wazuhwork01.mydomain.org] [Main] Compressing files
2021/01/25 10:49:52 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sending compressed file to master
2021/01/25 10:49:52 INFO: [Worker wazuhwork01.mydomain.org] [Main] Worker files sent to master
2021/01/25 10:49:52 INFO: [Worker wazuhwork01.mydomain.org] [Main] The master has verified that the integrity is right.
2021/01/25 10:49:56 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting agent-info sync process.
2021/01/25 10:49:56 INFO: [Client wazuhwork01.mydomain.org] [Main] Obtaining data to be sent to master's wazuh-db.
2021/01/25 10:49:56 INFO: [Client wazuhwork01.mydomain.org] [Main] Starting to send information to wazuh-db.
2021/01/25 10:49:56 INFO: [Client wazuhwork01.mydomain.org] [Main] Finished sending information to wazuh-db (0 chunks sent).
2021/01/25 10:50:01 INFO: [Worker wazuhwork01.mydomain.org] [Main] Permission to synchronize granted
2021/01/25 10:50:01 INFO: [Worker wazuhwork01.mydomain.org] [Main] Compressing files
2021/01/25 10:50:01 INFO: [Worker wazuhwork01.mydomain.org] [Main] Sending compressed file to master
2021/01/25 10:50:01 INFO: [Worker wazuhwork01.mydomain.org] [Main] Worker files sent to master
Agent.conf conf assigned to these Ubuntu agents:
<agent_config>
<client_buffer>
<disabled>no</disabled>
<queue_size>5000</queue_size>
<events_per_second>500</events_per_second>
</client_buffer>
<labels>
<label key="agent.group">rhel</label>
</labels>
<rootcheck>
<disabled>no</disabled>
<check_files>yes</check_files>
<check_trojans>yes</check_trojans>
<check_dev>yes</check_dev>
<check_sys>yes</check_sys>
<check_pids>yes</check_pids>
<check_ports>yes</check_ports>
<check_if>yes</check_if>
<frequency>43200</frequency>
<rootkit_files>/var/ossec/etc/shared/rootkit_files.txt</rootkit_files>
<rootkit_trojans>/var/ossec/etc/shared/rootkit_trojans.txt</rootkit_trojans>
<skip_nfs>yes</skip_nfs>
</rootcheck>
<syscheck>
<disabled>yes</disabled>
<frequency>43200</frequency>
<scan_on_start>yes</scan_on_start>
<alert_new_files>yes</alert_new_files>
<auto_ignore frequency="10" timeframe="3600">no</auto_ignore>
<directories report_changes="yes" realtime="yes" check_all="yes" whodata="yes">/etc,/boot</directories>
<directories report_changes="yes" realtime="yes" check_all="yes" whodata="no">/usr/bin,/usr/sbin,/bin,/sbin,/usr/local/bin,/usr/local/sbin</directories>
<ignore>/etc/mtab</ignore>
<ignore>/etc/hosts.deny</ignore>
<ignore>/etc/adjtime</ignore>
<ignore>/sys/kernel/security</ignore>
<ignore>/sys/kernel/debug</ignore>
<ignore>/dev/core</ignore>
<ignore>/etc/ld.so.cache</ignore>
<ignore type="sregex">.log$|.swp$</ignore>
<skip_nfs>yes</skip_nfs>
<skip_dev>yes</skip_dev>
<skip_proc>yes</skip_proc>
<skip_sys>yes</skip_sys>
<process_priority>10</process_priority>
<max_eps>100</max_eps>
<database>disk</database>
<synchronization>
<enabled>yes</enabled>
<interval>5m</interval>
<max_interval>1h</max_interval>
<max_eps>10</max_eps>
</synchronization>
</syscheck>
<wodle name="open-scap">
<disabled>yes</disabled>
<timeout>1800</timeout>
<interval>1d</interval>
<scan-on-start>yes</scan-on-start>
</wodle>
<wodle name="cis-cat">
<disabled>yes</disabled>
<timeout>1800</timeout>
<interval>1d</interval>
<scan-on-start>yes</scan-on-start>
<java_path>wodles/java</java_path>
<ciscat_path>wodles/ciscat</ciscat_path>
</wodle>
<wodle name="osquery">
<disabled>yes</disabled>
<run_daemon>yes</run_daemon>
<log_path>/var/log/osquery/osqueryd.results.log</log_path>
<config_path>/etc/osquery/osquery.conf</config_path>
<add_labels>yes</add_labels>
</wodle>
<wodle name="syscollector">
<disabled>no</disabled>
<interval>1h</interval>
<scan_on_start>yes</scan_on_start>
<hardware>yes</hardware>
<os>yes</os>
<network>yes</network>
<packages>yes</packages>
<ports all="no">yes</ports>
<processes>yes</processes>
</wodle>
<sca>
<enabled>yes</enabled>
<scan_on_start>yes</scan_on_start>
<interval>12h</interval>
<skip_nfs>yes</skip_nfs>
</sca>
<active-response>
<disabled>no</disabled>
<ca_store>/var/ossec/etc/wpk_root.pem</ca_store>
<ca_verification>yes</ca_verification>
</active-response>
<logging>
<log_format>plain</log_format>
</logging>
<localfile>
<log_format>command</log_format>
<command>df -P</command>
</localfile>
<localfile>
<log_format>full_command</log_format>
<command>netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \
4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d</command>
<alias>netstat listening ports</alias>
<frequency>360</frequency>
</localfile>
<localfile>
<log_format>full_command</log_format>
<command>last -n 20</command>
<frequency>360</frequency>
</localfile>
<!--<localfile>
<log_format>audit</log_format>
<location>/var/log/audit/audit.log</location>
</localfile>-->
<localfile>
<log_format>syslog</log_format>
<location>/var/ossec/logs/active-responses.log</location>
</localfile>
<localfile>
<log_format>syslog</log_format>
<location>/var/log/auth.log</location>
</localfile>
<localfile>
<log_format>syslog</log_format>
<location>/var/log/syslog</location>
</localfile>
<localfile>
<log_format>syslog</log_format>
<location>/var/log/kern.log</location>
</localfile>
<localfile>
<log_format>syslog</log_format>
<location>/var/log/mail.log</location>
</localfile>
</agent_config>
Regards.
From: Selu López <joselui...@wazuh.com>
Date: Monday, 25 January 2021 at 09:10
To: Carlos Lopez <clo...@outlook.com>
Cc: "wa...@googlegroups.com" <wa...@googlegroups.com>
Subject: Re: Ubuntu agents are not sync'ed when shared config us used
Hi Carlos,
I will try to reproduce your use case in my environment to see if I get any similar issues. Could you paste here the agent.conf that is not syncing (removing any sensitive information)? I think we will need to do a bit of debugging as well since the problem could be related to multiple things (some cluster nodes not syncing, incorrect conf, agent not belonging to the expected group, etc).
First of all, could you check if there are any related errors in the ossec.conf of your Wazuh master and your Wazuh agent?
grep -iE "err|warn" /var/ossec/logs/ossec.log
Also, if possible, check to which node that agent is reporting by using the following command in your master:
/var/ossec/bin/cluster_control -a
And then, get the logs of the cluster.log file in the manager node where the agent is reporting to, so we can check if there are any sync problem between cluster nodes:
tail -n100 /var/ossec/logs/cluster.log
By the way, is there any other difference between your RHEL-based agents and Ubuntu agents? Are they all on the same version of Wazuh? Do they belong to the same group of agents? If not, is there a difference in the settings you are trying to apply to each one?
Regards,
Selu.
On Sun, Jan 24, 2021 at 6:24 PM Carlos Lopez <clo...@outlook.com> wrote:
Hi all,
I have noticed that after updating my Ubuntu’s agents to release 4.0.4 config are not sync’ed in a cluster deployment. Example:
root@wazuhmaster:/var/ossec/etc/shared/rhel# agent_control -i 013
Wazuh agent_control. Agent information:
Agent ID: 013
Agent Name: ubu01.mydomain.org
IP address: 172.22.55.10
Status: Active
Operating system: Linux |ubu01 |5.8.0-40-generic |#45~20.04.1-Ubuntu SMP Fri Jan 15 11:35:04 UTC 2021 |x86_64
Client version: Wazuh v4.0.4
Configuration hash: 13eb20c2f8717608cb9e08ade77a2abf
Shared file hash: e65d07f9743cbf61fad1132498ccaead
Last keep alive: 1611508761
Syscheck last started at: Unknown
Syscheck last ended at: Unknown
Rootcheck last started at: Unknown
root@wazuhmaster:/var/ossec/etc/shared/ubuntu# md5sum merged.mg
28d9b410ff4d9f31c6d663841239283f merged.mg
But this is not occurs with RHEL based agents ….Any idea?
Regards,
C. L. Martinez
Hi Selu,
Yes it is correct …. But, maybe I have found a problem. As a test I have created a dir in /var/ossec/etc/shared/ubunut/ to replicate it across these ubuntu agents … and agent config is updated and dir appears empty in the agent side, as you can see here:
root@ubu01:~# ls -la /var/ossec/etc/shared/
total 72
drwxrwx--- 2 root ossec 166 Jan 24 17:42 .
drwxrwx--- 3 ossec ossec 181 Jan 24 16:43 ..
-rw-r--r-- 1 ossec ossec 5255 Jan 24 17:42 agent.conf
-rw-r--r-- 1 ossec ossec 217 Jan 24 17:42 ar.conf
-rw-r--r-- 1 ossec ossec 2436 Jan 24 17:42 client.config.yaml
-rw-r--r-- 1 ossec ossec 29780 Jan 24 17:42 merged.mg
-rw-r--r-- 1 ossec ossec 16179 Jan 24 17:42 rootkit_files.txt
-rw-r--r-- 1 ossec ossec 5553 Jan 24 17:42 rootkit_trojans.txt
-rw-r--r-- 1 ossec ossec 0 Jan 26 09:42 test
And if I remove this empty file (test), agent stops updating config ……
Hi Selu,
Another time, but in this case with agent release 4.1.0:
root@ubu01:/var/ossec/etc/shared# ls -al
total 92
drwxrwx--- 2 root ossec 147 Feb 20 16:40 .
drwxrwx--- 3 ossec ossec 181 Feb 20 16:25 ..
-rw-r--r-- 1 ossec ossec 4794 Feb 20 16:40 agent.conf
-rw-r--r-- 1 ossec ossec 254 Feb 20 16:40 agent.conf~
-rw-r--r-- 1 ossec ossec 217 Feb 20 16:40 ar.conf
-rw-r--r-- 1 ossec ossec 37324 Feb 20 16:40 merged.mg
-rw-r--r-- 1 ossec ossec 10199 Feb 20 16:40 osquery.conf
-rw-r--r-- 1 ossec ossec 16179 Feb 20 16:40 rootkit_files.txt
-rw-r--r-- 1 ossec ossec 5553 Feb 20 16:40 rootkit_trojans.txt
Contents of the agent.conf~ file is:
<agent_config>
<localfile>
<log_format>apache</log_format>
<location>/var/log/nginx/access.log</location>
</localfile>
<localfile>
<log_format>apache</log_format>
<location>/var/log/nginx/access.log</location>
</localfile>
</agent_config>
But this content is included in the agent.conf … What is happening?
To view this discussion on the web visit
https://groups.google.com/d/msgid/wazuh/92fd4509-c112-45dc-93f4-44657b1f6da9n%40googlegroups.com.