Wazuh-agent stops sending journald logs

[Francesc G]

Hi,

I have 21 wazuh-agents running on Linux (Debian 11). They are all configured as follow to send journald logs:

  <localfile>
    <log_format>journald</log_format>
    <location>journald</location>
    <filter field="_SYSTEMD_UNIT">^(?!auditd.service).*</filter>
  </localfile>

After several days I noticed that some of them stopped sending journald logs. Yesterday I restarted wazuh-agent on them and they started sending journald logs again. Now I checked the status and again there are 6 agents that have stopped sending journald logs.

This only happens with journald logs. Other logs are being set correctly. I have checked if journald generates logs and yes.

I use Wazuh 4.9.0 both in server and agents.

Best regards.

[Obinna Uchubilo]

Hello Francesc

You need to review the Wazuh agent logs for more information on the issue. Please share the ossec.log file (/var/ossec/logs/ossec.log), and feel free to redact any information you consider confidential.

Regards

[Francesc G]

Hi Obinna,

Nothing relevant is shown in ossec.log.

For example, in one specific agent the last journald log was received today at 10:17:14. After restarting wazuh-agent (11:49:14) the first journald log arrived at 11:49:26.  Below the ossec.log log:

2024/11/06 00:00:10 wazuh-agentd: INFO: Starting new log after rotation.
2024/11/06 00:27:38 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 00:27:45 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 01:27:46 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 01:27:53 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 02:27:54 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 02:28:01 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 03:26:31 sca: INFO: Starting Security Configuration Assessment scan.
2024/11/06 03:26:31 sca: INFO: Starting evaluation of policy: '/var/ossec/etc/custom/sca/cis_debian11_custom.yml'
2024/11/06 03:26:46 sca: INFO: Evaluation finished for policy '/var/ossec/etc/custom/sca/cis_debian11_custom.yml'
2024/11/06 03:26:46 sca: INFO: Security Configuration Assessment scan finished. Duration: 15 seconds.
2024/11/06 03:27:08 wazuh-syscheckd: INFO: (6008): File integrity monitoring scan started.
2024/11/06 03:27:37 rootcheck: INFO: Starting rootcheck scan.
2024/11/06 03:27:45 wazuh-syscheckd: INFO: (6009): File integrity monitoring scan ended.
2024/11/06 03:28:02 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 03:28:11 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 03:28:15 rootcheck: INFO: Ending rootcheck scan.
2024/11/06 04:28:12 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 04:28:20 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 05:28:20 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 05:28:28 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 06:28:28 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 06:28:36 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 07:28:37 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 07:28:44 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 08:28:45 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 08:28:52 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 09:28:53 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 09:29:00 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 10:29:01 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 10:29:07 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 11:29:08 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 11:29:16 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 11:49:14 wazuh-modulesd:syscollector: INFO: Stop received for Syscollector.
2024/11/06 11:49:14 wazuh-modulesd:syscollector: INFO: Module finished.
2024/11/06 11:49:14 wazuh-logcollector: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/06 11:49:15 wazuh-syscheckd: INFO: (1756): Shutdown received. Releasing resources.
2024/11/06 11:49:15 wazuh-syscheckd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/06 11:49:15 wazuh-agentd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/06 11:49:15 wazuh-execd: INFO: (1314): Shutdown received. Deleting responses.
2024/11/06 11:49:15 wazuh-execd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/06 11:49:15 sca: WARNING: File 'ruleset/sca/ruleset/sca/cis_debian11.yml' not found.
2024/11/06 11:49:15 wazuh-execd: INFO: Started (pid: 981059).
2024/11/06 11:49:15 wazuh-agentd: INFO: (1410): Reading authentication keys file.
2024/11/06 11:49:16 wazuh-agentd: INFO: Using notify time: 10 and max time to reconnect: 60
2024/11/06 11:49:16 wazuh-agentd: INFO: Version detected -> Linux |hostname |5.10.0-28-amd64 |#1 SMP Debian 5.10.209-2 (2024-01-31) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0
2024/11/06 11:49:16 wazuh-agentd: INFO: Started (pid: 981067).
2024/11/06 11:49:16 wazuh-agentd: INFO: Using AES as encryption method.
2024/11/06 11:49:16 wazuh-agentd: INFO: Trying to connect to server ([wazuh.domain.local]:1514/tcp).
2024/11/06 11:49:16 wazuh-agentd: INFO: (4102): Connected to the server ([wazuh.domain.local]:1514/tcp).
2024/11/06 11:49:17 wazuh-syscheckd: INFO: Started (pid: 981081).
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6003): Monitoring path: '/bin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6003): Monitoring path: '/boot', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6003): Monitoring path: '/etc', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6003): Monitoring path: '/sbin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6003): Monitoring path: '/usr/bin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6003): Monitoring path: '/usr/sbin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mtab'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/hosts.deny'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mail/statistics'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random-seed'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random.seed'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/adjtime'

2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/adjtime'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/httpd/logs'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/utmpx'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/wtmpx'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/cups/certs'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/dumpdates'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/svc/volatile'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6207): Ignore 'file' sregex '.log$|.swp$'
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6004): No diff for file: '/etc/ssl/private.key'
2024/11/06 11:49:17 rootcheck: INFO: Starting rootcheck scan.
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6000): Starting daemon...
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6010): File integrity monitoring scan frequency: 43200 seconds
2024/11/06 11:49:17 wazuh-syscheckd: INFO: (6008): File integrity monitoring scan started.
2024/11/06 11:49:17 wazuh-logcollector: INFO: Monitoring output of command(360): df -P
2024/11/06 11:49:17 wazuh-logcollector: INFO: Monitoring full output of command(360): netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d
2024/11/06 11:49:17 wazuh-logcollector: INFO: Monitoring full output of command(360): last -n 20
2024/11/06 11:49:17 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/ossec/logs/active-responses.log'.
2024/11/06 11:49:17 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/dpkg.log'.
2024/11/06 11:49:17 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/audit/audit.log'.
2024/11/06 11:49:17 sca: WARNING: File 'ruleset/sca/ruleset/sca/cis_debian11.yml' not found.
2024/11/06 11:49:17 wazuh-modulesd: INFO: Started (pid: 981099).
2024/11/06 11:49:17 wazuh-modulesd:agent-upgrade: INFO: (8153): Module Agent Upgrade started.
2024/11/06 11:49:17 wazuh-modulesd:ciscat: INFO: Module disabled. Exiting...
2024/11/06 11:49:17 sca: INFO: Module started.
2024/11/06 11:49:17 sca: INFO: Loaded policy '/var/ossec/etc/custom/sca/cis_debian11_custom.yml'
2024/11/06 11:49:17 wazuh-modulesd:osquery: INFO: Module disabled. Exiting...
2024/11/06 11:49:17 wazuh-modulesd:control: INFO: Starting control thread.
2024/11/06 11:49:17 sca: INFO: Starting Security Configuration Assessment scan.
2024/11/06 11:49:17 wazuh-logcollector: INFO: Started (pid: 981091).
2024/11/06 11:49:17 wazuh-modulesd:syscollector: INFO: Module started.
2024/11/06 11:49:17 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 11:49:17 sca: INFO: Starting evaluation of policy: '/var/ossec/etc/custom/sca/cis_debian11_custom.yml'
2024/11/06 11:49:17 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 11:49:23 wazuh-logcollector: INFO: (9203): Monitoring journal entries.
2024/11/06 11:49:43 sca: INFO: Evaluation finished for policy '/var/ossec/etc/custom/sca/cis_debian11_custom.yml'
2024/11/06 11:49:43 sca: INFO: Security Configuration Assessment scan finished. Duration: 26 seconds.
2024/11/06 11:49:57 wazuh-syscheckd: INFO: (6009): File integrity monitoring scan ended.
2024/11/06 11:49:57 wazuh-syscheckd: INFO: FIM sync module started.
2024/11/06 11:50:21 rootcheck: INFO: Ending rootcheck scan.
2024/11/06 12:49:18 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 12:49:28 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/06 13:49:29 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/06 13:49:36 wazuh-modulesd:syscollector: INFO: Evaluation finished.

[Francesc G]

Hi,

Still having this problem.

Every day I have to restart wazuh-agent on several agents so that the journald information is sent to the Wazuh server.

To try to find out what the problem is, can I run the agent with some parameter to get a higher log level?

Best regards.

[Obinna Uchubilo]

Hello,

To get a better debugging log, you can set the agent.debug value to 2 in the agent internal_options.conf. Hopefully, we can get more information on the issue. The internal configuration features are located in the /var/ossec/etc/internal_options.conf.


Regards

[Francesc G]

Hi,

I have an agent that disconnects quite often so I have set the parametersI agent.debug=2 and logcollector.debug=2 in internal_options.conf.

These logs are an excerpt from when Wazuh stopped receiving logs from journald on November 12th. The last log was from 13:55:37.678.

Regarding agent.debug nothing relevant is shown in ossec.log. You can see these 6 lines repeated constantly:

2024/11/12 13:55:01 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 13:55:01 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 13:55:01 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 13:55:02 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:07 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:11 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 13:55:11 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 13:55:11 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '

2024/11/12 13:55:12 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.

2024/11/12 13:55:17 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.

2024/11/12 13:55:21 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.

2024/11/12 13:55:21 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf

2024/11/12 13:55:21 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 13:55:22 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:27 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:31 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 13:55:31 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 13:55:31 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 13:55:32 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:37 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:41 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 13:55:41 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 13:55:41 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 13:55:42 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:47 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:51 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 13:55:51 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 13:55:51 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 13:55:52 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:55:57 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:56:01 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 13:56:01 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 13:56:01 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 13:56:02 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:56:07 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:56:11 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 13:56:11 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 13:56:11 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 13:56:12 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 13:56:17 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.


A few minutes later there is an agent disconnection due to a restart of the wazuh-manager on the server due to a rule reconfiguration. Everything seems to be OK:

2024/11/12 14:19:11 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 14:19:11 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 14:19:11 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agenqt ack '
2024/11/12 14:19:12 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:17 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:21 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 14:19:21 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 14:19:21 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 14:19:22 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:26 wazuh-agentd[402711] receiver.c:75 at receive_msg(): DEBUG: Manager disconnected.
2024/11/12 14:19:26 wazuh-agentd[402711] agentd.c:192 at AgentdStart(): ERROR: (1137): Lost connection with manager. Setting lock.
2024/11/12 14:19:26 wazuh-agentd[402711] start_agent.c:61 at connect_server(): INFO: Closing connection to server ([wazuh.domain.local]:1514/tcp).
2024/11/12 14:19:26 wazuh-agentd[402711] start_agent.c:93 at connect_server(): INFO: Trying to connect to server ([wazuh.domain.local]:1514/tcp).
2024/11/12 14:19:26 wazuh-agentd[402711] start_agent.c:111 at connect_server(): ERROR: (1216): Unable to connect to '[192.168.1.21]:1514/tcp': 'Connection refused'.
2024/11/12 14:19:27 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:32 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:36 wazuh-agentd[402711] start_agent.c:93 at connect_server(): INFO: Trying to connect to server ([wazuh.domain.local]:1514/tcp).
2024/11/12 14:19:36 wazuh-agentd[402711] start_agent.c:111 at connect_server(): ERROR: (1216): Unable to connect to '[192.168.1.21]:1514/tcp': 'Connection refused'.
2024/11/12 14:19:37 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:42 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:46 wazuh-agentd[402711] start_agent.c:93 at connect_server(): INFO: Trying to connect to server ([wazuh.domain.local]:1514/tcp).
2024/11/12 14:19:46 wazuh-agentd[402711] start_agent.c:365 at agent_handshake_to_server(): INFO: (4102): Connected to the server ([wazuh.domain.local]:1514/tcp).
2024/11/12 14:19:46 wazuh-agentd[402711] agentd.c:195 at AgentdStart(): INFO: Server responded. Releasing lock.
2024/11/12 14:19:46 wazuh-agentd[402711] notify.c:129 at run_notify(): DEBUG: Sending agent notification.
2024/11/12 14:19:46 wazuh-agentd[402711] notify.c:198 at run_notify(): DEBUG: Sending keep alive: #!-Linux |hostname |4.14.0-0.bpo.3-amd64 |#1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0 / 35d639ad3f6039d93b2d3a6b71311faf
2024/11/12 14:19:46 wazuh-agentd[402711] receiver.c:96 at receive_msg(): DEBUG: Received message: '#!-agent ack '
2024/11/12 14:19:47 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.
2024/11/12 14:19:52 wazuh-agentd[402711] state.c:78 at write_state(): DEBUG: Updating state file.

Regarding logcollector.debug, the following lines can be found repeatedly in the logs. No errors are shown after 13:55:37 when the last line arrived from journald. From that time onwards all that is shown concerning journald is "read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal."

As you can see in the logs logcollector is still working as far as rsyslog is concerned without any problem, so the problem is restricted to journald.

2024/11/12 13:55:35 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 2 lines from /var/log/vmware.log
2024/11/12 13:55:35 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:35 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:35 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:35 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:35 vmware.DOMAIN.local Rhttpproxy: verbose rh'...
2024/11/12 13:55:35 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:37 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:37 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:37 vmware.DOMAIN.local Rhttpproxy: verbose rh'...
2024/11/12 13:55:37 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:37 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:37 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:37 wazuh-logcollector[402738] journal_log.c:394 at w_journal_context_next_newest_filtered(): DEBUG: (9004): Checking filters for timestamp '2024-11-12 12:55:37'
2024/11/12 13:55:37 wazuh-logcollector[402738] read_journald.c:168 at read_journald(): DEBUG: (9008): Reading from journal: 'Nov 12 12:55:37 hostname named[611]: address not available resolving 'www.google.com/A/IN': 2a01:111:4000:700::5#53'.
2024/11/12 13:55:37 wazuh-logcollector[402738] journal_log.c:394 at w_journal_context_next_newest_filtered(): DEBUG: (9004): Checking filters for timestamp '2024-11-12 12:55:37'
2024/11/12 13:55:37 wazuh-logcollector[402738] read_journald.c:168 at read_journald(): DEBUG: (9008): Reading from journal: 'Nov 12 12:55:37 hostname named[611]: address not available resolving 'www.google.com/AAAA/IN': 2a01:111:4000:700::5#53'.
2024/11/12 13:55:37 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:37 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:37 vmware.DOMAIN.local snmpd: load_ipmi_sel: l'...
2024/11/12 13:55:37 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:37 vmware.DOMAIN.local snmpd: send_env_notific'...
2024/11/12 13:55:37 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 2 lines from /var/log/vmware.log
2024/11/12 13:55:39 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:39 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:39 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:39 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:41 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:41 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:40 vmware.DOMAIN.local Rhttpproxy: verbose rh'...
2024/11/12 13:55:41 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:41 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:41 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:41 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:43 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:43 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:43 vmware.DOMAIN.local Rhttpproxy: verbose rh'...
2024/11/12 13:55:43 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:43 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:43 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:43 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:45 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:45 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:44 vmware.DOMAIN.local Rhttpproxy: verbose rh'...
2024/11/12 13:55:45 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:45 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:44 vmware.DOMAIN.local Rhttpproxy: verbose rht'...
2024/11/12 13:55:45 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:45 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:45 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:45 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:45 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:45 vmware.DOMAIN.local Hostd: info hostd[2099'...
2024/11/12 13:55:45 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:47 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:47 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:45 vmware.DOMAIN.local Hostd: info hostd[2099'...
2024/11/12 13:55:47 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:47 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:47 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:47 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:49 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:49 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:49 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:49 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.
2024/11/12 13:55:49 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:49 vmware.DOMAIN.local Rhttpproxy: verbose rh'...
2024/11/12 13:55:49 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:51 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:51 wazuh-logcollector[402738] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Nov 12 12:55:50 vmware.DOMAIN.local Rhttpproxy: verbose rh'...
2024/11/12 13:55:51 wazuh-logcollector[402738] read_syslog.c:152 at read_syslog(): DEBUG: Read 1 lines from /var/log/vmware.log
2024/11/12 13:55:51 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:51 wazuh-logcollector[402738] logcollector.c:2123 at w_input_thread(): DEBUG: (9005): Skipping is not the owner of the journal log.
2024/11/12 13:55:51 wazuh-logcollector[402738] read_journald.c:142 at read_journald(): DEBUG: (9006): No new entries in the journal.



I have no reason to do this but... I have also disabled sending auditd logs through journald to remove the filter from the <localfile> configuration.

<localfile>
<log_format>journald</log_format>
<location>journald</location>

<!-- <filter field="_SYSTEMD_UNIT">^(?!auditd.service).*</filter> -->
</localfile>

Is there another *.debug setting I can enable that would help?

Best regards.

[Obinna Uchubilo]

Hello,

From the Agent's logs shared, I can't see any error as to why the log collection stops.

Can share you the logs from the Wazuh server, so we can confirm everything is fine on that end before we troubleshoot further?

Regards

[Francesc G]

Hi,

Thanks for your reply.

The logs from the Wazuh Server for the related date and time are the following.

There are some invalid JSON alert read from 'logs/alerts/alerts.json' but it doesn't seem to be related.

2024/11/12 11:40:55 wazuh-maild: WARNING: Invalid JSON alert read from 'logs/alerts/alerts.json': '"172.20.1.25"},"manager":{"name":"wazuh"},"id":"1731408053.125458018","decoder":{"name":"windows_eventchannel"},"data":{"win":{"system":{"providerName":"Microsoft-Windows-Security-Auditing","providerGuid":"{54849625-5478-4994-A5BA-3E3B0328C30D}","eventID":"5140","version":"1","level":"0","task":"12808","opcode":"0","keywords":"0x8020000000000000","systemTime":"2024-11-12T10:40:32.473681100Z","eventRecordID":"658034673","processID":"4","threadID":"7764","channel":"Security","computer":"fileserver.DOMAIN.local","severityValue":"AUDIT_SUCCESS","message":"\"Se tuvo acceso a un objeto de recurso compartido de red.\r\n\t\r\nSujeto:\r\n\tId. de seguridad:\t\tS-1-5-21-1004336348-1177238915-682003330-6634\r\n\tNombre de cuenta:\t\tdummy_username\r\n\tDominio de cuenta:\t\tDOMAIN\r\n\tId. de inicio de sesión:\t\t0x4AE2B303\r\n\r\nInformación de red:\t\r\n\tTipo de objeto:\t\tFile\r\n\tDirección de origen:\t\t192.168.1.127\r\n\tPuerto de origen:\t\t55325\r\n\t\r\nInformación de recurso compartido:\r\n\tNombre de recurso compartido:\t\t\\\\*\\Shared-folder\r\n\tRuta de acceso de recurso compartido:\t\t\\??\\E:\\Shared-folder\r\n\r\nInformación de solicitud de acceso:\r\n\tMáscara de acceso:\t\t0x1\r\n\tAccesos:\t\tReadData (o ListDirectory)\r\n\t\t\t\t\r\n\""},"eventdata":{"subjectUserSid":"S-1-5-21-1004336348-1177238915-682003330-6634","subjectUserName":"dummy_username","subjectDomainName":"DOMAIN","subjectLogonId":"0x4ae2b303","objectType":"File","ipAddress":"192.168.1.127","ipPort":"55325","shareName":"\\\\\\\\*\\\\Shared-folder","shareLocalPath":"\\\\??\\\\E:\\\\Shared-folder","accessMask":"0x1","accessList":"%%4416"}}},"location":"EventChannel"}'
2024/11/12 12:05:26 wazuh-maild: WARNING: Invalid JSON alert read from 'logs/alerts/alerts.json': 'lete\"\u001dARCH=x86_64 SYSCALL=rename AUID=\"admin\" UID=\"root\" GID=\"root\" EUID=\"root\" SUID=\"root\" FSUID=\"root\" EGID=\"root\" SGID=\"root\" FSGID=\"root\" type=PATH msg=audit(1731409520.078:23393): item=0 name=\"/root/\" inode=262145 dev=fe:00 mode=040700 ouid=0 ogid=0 rdev=00:00 nametype=PARENT cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0\u001dOUID=\"root\" OGID=\"root\" type=PATH msg=audit(1731409520.078:23393): item=1 name=\"/root/\" inode=262145 dev=fe:00 mode=040700 ouid=0 ogid=0 rdev=00:00 nametype=PARENT cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0\u001dOUID=\"root\" OGID=\"root\" type=PATH msg=audit(1731409520.078:23393): item=2 name=\"/root/.bash_history-72292.tmp\" inode=262459 dev=fe:00 mode=0100600 ouid=0 ogid=0 rdev=00:00 nametype=DELETE cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0\u001dOUID=\"root\" OGID=\"root\" type=PATH msg=audit(1731409520.078:23393): item=3 name=\"/root/.bash_history\" inode=262183 dev=fe:00 mode=0100600 ouid=0 ogid=0 rdev=00:00 nametype=DELETE cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0\u001dOUID=\"root\" OGID=\"root\" type=PATH msg=audit(1731409520.078:23393): item=4 name=\"/root/.bash_history\" inode=262459 dev=fe:00 mode=0100600 ouid=0 ogid=0 rdev=00:00 nametype=CREATE cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0\u001dOUID=\"root\" OGID=\"root\" type=PROCTITLE msg=audit(1731409520.078:23393): proctitle=\"-bash\"","decoder":{"parent":"auditd","name":"auditd"},"data":{"audit":{"type":"SYSCALL","id":"23393","arch":"c000003e","syscall":"82","success":"yes","exit":"0","ppid":"1572291","pid":"1572292","auid":"471002242","uid":"0","gid":"0","euid":"0","suid":"0","fsuid":"0","egid":"0","sgid":"0","fsgid":"0","tty":"pts2","session":"995","command":"bash","exe":"/usr/bin/bash","key":"delete","directory":{"name":"/root/","inode":"262145","mode":"040700"},"file":{"name":"/root/","inode":"262145","mode":"040700"}}},"location":"/var/log/audit/audit.log"}'  
2024/11/12 12:20:39 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/12 12:20:50 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/12 13:20:51 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/12 13:21:15 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/11/12 14:19:26 wazuh-modulesd:syscollector: INFO: Stop received for Syscollector.
2024/11/12 14:19:26 wazuh-modulesd:syscollector: INFO: Module finished.
2024/11/12 14:19:26 wazuh-modulesd:vulnerability-scanner: INFO: Stopping vulnerability_scanner module.
2024/11/12 14:19:26 wazuh-modulesd:router: INFO: Stopping router module.
2024/11/12 14:19:26 wazuh-modulesd:content_manager: INFO: Stopping content_manager module.
2024/11/12 14:19:26 wazuh-monitord: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:26 wazuh-logcollector: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:26 wazuh-remoted: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:26 wazuh-remoted: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:26 wazuh-remoted: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:26 wazuh-syscheckd: INFO: (1756): Shutdown received. Releasing resources.
2024/11/12 14:19:26 wazuh-syscheckd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:27 wazuh-analysisd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:27 wazuh-maild: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:27 wazuh-execd: INFO: (1314): Shutdown received. Deleting responses.
2024/11/12 14:19:27 wazuh-execd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:27 wazuh-db: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:28 wazuh-db: INFO: Graceful process shutdown.
2024/11/12 14:19:28 wazuh-authd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:29 wazuh-authd: INFO: Exiting...
2024/11/12 14:19:29 wazuh-integratord: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/11/12 14:19:32 wazuh-modulesd:router: INFO: Loaded router module.
2024/11/12 14:19:32 wazuh-modulesd:content_manager: INFO: Loaded content_manager module.
2024/11/12 14:19:36 wazuh-csyslogd: INFO: Remote syslog server not configured. Clean exit.
2024/11/12 14:19:36 wazuh-dbd: INFO: Database not configured. Clean exit.
2024/11/12 14:19:36 wazuh-integratord: INFO: Started (pid: 280035).
2024/11/12 14:19:36 wazuh-integratord: INFO: Enabling integration for: 'virustotal'.
2024/11/12 14:19:36 wazuh-agentlessd: INFO: Not configured. Exiting.
2024/11/12 14:19:36 wazuh-authd: INFO: Started (pid: 280056).
2024/11/12 14:19:36 wazuh-authd: INFO: Accepting connections on port 1515. No password required.
2024/11/12 14:19:36 wazuh-authd: INFO: Setting network timeout to 1.000000 sec.
2024/11/12 14:19:36 wazuh-db: INFO: Started (pid: 280069).
2024/11/12 14:19:37 wazuh-execd: INFO: Started (pid: 280095).
2024/11/12 14:19:37 wazuh-maild: INFO: Started (pid: 280106).
2024/11/12 14:19:37 wazuh-maild: INFO: Getting alerts in JSON format.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: Started (pid: 280125).
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6003): Monitoring path: '/bin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6003): Monitoring path: '/boot', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6003): Monitoring path: '/etc', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6003): Monitoring path: '/sbin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6003): Monitoring path: '/usr/bin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6003): Monitoring path: '/usr/sbin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mtab'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/hosts.deny'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mail/statistics'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random-seed'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random.seed'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/adjtime'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/httpd/logs'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/utmpx'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/wtmpx'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/cups/certs'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/dumpdates'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/svc/volatile'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6207): Ignore 'file' sregex '.log$|.swp$'
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6004): No diff for file: '/etc/ssl/private.key'
2024/11/12 14:19:38 wazuh-analysisd: INFO: Total rules enabled: '10589'
2024/11/12 14:19:38 rootcheck: INFO: Starting rootcheck scan.
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6000): Starting daemon...
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6010): File integrity monitoring scan frequency: 43200 seconds
2024/11/12 14:19:38 wazuh-syscheckd: INFO: (6008): File integrity monitoring scan started.
2024/11/12 14:19:38 wazuh-analysisd: INFO: Started (pid: 280113).
2024/11/12 14:19:38 wazuh-analysisd: INFO: (7200): Logtest started
2024/11/12 14:19:38 wazuh-analysisd: INFO: EPS limit disabled
2024/11/12 14:19:39 wazuh-remoted: INFO: Remote syslog allowed from: '0.0.0.0/0'
2024/11/12 14:19:39 wazuh-remoted: INFO: Remote syslog allowed from: '0.0.0.0/0'
2024/11/12 14:19:39 wazuh-remoted: INFO: Remote syslog allowed from: '0.0.0.0/0'
2024/11/12 14:19:39 wazuh-remoted: INFO: Remote syslog allowed from: '0.0.0.0/0'
2024/11/12 14:19:39 wazuh-remoted: INFO: Started (pid: 280194). Listening on port 514/UDP (syslog).
2024/11/12 14:19:39 wazuh-remoted: INFO: Started (pid: 280193). Listening on port 10514/TCP (syslog).
2024/11/12 14:19:39 wazuh-remoted: INFO: Started (pid: 280192). Listening on port 1514/TCP (secure).
2024/11/12 14:19:39 wazuh-remoted: INFO: (1410): Reading authentication keys file.
2024/11/12 14:19:40 wazuh-logcollector: INFO: Monitoring output of command(360): df -P
2024/11/12 14:19:40 wazuh-logcollector: INFO: Monitoring full output of command(360): netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d
2024/11/12 14:19:40 wazuh-logcollector: INFO: Monitoring full output of command(360): last -n 20
2024/11/12 14:19:40 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/ossec/logs/active-responses.log'.
2024/11/12 14:19:40 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/auth.log'.
2024/11/12 14:19:40 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/syslog'.
2024/11/12 14:19:40 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/dpkg.log'.
2024/11/12 14:19:40 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/kern.log'.
2024/11/12 14:19:40 wazuh-logcollector: INFO: Started (pid: 280227).
2024/11/12 14:19:41 wazuh-monitord: INFO: Started (pid: 280250).
2024/11/12 14:19:42 wazuh-modulesd:router: INFO: Loaded router module.
2024/11/12 14:19:42 wazuh-modulesd:content_manager: INFO: Loaded content_manager module.
2024/11/12 14:19:42 wazuh-modulesd: INFO: Started (pid: 280272).
2024/11/12 14:19:42 wazuh-modulesd:agent-upgrade: INFO: (8153): Module Agent Upgrade started.
2024/11/12 14:19:42 wazuh-modulesd:task-manager: INFO: (8200): Module Task Manager started.
2024/11/12 14:19:42 wazuh-modulesd:content_manager: INFO: Starting content_manager module.
2024/11/12 14:19:42 wazuh-modulesd:osquery: INFO: Module disabled. Exiting...
2024/11/12 14:19:42 wazuh-modulesd:database: INFO: Module started.
2024/11/12 14:19:42 wazuh-modulesd:download: INFO: Module started.
2024/11/12 14:19:42 wazuh-modulesd:router: INFO: Starting router module.
2024/11/12 14:19:42 wazuh-modulesd:control: INFO: Starting control thread.
2024/11/12 14:19:42 wazuh-modulesd:office365: INFO: Module Office365 started.
2024/11/12 14:19:42 wazuh-modulesd:vulnerability-scanner: INFO: Starting vulnerability_scanner module.
2024/11/12 14:19:42 sca: INFO: Module started.
2024/11/12 14:19:42 sca: INFO: Loaded policy '/var/ossec/ruleset/sca/cis_ubuntu22-04.yml'
2024/11/12 14:19:42 wazuh-modulesd:ciscat: INFO: Module disabled. Exiting...
2024/11/12 14:19:42 sca: INFO: Starting Security Configuration Assessment scan.
2024/11/12 14:19:42 sca: INFO: Starting evaluation of policy: '/var/ossec/ruleset/sca/cis_ubuntu22-04.yml'
2024/11/12 14:19:42 wazuh-modulesd:syscollector: INFO: Module started.
2024/11/12 14:19:42 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/11/12 14:19:42 wazuh-modulesd:vulnerability-scanner: INFO: Vulnerability scanner module is disabled.

2024/11/12 14:19:43 wazuh-modulesd:syscollector: INFO: Evaluation finished.

2024/11/12 14:19:44 wazuh-syscheckd: INFO: (6009): File integrity monitoring scan ended.
2024/11/12 14:19:44 wazuh-syscheckd: INFO: FIM sync module started.
2024/11/12 14:19:49 sca: INFO: Evaluation finished for policy '/var/ossec/ruleset/sca/cis_ubuntu22-04.yml'
2024/11/12 14:19:49 sca: INFO: Security Configuration Assessment scan finished. Duration: 7 seconds.
2024/11/12 14:20:37 rootcheck: INFO: Ending rootcheck scan.
2024/11/12 14:23:59 wazuh-maild: WARNING: Invalid JSON alert read from 'logs/alerts/alerts.json': 'ctDomainName":"DOMAIN","subjectLogonId":"0x50ff7827"}}},"location":"EventChannel"}'
2024/11/12 14:24:01 wazuh-maild: WARNING: Invalid JSON alert read from 'logs/alerts/alerts.json': 'n Success","id":"60106","mitre":{"id":["T1078"],"tactic":["Defense Evasion","Persistence","Privilege Escalation","Initial Access"],"technique":["Valid Accounts"]},"firedtimes":31,"mail":false,"groups":["windows","windows_security","authentication_success"],"pci_dss":["10.2.5"],"gpg13":["7.1","7.2"],"gdpr":["IV_32.2"],"hipaa":["164.312.b"],"nist_800_53":["AU.14","AC.7"],"tsc":["CC6.8","CC7.2","CC7.3"]},"agent":{"id":"016","name":"dc","ip":"192.168.1.250"},"manager":{"name":"wazuh"},"id":"1731417840.256476295","decoder":{"name":"windows_eventchannel"},"data":{"win":{"system":{"providerName":"Microsoft-Windows-Security-Auditing","providerGuid":"{54849625-5478-4994-a5ba-3e3b0328c30d}","eventID":"4624","version":"2","level":"0","task":"12544","opcode":"0","keywords":"0x8020000000000000","systemTime":"2024-11-12T13:23:59.237572600Z","eventRecordID":"10288069182","processID":"688","threadID":"6676","channel":"Security","computer":"dc.DOMAIN.local","severityValue":"AUDIT_SUCCESS","message":"\"Se inició sesión correctamente en una cuenta.\r\n\r\nFirmante:\r\n\tId. de seguridad:\t\tS-1-0-0\r\n\tNombre de cuenta:\t\t-\r\n\tDominio de cuenta:\t\t-\r\n\tId. de inicio de sesión:\t\t0x0\r\n\r\nInformación de inicio de sesión:\r\n\tTipo de inicio de sesión:\t\t3\r\n\tModo de administrador restringido:\t-\r\n\tCuenta virtual:\t\tNo\r\n\tToken elevado:\t\tSí\r\n\r\nNivel de suplantación:\t\tSuplantación\r\n\r\nNuevo inicio de sesión:\r\n\tId. de seguridad:\t\tS-1-5-18\r\n\tNombre de cuenta:\t\tdc$\r\n\tDominio de cuenta:\t\tDOMAIN.LOCAL\r\n\tId. de inicio de sesión:\t\t0x65D2E058\r\n\tInicio de sesión vinculado:\t\t0x0\r\n\tNombre de cuenta de red:\t-\r\n\tDominio de cuenta de red:\t-\r\n\tGUID de inicio de sesión:\t\t{6545b3b8-bcab-ebaf-92c1-233d494d0c35}\r\n\r\nInformación de proceso:\r\n\tId. de proceso:\t\t0x0\r\n\tNombre de proceso:\t\t-\r\n\r\nInformación de red:\r\n\tNombre de estación de trabajo:\t-\r\n\tDirección de red de origen:\t127.0.0.1\r\n\tPuerto de origen:\t\t60336\r\n\r\nInformación de autenticación detallada:\r\n\tProceso de inicio de sesión:\t\tKerberos\r\n\tPaquete de autenticación:\tKerberos\r\n\tServicios transitados:\t-\r\n\tNombre de paquete (solo NTLM):\t-\r\n\tLongitud de clave:\t\t0\r\n\r\nEste evento se genera cuando se crea una sesión de inicio. Lo genera el equipo al que se tuvo acceso.\r\n\r\nLos campos de firmante indican la cuenta del sistema local que solicitó el inicio de sesión. Suele ser un servicio como el servicio de servidor o un proceso local como Winlogon.exe o Services.exe.\r\n\r\nEl campo Tipo de inicio de sesión indica la clase de inicio de sesión que se realizó. Los tipos más comunes son 2 (interactivo) y 3 (red).\r\n\r\nLos campos Nuevo inicio de sesión indican la cuenta para la que se creó el nuevo inicio de sesión, es decir, aquella en la que se inició la sesión.\r\n\r\nLos campos de red indican dónde se originó una solicitud de inicio de sesión remota. Nombre de estación de trabajo no está siempre disponible y se puede dejar en blanco en algunos casos.\r\n\r\nEl campo de nivel de suplantación indica en qué medida un proceso en la sesión de inicio de sesión puede suplantar.\r\n\r\nLos campos de información de autenticación proporcionan información detallada sobre esta solicitud de inicio de sesión específica.\r\n\t- GUID de inicio de sesión es un identificador único que se puede usar para correlacionar este evento con un evento KDC.\r\n\t- Servicios transitados indica los servicios intermedios que participaron en esta solicitud de inicio de sesión.\r\n\t- Nombre de paquete indica el subprotocolo que se usó entre los protocolos NTLM.\r\n\t- Longitud de clave indica la longitud de la clave de sesión generada. Será 0 si no se solicitó una clave de sesión.\""},"eventdata":{"subjectUserSid":"S-1-0-0","subjectLogonId":"0x0","targetUserSid":"S-1-5-18","targetUserName":"dc$","targetDomainName":"DOMAIN.LOCAL","targetLogonId":"0x65d2e058","logonType":"3","logonProcessName":"Kerberos","authenticationPackageName":"Kerberos","logonGuid":"{6545b3b8-bcab-ebaf-92c1-233d494d0c35}","keyLength":"0","processId":"0x0","ipAddress":"127.0.0.1","ipPort":"60336","impersonationLevel":"%%1833","virtualAccount":"%%1843","targetLinkedLogonId":"0x0","elevatedToken":"%%1842"}}},"location":"EventChannel"}'
2024/11/12 14:25:31 wazuh-maild: WARNING: Invalid JSON alert read from 'logs/alerts/alerts.json': 'ion":"0","level":"0","task":"13824","opcode":"0","keywords":"0x8020000000000000","systemTime":"2024-11-12T13:25:29.540057700Z","eventRecordID":"4654902770","processID":"664","threadID":"7808","channel":"Security","computer":"moon.DOMAIN.local","severityValue":"AUDIT_SUCCESS","message":"\"Se habilitó una cuenta de usuario.\r\n\r\nSujeto:\r\n\tId. de seguridad:\t\tS-1-5-21-1004336348-1177238915-682003330-2186\r\n\tNombre de cuenta:\t\tdummy_user\r\n\tDominio de cuenta:\t\tDOMAIN\r\n\tId. de inicio de sesión:\t\t0x50FF7827\r\n\r\nCuenta de destino:\r\n\tId. de seguridad:\t\tS-1-5-21-1004336348-1177238915-682003330-7123\r\n\tNombre de cuenta:\t\ttest_user\r\n\tDominio de cuenta:\t\tDOMAIN\""},"eventdata":{"targetUserName":"test_user","targetDomainName":"DOMAIN","targetSid":"S-1-5-21-1004336348-1177238915-682003330-7123","subjectUserSid":"S-1-5-21-1004336348-1177238915-682003330-2186","subjectUserName":"dummy_user","subjectDomainName":"DOMAIN","subjectLogonId":"0x50ff7827"}}},"location":"EventChannel"}'
2024/11/12 14:25:59 wazuh-maild: WARNING: Invalid JSON alert read from 'logs/alerts/alerts.json': 'enta de destino:\r\n\tId. de seguridad:\t\tS-1-5-21-1004336348-1177238915-682003330-7123\r\n\tNombre de cuenta:\t\ttest_user\r\n\tDominio de cuenta:\t\tDOMAIN\""},"eventdata":{"targetUserName":"test_user","targetDomainName":"DOMAIN","targetSid":"S-1-5-21-1004336348-1177238915-682003330-7123","subjectUserSid":"S-1-5-21-1004336348-1177238915-682003330-2186","subjectUserName":"dummy_user","subjectDomainName":"DOMAIN","subjectLogonId":"0x51004e77"}}},"location":"EventChannel"}'
2024/11/12 14:26:01 wazuh-maild: WARNING: Invalid JSON alert read from 'logs/alerts/alerts.json': 'roceso:\t\t0x0\r\n\tNombre de proceso:\t\t-\r\n\r\nInformación de red:\r\n\tNombre de estación de trabajo:\t-\r\n\tDirección de red de origen:\t127.0.0.1\r\n\tPuerto de origen:\t\t60359\r\n\r\nInformación de autenticación detallada:\r\n\tProceso de inicio de sesión:\t\tKerberos\r\n\tPaquete de autenticación:\tKerberos\r\n\tServicios transitados:\t-\r\n\tNombre de paquete (solo NTLM):\t-\r\n\tLongitud de clave:\t\t0\r\n\r\nEste evento se genera cuando se crea una sesión de inicio. Lo genera el equipo al que se tuvo acceso.\r\n\r\nLos campos de firmante indican la cuenta del sistema local que solicitó el inicio de sesión. Suele ser un servicio como el servicio de servidor o un proceso local como Winlogon.exe o Services.exe.\r\n\r\nEl campo Tipo de inicio de sesión indica la clase de inicio de sesión que se realizó. Los tipos más comunes son 2 (interactivo) y 3 (red).\r\n\r\nLos campos Nuevo inicio de sesión indican la cuenta para la que se creó el nuevo inicio de sesión, es decir, aquella en la que se inició la sesión.\r\n\r\nLos campos de red indican dónde se originó una solicitud de inicio de sesión remota. Nombre de estación de trabajo no está siempre disponible y se puede dejar en blanco en algunos casos.\r\n\r\nEl campo de nivel de suplantación indica en qué medida un proceso en la sesión de inicio de sesión puede suplantar.\r\n\r\nLos campos de información de autenticación proporcionan información detallada sobre esta solicitud de inicio de sesión específica.\r\n\t- GUID de inicio de sesión es un identificador único que se puede usar para correlacionar este evento con un evento KDC.\r\n\t- Servicios transitados indica los servicios intermedios que participaron en esta solicitud de inicio de sesión.\r\n\t- Nombre de paquete indica el subprotocolo que se usó entre los protocolos NTLM.\r\n\t- Longitud de clave indica la longitud de la clave de sesión generada. Será 0 si no se solicitó una clave de sesión.\""},"eventdata":{"subjectUserSid":"S-1-0-0","subjectLogonId":"0x0","targetUserSid":"S-1-5-18","targetUserName":"dc$","targetDomainName":"DOMAIN.LOCAL","targetLogonId":"0x65d3bedb","logonType":"3","logonProcessName":"Kerberos","authenticationPackageName":"Kerberos","logonGuid":"{6545b3b8-bcab-ebaf-92c1-233d494d0c35}","keyLength":"0","processId":"0x0","ipAddress":"127.0.0.1","ipPort":"60359","impersonationLevel":"%%1833","virtualAccount":"%%1843","targetLinkedLogonId":"0x0","elevatedToken":"%%1842"}}},"location":"EventChannel"}'

Please let me know if you need any other information.

Best regards.

[Ryan Allen]

I'm also having the exact same issue.

Currently I only have 2 Linux machines reporting to Wazuh, same configuration one works fine and the other isn't reporting any journald logs. The only fix thus far has been restarting the Wazuh-Agent service.

Any help would be great!

[Obinna Uchubilo]

Hello Francesc,

There is a current  Github issue detailing the problem with the Journald and is being investigated. You can use this link to stay updated on the issue.

Regards