help with never active connections from agent

[kongf...@gmail.com]

I am running 

wazuh-manager-4.2.5-1.x86_64 on an amazon linux 2 instance

on several other servers all running amazon linux 2 I am running the following agent:

wazuh-agent-4.2.5-1.x86_64

I am able to register them and when I look at the manager running

/var/ossec/bin/manage_agents

I see them listed so I know they registered successfully.  But in wazuh itself it always shows them as never active

Here is what I see in the logs on the manager:

2021/12/14 15:44:40 wazuh-authd: INFO: New connection from 172.x.x.x
2021/12/14 15:44:40 wazuh-authd: INFO: Received request for a new agent (servername) from: 172.x.x.x
2021/12/14 15:44:40 wazuh-authd: INFO: Duplicate name 'servername' (002).
2021/12/14 15:44:40 wazuh-authd: INFO: Agent '002' key already exists on the manager.
2021/12/14 15:51:20 wazuh-authd: INFO: New connection from 172.x.x.x
2021/12/14 15:51:20 wazuh-authd: INFO: Received request for a new agent (servername) from: 172.x.x.x
2021/12/14 15:51:20 wazuh-authd: INFO: Duplicate name 'servername' (003).
2021/12/14 15:51:20 wazuh-authd: INFO: Agent '003' key already exists on the manager.
2021/12/14 15:52:03 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2021/12/14 15:52:03 wazuh-modulesd:syscollector: INFO: Evaluation finished.

Now I notice the two servers keep trying a new connecting and trying to request a key over and over again every few minutes, even though they are already registered

Here is what I see in the client log after restarting the agent:

2021/12/14 16:06:26 wazuh-modulesd:syscollector: INFO: Stop received for Syscollector.
2021/12/14 16:06:26 wazuh-modulesd:syscollector: INFO: Module finished.
2021/12/14 16:06:26 wazuh-logcollector: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2021/12/14 16:06:26 wazuh-syscheckd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2021/12/14 16:06:27 wazuh-agentd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2021/12/14 16:06:27 wazuh-execd: INFO: (1314): Shutdown received. Deleting responses.
2021/12/14 16:06:27 wazuh-execd: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2021/12/14 16:06:27 wazuh-execd: INFO: Started (pid: 9122).
2021/12/14 16:06:28 wazuh-agentd: INFO: (1410): Reading authentication keys file.
2021/12/14 16:06:28 wazuh-agentd: INFO: Using notify time: 10 and max time to reconnect: 60
2021/12/14 16:06:28 wazuh-agentd: INFO: Version detected -> Linux | servername  [Linux|linux: 0.0] - Wazuh v4.2.5
2021/12/14 16:06:28 wazuh-agentd: INFO: Started (pid: 9134).
2021/12/14 16:06:28 wazuh-agentd: INFO: Server IP Address: 172.x.x.x
2021/12/14 16:06:28 wazuh-agentd: INFO: Using AES as encryption method.
2021/12/14 16:06:28 wazuh-agentd: INFO: Trying to connect to server (172.x.x.x1514/tcp).
2021/12/14 16:06:29 wazuh-syscheckd: INFO: Started (pid: 9149).
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6003): Monitoring path: '/bin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6003): Monitoring path: '/boot', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6003): Monitoring path: '/etc', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6003): Monitoring path: '/sbin', with options 'size | permissions | owner | group | mtime | inode | has
h_md5 | hash_sha1 | hash_sha256 | scheduled'.
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6003): Monitoring path: '/usr/bin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6003): Monitoring path: '/usr/sbin', with options 'size | permissions | owner | group | mtime | inode | hash_md5 | hash_sha1 | hash_sha256 | scheduled'.
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mtab'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/hosts.deny'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mail/statistics'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random-seed'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random.seed'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/adjtime'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/httpd/logs'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/utmpx'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/wtmpx'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/cups/certs'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/dumpdates'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/svc/volatile'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6207): Ignore 'file' sregex '.log$|.swp$'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6004): No diff for file: '/etc/ssl/private.key'
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6000): Starting daemon...
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6010): File integrity monitoring scan frequency: 43200 seconds
2021/12/14 16:06:29 wazuh-syscheckd: INFO: (6008): File integrity monitoring scan started.
2021/12/14 16:06:30 wazuh-logcollector: INFO: Monitoring output of command(360): df -P
2021/12/14 16:06:30 wazuh-logcollector: INFO: Monitoring full output of command(360): netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \
+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d
2021/12/14 16:06:30 wazuh-logcollector: INFO: Monitoring full output of command(360): last -n 20
2021/12/14 16:06:30 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.
2021/12/14 16:06:30 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/access_log'.
2021/12/14 16:06:30 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/audit/audit.log'.
2021/12/14 16:06:30 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/ossec/logs/active-responses.log'.
2021/12/14 16:06:30 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/messages'.
2021/12/14 16:06:30 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/secure'.
2021/12/14 16:06:30 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/maillog'.
2021/12/14 16:06:30 wazuh-logcollector: INFO: Started (pid: 9162).
2021/12/14 16:06:31 wazuh-modulesd: INFO: Started (pid: 9180).
2021/12/14 16:06:31 wazuh-modulesd:control: INFO: Starting control thread.
2021/12/14 16:06:31 wazuh-modulesd:agent-upgrade: INFO: (8153): Module Agent Upgrade started.
2021/12/14 16:06:31 sca: INFO: Module started.
2021/12/14 16:06:31 wazuh-modulesd:osquery: INFO: Module disabled. Exiting...
2021/12/14 16:06:31 wazuh-modulesd:ciscat: INFO: Module disabled. Exiting...
2021/12/14 16:06:31 sca: INFO: Loaded policy '/var/ossec/ruleset/sca/sca_unix_audit.yml'
2021/12/14 16:06:31 sca: INFO: Starting Security Configuration Assessment scan.
2021/12/14 16:06:31 sca: INFO: Starting evaluation of policy: '/var/ossec/ruleset/sca/sca_unix_audit.yml'
2021/12/14 16:06:31 wazuh-modulesd:syscollector: INFO: Module started.
2021/12/14 16:06:31 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2021/12/14 16:06:31 wazuh-modulesd:syscollector: INFO: Evaluation finished.

I am not sure why if registered it not showing as Active agent right now I have zero listed, even though two are registered.  but undere Never connected agents it shows the value of 2

***************************************
* Wazuh v4.2.5 Agent manager.          *
* The following options are available: *
****************************************
   (A)dd an agent (A).
   (E)xtract key for an agent (E).
   (L)ist already added agents (L).
   (R)emove an agent (R).
   (Q)uit.
Choose your action: A,E,L,R or Q: L

Available agents:
   ID: 002, Name: server1, IP: any
   ID: 003, Name: server2, IP: any

Thanks for any help in advance.

[Matias Pereyra]

Hello! Thanks for using Wazuh!

When the agent has been registered and it has a valid key, it tries to connect to the manager in the port 1514. After several failed attempts, it'll request a new key automatically (auto-enrollment) to authd in port 1515. The manager's rejects the request, because the agent already has a valid key

        Agent '002' key already exists on the manager.

If you want to check the connection status of the agents, use agent_control instead of manage_agents like this

    /var/ossec/bin/agent_control -l

Now if both are in a never connected state, something is preventing them to report to the manager. But the registration request in port 1515 arrives the manager successfully. Also, the portion of the logs you've shared from the agent doesn't show any message related to a connectivity issue.

So, please, can you upload the full ossec.log files from both agent and manager?

Can you ping the manager's 1514 port from the agents?

Regards.

[kongf...@gmail.com]

Thanks for your reply, 

 so I was using old version of wazuh  3.x on a different instance  and had UDP 1514 open, but I guess in version 4.x the default is TCP 1514, once I opened that up, it now connects!

Thanks again and simple fix.

[Matias Pereyra]

Glad to hear it's solved!

Now TCP is used by default, you are right.

Thanks to you.
Regards.