[Wazuh-AP] Duplicate ip field for various agent nodes
428 views
Skip to first unread message
Vijayakumar U
Jul 24, 2019, 7:32:32 AM7/24/19
to Wazuh mailing list
Hi Team,
Below is our Setup Details :
Wazuh Cluster environment with one master & 3 Workers nodes with 6k+ agents.
OS : CentOS 7.
Server, Agent & API - Version : 3.9.0
While getting agents information using Wazuh-API we get same ip for different agent nodes. Attached a sample API response.
Not sure whether it is API bug or info stored inside DB itself wrong. In which DB I need to verify this?
All the agents were registered using below command.
/var/ossec/bin/agent-auth -i -m hids-master
- In client.keys and in manage_agents -l it is showing the proper IP.
- Inside agent's file under dir /var/ossec/queue/agent-info, _agent_ip is having wrong value.
Had a look at this discussion https://groups.google.com/forum/#!msg/wazuh/8Myi1GGVt4Y/MTlN2XvsCAAJ and this issue https://github.com/wazuh/wazuh/issues/3457 too.
But, still opening a new discussion, Because, 400+ agents were affected.
Let me know if any other details needed from my side.
Thanks and Regards,
Vijay.
Vijay.
Daniel Ruiz
Jul 25, 2019, 5:04:51 AM7/25/19
to Wazuh mailing list
Hi Vijay,
Having had a look to the API
response you have attached, it does not seem a Wazuh API bug to me. Since you
are running a cluster, I assume that you have a LB with IP address
192.168.187.10. If you study the API response carefully, you will see a field registerIP which probably has the actual IP of the agent.
If my asumption is correct, the problem here may be related with the way you are registrating agents. Firstly, I recommend checking your ossec.conf file auth configuration and ensure you have set the use_source_ip option to "no":
<!-- Configuration for ossec-authd -->
<auth>
...
<use_source_ip>no</use_source_ip>
...
</auth>
Furthermore, you should avoid the -i option when registering agents with agent-auth behind a LB, otherwise the IP value will be taken from the LB itself instead of the agent. If you check documentation in https://documentation.wazuh.com/current/user-manual/reference/tools/agent-auth.html, the option -i "Lets the agent IP address be set by the manager connection.". Since the manager sees the agent through the LB, you don't want the manager to assign the IP.
I have tested both options in a simulated cluster using Docker and nginx as LB, without the -i option I see this, which is correct:
root@4a2e73e63209:/# /var/ossec/bin/manage_agents -l
Available agents:
ID: 001, Name: 112160d17534, IP: any
ID: 002, Name: b43184ec634e, IP: any
ID: 003, Name: abbd97e0da60, IP: any
ID: 004, Name: 5e5e1a0233a9, IP: any
ID: 005, Name: 5fdffd13ce2a, IP: any
root@4a2e73e63209:/# /var/ossec/bin/cluster_control -a
ID NAME IP STATUS VERSION NODE NAME
000 4a2e73e63209 127.0.0.1 Active Wazuh v3.9.3 master-node
001 112160d17534 172.21.0.6 Active Wazuh v3.9.3 master-node
002 b43184ec634e 172.21.0.7 Active Wazuh v3.9.3 worker1
003 abbd97e0da60 172.21.0.8 Active Wazuh v3.9.3 worker1
004 5e5e1a0233a9 172.21.0.9 Active Wazuh v3.9.3 master-node
005 5fdffd13ce2a 172.21.0.11 Active Wazuh v3.9.3 worker1
root@4a2e73e63209:/# curl -u foo:bar http://localhost:55000/agents?pretty
{
"error": 0,
"data": {
"items": [
{
"os": {
"arch": "x86_64",
"codename": "Bionic Beaver",
"major": "18",
"minor": "04",
"name": "Ubuntu",
"platform": "ubuntu",
"uname": "Linux |4a2e73e63209 |4.15.0-54-generic |#58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 |x86_64",
"version": "18.04.1 LTS"
},
"status": "Active",
"id": "000",
"node_name": "master-node",
"registerIP": "127.0.0.1",
"lastKeepAlive": "9999-12-31 23:59:59",
"name": "4a2e73e63209",
"dateAdd": "2019-07-25 10:21:57",
"manager": "4a2e73e63209",
"ip": "127.0.0.1",
"version": "Wazuh v3.9.3"
},
{
"os": {
"arch": "x86_64",
"codename": "Bionic Beaver",
"major": "18",
"minor": "04",
"name": "Ubuntu",
"platform": "ubuntu",
"uname": "Linux |112160d17534 |4.15.0-54-generic |#58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 |x86_64",
"version": "18.04.1 LTS"
},
"status": "Active",
"id": "001",
"configSum": "ab73af41699f13fdd81903b5f23d8d00",
"node_name": "master-node",
"registerIP": "any",
"lastKeepAlive": "2019-07-25 10:58:05",
"group": [
"default"
],
"mergedSum": "c6309ff81a74781f6b55b68129a76738",
"name": "112160d17534",
"dateAdd": "2019-07-25 10:41:11",
"manager": "4a2e73e63209",
"ip": "172.21.0.6",
"version": "Wazuh v3.9.3"
},
{
"os": {
"arch": "x86_64",
"codename": "Bionic Beaver",
"major": "18",
"minor": "04",
"name": "Ubuntu",
"platform": "ubuntu",
"uname": "Linux |b43184ec634e |4.15.0-54-generic |#58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 |x86_64",
"version": "18.04.1 LTS"
},
"status": "Active",
"id": "002",
"configSum": "ab73af41699f13fdd81903b5f23d8d00",
"node_name": "worker1",
"registerIP": "any",
"lastKeepAlive": "2019-07-25 10:58:04",
"group": [
"default"
],
"mergedSum": "c6309ff81a74781f6b55b68129a76738",
"name": "b43184ec634e",
"dateAdd": "2019-07-25 10:41:13",
"manager": "7eb4dea22ce0",
"ip": "172.21.0.7",
"version": "Wazuh v3.9.3"
},
{
"os": {
"arch": "x86_64",
"codename": "Bionic Beaver",
"major": "18",
"minor": "04",
"name": "Ubuntu",
"platform": "ubuntu",
"uname": "Linux |abbd97e0da60 |4.15.0-54-generic |#58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 |x86_64",
"version": "18.04.1 LTS"
},
"status": "Active",
"id": "003",
"configSum": "ab73af41699f13fdd81903b5f23d8d00",
"node_name": "worker1",
"registerIP": "any",
"lastKeepAlive": "2019-07-25 10:58:05",
"group": [
"default"
],
"mergedSum": "c6309ff81a74781f6b55b68129a76738",
"name": "abbd97e0da60",
"dateAdd": "2019-07-25 10:41:14",
"manager": "7eb4dea22ce0",
"ip": "172.21.0.8",
"version": "Wazuh v3.9.3"
},
{
"os": {
"arch": "x86_64",
"codename": "Bionic Beaver",
"major": "18",
"minor": "04",
"name": "Ubuntu",
"platform": "ubuntu",
"uname": "Linux |5e5e1a0233a9 |4.15.0-54-generic |#58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 |x86_64",
"version": "18.04.1 LTS"
},
"status": "Active",
"id": "004",
"configSum": "ab73af41699f13fdd81903b5f23d8d00",
"node_name": "master-node",
"registerIP": "any",
"lastKeepAlive": "2019-07-25 10:57:59",
"group": [
"default"
],
"mergedSum": "c6309ff81a74781f6b55b68129a76738",
"name": "5e5e1a0233a9",
"dateAdd": "2019-07-25 10:41:15",
"manager": "4a2e73e63209",
"ip": "172.21.0.9",
"version": "Wazuh v3.9.3"
},
{
"os": {
"arch": "x86_64",
"codename": "Bionic Beaver",
"major": "18",
"minor": "04",
"name": "Ubuntu",
"platform": "ubuntu",
"uname": "Linux |5fdffd13ce2a |4.15.0-54-generic |#58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 |x86_64",
"version": "18.04.1 LTS"
},
"status": "Active",
"id": "005",
"configSum": "ab73af41699f13fdd81903b5f23d8d00",
"node_name": "worker1",
"registerIP": "any",
"lastKeepAlive": "2019-07-25 10:58:02",
"group": [
"default"
],
"mergedSum": "c6309ff81a74781f6b55b68129a76738",
"name": "5fdffd13ce2a",
"dateAdd": "2019-07-25 10:41:18",
"manager": "7eb4dea22ce0",
"ip": "172.21.0.11",
"version": "Wazuh v3.9.3"
}
],
"totalItems": 6
}
}
Please, give it a try and tell me if your issue gets solved.
Regards,
Dani
Vijayakumar U
Jul 25, 2019, 8:52:13 AM7/25/19
to Daniel Ruiz, Wazuh mailing list
Hi Daniel,
Thank you the detailed explanation. 192.168.187.10 is not my LB ip address. registerIP shows the correct ip details.
The use_source_ip is set to "no". Pasted below the configuration of <auth> from Master.
<!-- Configuration for ossec-authd -->
<auth>
<disabled>no</disabled>
<port>1515</port>
<use_source_ip>no</use_source_ip>
<force_insert>yes</force_insert>
<force_time>0</force_time>
<purge>yes</purge>
<use_password>no</use_password>
<limit_maxagents>yes</limit_maxagents>
<ciphers>HIGH:!ADH:!EXP:!MD5:!RC4:!3DES:!CAMELLIA:@STRENGTH</ciphers>
<!-- <ssl_agent_ca></ssl_agent_ca> -->
<ssl_verify_host>no</ssl_verify_host>
<ssl_manager_cert>/var/ossec/etc/sslmanager.cert</ssl_manager_cert>
<ssl_manager_key>/var/ossec/etc/sslmanager.key</ssl_manager_key>
<ssl_auto_negotiate>no</ssl_auto_negotiate>
</auth>
I don't think the -i option is creating problem here. Almost more than 90% of the agents were displaying normally in API and manage_agents -l lists all of them properly (100%).
I don't hold full control on all the agent machines, So that I can just test in few machines without option -i . The registration has been automated and handed over to the other team. Attached the script for reference.
Is there any other way to ensure that -i is creating problem?
Can I check the db files in Master? Let me know which db file and table to be checked here.
Thanks and Regards,
Vijay.
As the registration and
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e1f97f73-60e3-400f-b7e1-761aea8f1916%40googlegroups.com.
Cheers,
Vijay.
Vijay.
Daniel Ruiz
Jul 25, 2019, 11:25:37 AM7/25/19
to Wazuh mailing list
Hi Vijay,
once you have told me that ip is not from your LB, I have being asking to the team for any other issues and we think your problem could be related with this one: https://github.com/wazuh/wazuh/issues/3511. As you can see, it has been fixed for 3.9.4 release which is coming very soon.
In order to be sure of this, just check if the ip and registerIp field of the API response is always the same excluding the last digit of the registerIp.
If you still want to check the database, the Wazuh API takes the data from /var/ossec/var/db/global.db. In order to check these data execute:
# sqlite3 /var/ossec/var/db/global.db
sqlite> .headers on
sqlite> select id, name, ip, register_ip from agent;
id|name|ip|register_ip
0|4a2e73e63209|127.0.0.1|127.0.0.1
1|112160d17534|172.21.0.6|any
2|b43184ec634e|172.21.0.7|any
3|abbd97e0da60|172.21.0.8|any
4|5e5e1a0233a9|172.21.0.9|any
5|5fdffd13ce2a|172.21.0.11|anyHowever, I still recommend not using the -i option, although as you pointed, it is not the cause of the issue apparently.
Let me know if we hit the issue.
Thanks.
Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e1f97f73-60e3-400f-b7e1-761aea8f1916%40googlegroups.com.
--Cheers,
Vijay.
Vijayakumar U
Aug 1, 2019, 8:52:10 AM8/1/19
to Daniel Ruiz, Wazuh mailing list
Hi Daniel,
Yes. ip and registerIp field just differs by the last digit. Thanks for your clarifications.
Few entries were having empty ip field in db and attached the output for reference.
Thanks and Regards,
Vijay.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e1f97f73-60e3-400f-b7e1-761aea8f1916%40googlegroups.com.
----Cheers,
Vijay.
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/49cbff34-0d05-439a-b39a-81922e19299b%40googlegroups.com.
Cheers,
Vijay.
Vijay.
David José Iglesias Lopez
Aug 6, 2019, 2:59:03 AM8/6/19
to Wazuh mailing list
Hello Vijay,
Thank you for your contributions, it helps us keep improving. I will pass the referenced output to the team.
Please do not hesitate to contact back if you have any other issues.
Best regards,
David J. Iglesias
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e1f97f73-60e3-400f-b7e1-761aea8f1916%40googlegroups.com.
--Cheers,
Vijay.
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/49cbff34-0d05-439a-b39a-81922e19299b%40googlegroups.com.
--Cheers,
Vijay.
Reply all
Reply to author
Forward
0 new messages