ARM-based wazuh server's "ossec-remoted" process dies when trying to connect a Linux/Win agent

225 views

Skip to first unread message

mcarn...@nextel.es

unread,

Oct 31, 2018, 12:54:07 PM10/31/18

to Wazuh mailing list

Hello,

We're trying to get Wazuh running on a RaspberryPi with Raspbian OS. We have successfully compiled and run the Wazuh Server v3.6.1 and this is the output status that we get before connecting any agent:

root@bc50cca57dff:/var/ossec/bin# ./ossec-control status

wazuh-clusterd not running...

wazuh-modulesd not running...

ossec-monitord is running...

ossec-logcollector is running...

ossec-remoted is running...

ossec-syscheckd is running...

ossec-analysisd is running...

ossec-maild not running...

ossec-execd is running...

wazuh-db is running...

As you can see, the "ossec-remoted" process is up and running. Then, we succesfully register a remote agent:

****************************************

* Wazuh v3.6.1 Agent manager. *

* The following options are available: *

****************************************

(A)dd an agent (A).

(E)xtract key for an agent (E).

(L)ist already added agents (L).

(R)emove an agent (R).

(Q)uit.

Choose your action: A,E,L,R or Q: L

Available agents:

ID: 003, Name: debianAgent01, IP: 192.168.15.101

The problem comes when the agent is started. Taking a look at the /var/ossec/logs/ossec.log (on the agent host) file it seems that the connection is succesfully established but after a while some ERROR messages start to appear:

2018/10/31 16:45:00 ossec-execd: INFO: No option <ca_store> defined. Using Wazuh default CA (/var/ossec/etc/wpk_root.pem).

2018/10/31 16:45:00 ossec-execd: INFO: Started (pid: 20377).

2018/10/31 16:45:00 ossec-agentd: WARNING: The <server-ip> tag is deprecated, please use <server><address> instead.

2018/10/31 16:45:00 ossec-agentd: WARNING: The <protocol> tag is deprecated, please use <server><protocol> instead.

2018/10/31 16:45:00 ossec-agentd: WARNING: The <length> tag is deprecated for version newer than 2.1.1, please use <queue_size> instead.

2018/10/31 16:45:00 ossec-agentd: INFO: Using notify time: 10 and max time to reconnect: 60

2018/10/31 16:45:00 ossec-agentd: INFO: (1410): Reading authentication keys file.

2018/10/31 16:45:00 ossec-agentd: INFO: Using AES as encryption method.

2018/10/31 16:45:00 ossec-agentd: INFO: Started (pid: 20383).

2018/10/31 16:45:00 ossec-agentd: INFO: Server IP Address: 192.168.20.2

2018/10/31 16:45:00 ossec-agentd: INFO: Trying to connect to server (192.168.20.2:1514/udp).

2018/10/31 16:45:00 wazuh-modulesd: INFO: Process started.

2018/10/31 16:45:00 wazuh-modulesd:oscap: INFO: Module disabled. Exiting...

2018/10/31 16:45:01 ossec-agentd: INFO: (4102): Connected to the server (192.168.20.2:1514/udp).

2018/10/31 16:45:03 ossec-syscheckd: INFO: Started (pid: 20390).

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/mtab'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/hosts.deny'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/mail/statistics'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/random-seed'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/random.seed'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/adjtime'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/httpd/logs'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/utmpx'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/wtmpx'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/cups/certs'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/dumpdates'

2018/10/31 16:45:03 ossec-syscheckd: INFO: Ignoring: '/etc/svc/volatile'

2018/10/31 16:45:03 ossec-syscheckd: INFO: No diff for file: '/etc/ssl/private.key'

2018/10/31 16:45:03 rootcheck: INFO: Started (pid: 20390).

2018/10/31 16:45:06 ossec-logcollector: INFO: Monitoring output of command(360): df -P

2018/10/31 16:45:06 ossec-logcollector: INFO: Monitoring full output of command(360): netstat -tulpen | sort

2018/10/31 16:45:06 ossec-logcollector: INFO: Monitoring full output of command(360): last -n 20

2018/10/31 16:45:06 ossec-logcollector: INFO: (1950): Analyzing file: '/var/ossec/logs/active-responses.log'.

2018/10/31 16:45:06 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/messages'.

2018/10/31 16:45:06 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/auth.log'.

2018/10/31 16:45:06 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/syslog'.

2018/10/31 16:45:06 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/dpkg.log'.

2018/10/31 16:45:06 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/kern.log'.

2018/10/31 16:45:06 ossec-logcollector: INFO: Started (pid: 20394).

2018/10/31 16:45:18 ossec-syscheckd: INFO: Syscheck scan frequency: 43200 seconds

2018/10/31 16:45:18 rootcheck: INFO: Starting rootcheck scan.

2018/10/31 16:45:33 ossec-syscheckd: INFO: Starting syscheck scan (forwarding database).

2018/10/31 16:45:33 ossec-syscheckd: INFO: Starting syscheck database (pre-scan).

2018/10/31 16:46:02 ossec-agentd: WARNING: Server unavailable. Setting lock.

2018/10/31 16:46:02 ossec-agentd: WARNING: Process locked due to agent is offline. Waiting for connection...

2018/10/31 16:46:02 ossec-syscheckd: WARNING: Process locked due to agent is offline. Waiting for connection...

2018/10/31 16:46:03 ossec-logcollector: WARNING: Process locked due to agent is offline. Waiting for connection...

2018/10/31 16:46:12 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

2018/10/31 16:46:16 ossec-syscheckd: WARNING: Process locked due to agent is offline. Waiting for connection...

2018/10/31 16:46:24 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

2018/10/31 16:46:25 ossec-agentd: WARNING: (4101): Waiting for server reply (not started). Tried: '192.168.20.2'.

2018/10/31 16:46:27 ossec-agentd: INFO: Trying to connect to server (192.168.20.2:1514/udp).

2018/10/31 16:46:37 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

2018/10/31 16:46:49 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

2018/10/31 16:46:50 ossec-agentd: WARNING: (4101): Waiting for server reply (not started). Tried: '192.168.20.2'.

2018/10/31 16:47:01 ossec-agentd: INFO: Trying to connect to server (192.168.20.2:1514/udp).

2018/10/31 16:47:11 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

2018/10/31 16:47:23 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

2018/10/31 16:47:24 ossec-agentd: WARNING: (4101): Waiting for server reply (not started). Tried: '192.168.20.2'.

2018/10/31 16:47:35 ossec-agentd: INFO: Trying to connect to server (192.168.20.2:1514/udp).

2018/10/31 16:47:45 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

2018/10/31 16:47:57 ossec-agentd: ERROR: (1218): Unable to send message to 'server': Connection refused

Going back to the Wazuh-server host now we can see that the "ossec-remoted" process is not running anymore:

root@bc50cca57dff:/var/ossec/bin# ./ossec-control status

wazuh-clusterd not running...

wazuh-modulesd not running...

ossec-monitord is running...

ossec-logcollector is running...

ossec-remoted: Process 18679 not used by ossec, removing...

ossec-remoted not running...

ossec-syscheckd is running...

ossec-analysisd is running...

ossec-maild not running...

ossec-execd is running...

wazuh-db is running...

The Wazuh server (with all the processes) has been running successfuly for hours and only when the agent has been launched the "ossec-remoted" process has stopped.

Any ideas of what could be the problem? thanks in advance for your help.

Best Regards,

Manuel Carnerero

Chema Martinez

unread,

Nov 15, 2018, 8:44:41 AM11/15/18

to mcarn...@nextel.es, wa...@googlegroups.com

Hi Manuel,

I have been able to reproduce it in a Raspberry Pi 1 running Wazuh v3.7.0. But I have not determined the source of the crash yet.

Registering and connecting agents to the manager cause the crash of the remote daemon suddenly.

Which model of Raspberry are you using? Could you check the stack size set by default for your OS by running ulimit -a?

Thank you.

Best regards.

Chema Martinez | IT Engineer — Wazuh, Inc.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/b96582ea-dddc-43f4-829d-c4b56bb25309%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

mcarn...@nextel.es

unread,

Nov 27, 2018, 3:06:11 PM11/27/18

to Wazuh mailing list

Hi Chema,

The Raspberry model is Pi 3 and the stack size is 8192 kbytes. We also tried with a Pi 2 previously.

Thanks,

Manuel Carnerero

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

Victor Fernandez

unread,

Jan 5, 2019, 7:51:48 AM1/5/19

to mcarn...@nextel.es, Wazuh mailing list

Hi Manuel and Chema,

We finally found the cause of this bug. It was due to an invalid interpretation of the data structures size.

This is the PR that fixes the bug: #2214. We will merge it into branch 3.8 so version v3.8.0 will have this patch.

Please feel free to apply this fix to your source code: simply edit the file src/wazuh_modules/wmodules.h and make sure that "shared.h" is the very first include in the source:

#include "shared.h"
#include <pthread.h>
#include "config/config.h"

Then, repeat this step on src/remoted/secure.c —it is already fixed in 3.8—:

#include "shared.h"

#if defined(__linux__)
#include <sys/epoll.h>
#elif defined(__MACH__) || defined(__FreeBSD__) || defined(__OpenBSD__)
#include <sys/types.h>
#include <sys/event.h>
#endif /* __linux__ */

#include "os_net/os_net.h"
#include "remoted.h"

And finally the same change on src/remoted/sendmsg.c:

#include "shared.h"
#include <pthread.h>
#include "remoted.h"
#include "os_net/os_net.h"

Then remove the implied object files and recompile the project:

cd src
rm wazuh_modules/wm_database.o remoted/secure.o remoted/sendmsg.o
make TARGET=server -j4
cd ..
sudo ./install.sh

Hope this helps. Thank you for reporting this bug, we appreciate your detailed description.

Best regards,

Victor Manuel Fernandez-Castro
Core Engineering | vic...@wazuh.com

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/b96582ea-dddc-43f4-829d-c4b56bb25309%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/ee098d20-03f6-4ec8-b398-9638ae074bf3%40googlegroups.com.

Reply all

Reply to author

Forward

0 new messages