OSSEC Agents Not Communicating with Server

[Marc Baker]

OSSEC agents this morning were working without issue and then began reporting as Disconnected. Agent logs are returning the following error:

2017/03/27 10:14:38 ossec-agent: WARN: Process locked. Waiting for permission...

2017/03/27 10:14:49 ossec-agent(4101): WARN: Waiting for server reply (not started). Tried: '.

2017/03/27 10:14:51 ossec-agent: INFO: Trying to connect to server (:1514).

Nothing has changed on the server to the best of our knowledge. One anomaly we are seeing that may be related is the following when restarting Wazuh manager services:

Deleting PID file '/var/ossec/var/run/ossec-remoted-4816.pid' not used...

Killing ossec-monitord ..

Killing ossec-logcollector ..

ossec-remoted not running ..

Killing ossec-syscheckd ..

Killing ossec-analysisd ..

Killing ossec-maild ..

Killing ossec-execd ..

Killing wazuh-modulesd ..

Wazuh v2.0 Stopped

Starting Wazuh v2.0 (maintained by Wazuh Inc.)...

Started wazuh-modulesd...

Started ossec-maild...

Started ossec-execd...

Started ossec-analysisd...

Started ossec-logcollector...

Started ossec-remoted...

Started ossec-syscheckd...

Started ossec-monitord...

Completed.

/var/ossec/bin/ossec-analysisd -V

Wazuh v2.0 - Wazuh Inc.

This program is free software; you can redistribute it and/or modify

it under the terms of the GNU General Public License (version 2) as

published by the Free Software Foundation. For more details, go to

http://www.ossec.net/main/license/

/etc/ossec-init.conf

DIRECTORY="/var/ossec"

NAME="Wazuh"

VERSION="v2.0"

DATE="Wed Mar 15 11:38:44 UTC 2017"

TYPE="server"

Any suggestions for troubleshooting this issue would be greatly appreciated.

[Victor Fernandez]

Hi Marc,

It seems that Remoted crashed, this is the component that receives data from agents, so it should happen just when alerts stopped appearing, or 30 minutes approximately before alerts about disconnected agents appeared.

So it would be interesting if you could look for logs of that moment at manager's log, probably about process ossec-remoted, such:

2017/03/27 10:14:38 ossec-remoted: ERROR: (...)

Please try to find any log with errors or related to ossec-remoted and copy them to us so we could help you more easily.

Kind regards,

Victor.

[Marc Baker]

Victor,

Thank you for your response. The last keep-alive shown is 9:03:52:

Logs for that time-frame (

2017/03/27 06:53:42 ossec-syscheckd: INFO: Starting syscheck scan.

2017/03/27 07:01:34 ossec-syscheckd: INFO: Ending syscheck scan.

2017/03/27 07:11:34 rootcheck: INFO: Starting rootcheck scan.

2017/03/27 07:22:23 rootcheck: INFO: Ending rootcheck scan.

2017/03/27 09:24:42 ossec-monitord(1225): INFO: SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...

2017/03/27 09:24:42 ossec-logcollector(1225): INFO: SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...

2017/03/27 09:24:42 ossec-syscheckd(1225): INFO: SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...

2017/03/27 09:24:42 ossec-analysisd(1225): INFO: SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...

2017/03/27 09:24:42 ossec-execd(1314): INFO: Shutdown received. Deleting responses.

2017/03/27 09:24:42 ossec-execd(1225): INFO: SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...

2017/03/27 09:24:42 wazuh-modulesd: ERROR: Couldn't delete PID file.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0010-active-response_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0015-aix-ipsec_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0020-amazon_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0025-apache_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0030-arpwatch_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0035-asterisk_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0040-auditd_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0045-barracuda_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0050-checkpoint_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0055-cimserver_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0060-cisco-estreamer_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0065-cisco-ios_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0070-cisco-vpn_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0075-clamav_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0080-courier_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0085-dovecot_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0090-dragon-nids_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0095-dropbear_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0100-fortigate_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0105-freeipa_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0110-ftpd_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file ruleset/decoders/0115-grandstream_decoders.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading decoder file etc/decoders/local_decoder.xml.

2017/03/27 09:24:42 ossec-testrule: INFO: Reading the lists file: 'etc/lists/audit-keys'

2017/03/27 09:24:42 wazuh-modulesd: INFO: Process started.

2017/03/27 09:24:42 wazuh-modulesd:oscap: INFO: Module disabled. Exiting...

2017/03/27 09:24:42 wazuh-modulesd:database: INFO: Module started.

2017/03/27 09:24:42 ossec-execd: INFO: Started (pid: 704).

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0010-active-response_decoders.xml.

2017/03/27 09:24:42 wazuh-modulesd:oscap: INFO: Module disabled. Exiting...

2017/03/27 09:24:42 wazuh-modulesd:database: INFO: Module started.

2017/03/27 09:24:42 ossec-execd: INFO: Started (pid: 704).

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0010-active-response_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0015-aix-ipsec_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0020-amazon_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0025-apache_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0030-arpwatch_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0035-asterisk_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0040-auditd_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0045-barracuda_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0050-checkpoint_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0055-cimserver_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0060-cisco-estreamer_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0065-cisco-ios_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0070-cisco-vpn_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0075-clamav_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0080-courier_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0085-dovecot_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0090-dragon-nids_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0095-dropbear_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0100-fortigate_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0105-freeipa_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0110-ftpd_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0115-grandstream_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0120-horde_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0125-hp_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0130-imapd_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0135-imperva_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0140-kernel_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0145-mailscanner_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0150-mysql_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0155-named_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0160-netscaler_decoders.xml.

2017/03/27 09:24:42 ossec-remoted: INFO: Started (pid: 716).

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0165-netscreen_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0170-nginx_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0175-ntpd_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0180-openbsd_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0185-openldap_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0190-openvpn_decoders.xml.

2017/03/27 09:24:42 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0195-oscap_decoders.xml.

2017/03/27 09:29:15 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0115-grandstream_decoders.xml.

2017/03/27 09:29:15 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0120-horde_decoders.xml.

2017/03/27 09:29:15 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0125-hp_decoders.xml.

2017/03/27 09:29:15 ossec-remoted: INFO: Started (pid: 864).

2017/03/27 09:29:15 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0130-imapd_decoders.xml.

2017/03/27 09:29:15 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0135-imperva_decoders.xml.

2017/03/27 09:32:05 ossec-testrule: INFO: Reading the lists file: 'etc/lists/audit-keys'

2017/03/27 09:32:05 wazuh-modulesd: INFO: Process started.

2017/03/27 09:32:05 wazuh-modulesd:oscap: INFO: Module disabled. Exiting...

2017/03/27 09:32:05 wazuh-modulesd:database: INFO: Module started.

2017/03/27 09:32:05 ossec-execd: INFO: Started (pid: 1017).

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0010-active-response_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0015-aix-ipsec_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0020-amazon_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0025-apache_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0030-arpwatch_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0035-asterisk_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0040-auditd_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0045-barracuda_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0050-checkpoint_decoders.xml.

2017/03/27 09:32:05 ossec-remoted: INFO: Started (pid: 1029).

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0055-cimserver_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0060-cisco-estreamer_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0065-cisco-ios_decoders.xml.

2017/03/27 09:32:05 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0070-cisco-vpn_decoders.xml.

2017/03/27 09:34:26 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0155-named_decoders.xml.

2017/03/27 09:34:26 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0160-netscaler_decoders.xml.

2017/03/27 09:34:26 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0165-netscreen_decoders.xml.

2017/03/27 09:34:26 ossec-remoted: INFO: Started (pid: 1174).

2017/03/27 09:34:26 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0170-nginx_decoders.xml.

2017/03/27 09:34:26 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0175-ntpd_decoders.xml.

2017/03/27 09:38:07 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0155-named_decoders.xml.

2017/03/27 09:38:07 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0160-netscaler_decoders.xml.

2017/03/27 09:38:07 ossec-remoted: INFO: Started (pid: 1320).

2017/03/27 09:38:07 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0165-netscreen_decoders.xml.

2017/03/27 09:38:07 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0170-nginx_decoders.xml.

2017/03/27 10:06:57 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0115-grandstream_decoders.xml.

2017/03/27 10:06:57 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0120-horde_decoders.xml.

2017/03/27 10:06:57 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0125-hp_decoders.xml.

2017/03/27 10:06:57 ossec-remoted: INFO: Started (pid: 2923).

2017/03/27 10:06:57 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0130-imapd_decoders.xml.

2017/03/27 10:06:57 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0135-imperva_decoders.xml.

2017/03/27 10:06:57 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0140-kernel_decoders.xml.

2017/03/27 10:28:40 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0150-mysql_decoders.xml.

2017/03/27 10:28:40 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0155-named_decoders.xml.

2017/03/27 10:28:40 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0160-netscaler_decoders.xml.

2017/03/27 10:28:40 ossec-remoted: INFO: Started (pid: 4673).

2017/03/27 10:28:40 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0165-netscreen_decoders.xml.

2017/03/27 10:28:40 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0170-nginx_decoders.xml.

2017/03/27 10:28:40 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0175-ntpd_decoders.xml.

2017/03/27 10:28:40 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0180-openbsd_decoders.xml.

2017/03/27 10:30:12 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0170-nginx_decoders.xml.

2017/03/27 10:30:12 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0175-ntpd_decoders.xml.

2017/03/27 10:30:12 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0180-openbsd_decoders.xml.

2017/03/27 10:30:12 ossec-remoted: INFO: Started (pid: 4815).

2017/03/27 10:30:12 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0185-openldap_decoders.xml.

2017/03/27 10:30:12 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0190-openvpn_decoders.xml.

2017/03/27 10:37:18 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0155-named_decoders.xml.

2017/03/27 10:37:18 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0160-netscaler_decoders.xml.

2017/03/27 10:37:18 ossec-remoted: INFO: Started (pid: 5166).

2017/03/27 10:37:18 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0165-netscreen_decoders.xml.

2017/03/27 10:37:18 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0170-nginx_decoders.xml.

2017/03/27 10:37:18 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0175-ntpd_decoders.xml.

2017/03/27 10:37:18 ossec-analysisd: INFO: Reading decoder file ruleset/decoders/0180-openbsd_decoders.xml.

V/r

Marc Baker

[Marc Baker]

Victor,

I do not know if this matters but when I do "service wazuh-manager restart" an error is generated for remoted:

Deleting PID file '/var/ossec/var/run/ossec-remoted-7407.pid' not used...

We have a second Wazuh HIDS server at another location that is communicating with agents and this is not generated when the same command is run.An examination of the PIDs associated with the service does not match up to the PID being deleted so is there a possibility the system could be creating two remoted processes?

2017/03/27 12:23:59 ossec-remoted: INFO: Started (pid: 7664).

1	2017/03/27 12:21:50 ossec-remoted: INFO: Started (pid: 7405).

2	2017/03/27 12:21:21 ossec-remoted: INFO: Started (pid: 7247).

3

2017/03/27 10:37:18 ossec-remoted: INFO: Started (pid: 5166).

4

2017/03/27 10:30:12 ossec-remoted: INFO: Started (pid: 4815).

5

2017/03/27 10:28:40 ossec-remoted: INFO: Started (pid: 4673).

6	2017/03/27 10:10:41 ossec-remoted: INFO: Started (pid: 3075).

V/r

Marc

On Monday, March 27, 2017 at 11:23:33 AM UTC-4, Victor Fernandez wrote:

[Marc Baker]

During the install Jose had us use the link: https://documentation-dev.wazuh.com/installation-guide/installing-wazuh-server/wazuh_server_deb.html to install. We rebuilt a server per Jose's instruction and installed last Monday. The system has run without issue since.

Per your instruction I attempted to update the manager using sudo apt-get install wazuh-manager and received the following response:

Reading package lists... Done

Building dependency tree

Reading state information... Done

wazuh-manager is already the newest version.

0 upgraded, 0 newly installed, 0 to remove and 100 not upgraded.

After running this command I attempted a restart of the manager and received the PID error again:

Deleting PID file '/var/ossec/var/run/ossec-remoted-14267.pid' not used...

Killing ossec-monitord ..

Killing ossec-logcollector ..

ossec-remoted not running ..

Killing ossec-syscheckd ..

Killing ossec-analysisd ..

Killing ossec-maild ..

Killing ossec-execd ..

Killing wazuh-modulesd ..

Wazuh v2.0 Stopped

Starting Wazuh v2.0 (maintained by Wazuh Inc.)...

Started wazuh-modulesd...

Started ossec-maild...

Started ossec-execd...

Started ossec-analysisd...

Started ossec-logcollector...

Started ossec-remoted...

Started ossec-syscheckd...

Started ossec-monitord...

Completed.

Logs from the server for remoted:

0	2017/03/27 15:41:57 ossec-remoted: INFO: Started (pid: 14413).

1	2017/03/27 15:35:04 ossec-remoted: INFO: Started (pid: 14266).

2

2017/03/27 12:23:59 ossec-remoted: INFO: Started (pid: 7664).

3

2017/03/27 12:21:50 ossec-remoted: INFO: Started (pid: 7405).

4

2017/03/27 12:21:21 ossec-remoted: INFO: Started (pid: 7247).

5

2017/03/27 10:37:18 ossec-remoted: INFO: Started (pid: 5166).

2017/03/27 10:30:12 ossec-remoted: INFO: Started (pid: 4815).

Please let me know if there is another way to force an update of the manager or any other suggestions for troubleshooting. Thank you, Marc Baker

On Mon, Mar 27, 2017 at 2:34 PM, Victor Fernandez <vic...@wazuh.com> wrote:

Hi Marc,

both things (dangling PID file and no descriptive error at log) suggest that Remoted crashed. I see that you are using Wazuh 2.0, but that version has not been yet released and we made some changes on Remoted since we changed the version number to 2.0.

Could you tell us how and when did you installed? So we could get an idea of what version you are using. Please update the manager to the last version if you can.

Best regards.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/69def3f2-462d-4fb9-8773-d74e92564eb0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Victor M. Fernandez-Castro

IT Security Engineer

Wazuh Inc.

[Santiago Bassett]

Hi Marc,

try starting the process manually by running:

/var/ossec/bin/ossec-remoted -t          (to test the configuration)

/var/ossec/bin/ossec-remoted -f -d      (foreground and debug mode)

That should give us some more info to figure out what is going on.

Best regards,

Santiago.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CAB1uw7_MOH43zCFnnWpDcMdpEoq6dqJESMSGJ51kf9Jw2qVenQ%40mail.gmail.com.

[Victor Fernandez]

Hi Marc,

both things (dangling PID file and no descriptive error at log) suggest that Remoted crashed. I see that you are using Wazuh 2.0, but that version has not been yet released and we made some changes on Remoted since we changed the version number to 2.0.

Could you tell us how and when did you installed? So we could get an idea of what version you are using. Please update the manager to the last version if you can.

Best regards.

On Mon, Mar 27, 2017 at 6:29 PM, Marc Baker <marcjb...@gmail.com> wrote:

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/69def3f2-462d-4fb9-8773-d74e92564eb0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Marc Baker]

Santiago,

Thank you for your email. The first command returned nothing and the second:

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: ERROR: Unable to append merged file: '/etc/shared/merged.mg'.

2017/03/28 07:54:03 ossec-remoted: DEBUG: Running manager_init

2017/03/28 07:54:03 ossec-remoted: INFO: (unix_domain) Maximum send buffer set to: '212992'.

2017/03/28 07:54:03 ossec-remoted(4111): INFO: Maximum number of agents allowed: '8000'.

2017/03/28 07:54:03 ossec-remoted(1410): INFO: Reading authentication keys file.

2017/03/28 07:54:03 ossec-remoted: OS_StartCounter: keysize: 50

2017/03/28 07:54:03 ossec-remoted: Unable to open agent file. errno: 13

2017/03/28 07:54:03 ossec-remoted(1103): ERROR: Could not open file '/queue/rids/001' due to [(13)-(Permission denied)].

It appears that the permissions somehow were changed. Can you please advise on the best way to change the permissions to the correct level for remoted?

Thank you, 

Marc Baker

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/69def3f2-462d-4fb9-8773-d74e92564eb0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Victor M. Fernandez-Castro

IT Security Engineer

Wazuh Inc.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.

[Marc Baker]

Victor 

This is a clean installation on a new server that was built one week ago.I'll post the notes after completing the process of reinstalling the manager.

V/r 

Marc

On Tue, Mar 28, 2017 at 11:55 AM, Victor Fernandez <vic...@wazuh.com> wrote:

Hi Marc,

I can't figure out why the permissions have been changed. These are the default permissions:

drwxr-x--- root  ossec  /var/ossec

drwxr-x--- root   ossec /var/ossec/queue

drwxrwx--- ossecr ossec /var/ossec/queue/rids

-rw-r--r-- ossecr ossec /var/ossec/queue/rids/*

Did you make a clean installation? Please note that upgrading is not supported when installing from packages, so if you installed the package to upgrade an older version you should make a clean installation. I recommend you to save files "etc/client.keys" (agent information and encryption passwords) and "queue/rids/sender_counter" (numeration for packets delivered to agents):

cp /var/ossec/etc/client.keys /var/ossec/queue/rids/sender_counter ~

Then uninstall the package wazuh-manager and clean directories and repositories:

apt-get purge wazuh-manager

apt-get clean all

rm -rf /var/ossec

rm -f /etc/init.d/wazuh-manager

rm -f /etc/rc*.d/*wazuh-manager

Now install wazuh-manager again:

apt-get update

apt-get install wazuh-manager

Restore the saved files and ensure that they have the correct permissions:

mv ~/client.keys /var/ossec/etc

chown root:ossec /var/ossec/etc/client.keys

chmod 640 /var/ossec/etc/client.keys

mv ~/sender_counter /var/ossec/queue/rids

chown ossecr:ossec /var/ossec/queue/rids/sender_counter

chmod 644 /var/ossec/queue/rids/sender_counter

And restart Wazuh and Logstash (if you have it installed):

/var/ossec/bin/ossec-control restart

systemctl restart logstash.service

I hope that following these steps solve your problem. Please try it and write back to us with your results.

Thank you for your feedback.

Kind regards,

Victor.

[Marc Baker]

Victor,

Reinstalling the Wazuh manager seems to have fixed the issue. Agents are starting to show up in the Kibana interface. I will archive your instructions so that if this type of issue occurs again they will be available for reference. We appreciate the assistance and patience from the Wazuh Team as we learn the system and get things running smoothly.

V/r

Marc

On Tue, Mar 28, 2017 at 11:55 AM, Victor Fernandez <vic...@wazuh.com> wrote:

Hi Marc,

[Victor Fernandez]

Thank you Marc,

the pleasure is ours, your feedback is very valuable to us. Likewise if your problem happens again please let us know and we will investigate the cause.

Kind regards,

Victor.

[Victor Fernandez]