Receive logs from the remote devices as a sylog into kibana dashboard

[triv...@pidatacenters.com]

Hi,

I have configured wazuh server 3.2.2 on centos7 and installed agents on few machines receiving logs on the kibana dashboard from the agents.

And i want this wazuh to be a setup as centralized log capturing server hence allowed this IP in network devices (cisco firewall, switches ) and ESXi hosts. Also in the server end modified required changes like allowed ips in the remote section on /var/ossec/etc/ossec.conf. But unable to receive the syslog messages from the remote machines.

Please provide troubleshooting steps to get the  syslog messages. 

Any help would be appreciated.

[Syed]

I have similar problem. I do NOT see any events in the OSSEC Dashboard or Wazuh panel or discover. Here is what I've done for troubleshooting

My setup: Wazuh VM - All in one

1. tcpdump to check syslog traffic on port 514 = Yes

2. verified the /var/ossec/etc/ossec.conf file to ensure that firewall and routers IPs are in the IP allowed section, port is 514, protocol is tcp and udp. Here is what I used to setup the .conf file. 

https://www.rootusers.com/monitoring-devices-by-sending-syslog-to-ossec/

https://documentation.wazuh.com/current/user-manual/manager/remote-service.html

3. Netstat to ensure 514 is listening

4. Check /var/ossec/logs/ossec.log. It shows following errors

2018/03/28 23:32:40 ossec-remoted: WARNING: Accepting tcp connection from client failed.

2018/03/28 23:32:41 ossec-remoted: WARNING: Accepting tcp connection from client failed.

2018/03/28 23:32:42 ossec-remoted: WARNING: Accepting tcp connection from client failed.

2018/03/28 23:32:43 ossec-remoted: WARNING: Accepting tcp connection from client failed.

Need some expert help here...

[Santiago Bassett]

Hi,

@syed, it looks like you need to allow connections from the source IPs using allowed-ips option. 

@trivedi, for troubleshooting, as per Syed recommendation, you can use 'tcpdump' to monitor the traffic or 'netstat' to ensure the service is actually listening for incoming data. 

You can also add '<logall>yes</logall>' to your '/var/ossec/etc/ossec.conf' file on the Wazuh server, and then restart it. You should be able to see Syslog events in '/var/ossec/logs/archives/archives.log' (they do not even need to generate an alert, by matching decoders and rules, to be there)

In addition, I would advice to use Rsyslog to collect Syslog data and then configure the logcollector to read Rsyslog output file. This approach gives you the ability to filter data (using Rsyslog filters), normalize it (using templates) and the ability to use Syslog over TLS. I usually like it better.  You can also have ossec-remoted listening in a local socket only (loopback), and then have Rsyslog forwarding to that socket. This way you also save disk iops by not having to write Rsyslog output to a file. 

Best regards

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/9e5de8c8-0317-4277-812a-fa5e6fe203c5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Syed]

Thank you Santiago. I have configured the allowed-ips option as per https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/remote.html#allowed-ips

Here is what my remote config looks like.

  <remote>

        <connection>syslog</connection>

        <port>514</port>

        <protocol>udp</protocol>

        <protocol>tcp</protocol>

        <allowed-ips>X.X.X.X</allowed-ips>

        <local_ip>X.X.X.X</local_ip>

 </remote>

 <remote>

    <connection>secure</connection>

    <port>1514</port>

    <protocol>udp</protocol>

  </remote>

I have also enabled the debug, but not seeing much in ossec.log file. What can I check next ?

[jesus.g...@wazuh.com]

Hi Syed and @trivedi.n,

I've been trying with a lab environment in order to show a little test of rsyslog.

I have tested it with a Wazuh manager on CentOS 7 and a Wazuh agent on Debian 8.

Wazuh manager

Edit /etc/rsyslog.conf on the Wazuh manager machine:

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal

# --> Starts the TCP server <---
$ModLoad imtcp
$InputTCPServerRun 514

$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging on
$IMJournalStateFile imjournal.state

# --> Test file to throw every log on it <--
*.* /var/log/oneGiantHeapOfLogs.log

Once done, save and exit. Now we are going to start rsyslog on the Wazuh manager machine:

# set rsyslog enable
# systemctl restart rsyslog

Check if rsyslog server is running on TCP/514:

# netstat -tulpn | grep rsyslog
tcp        0      0 0.0.0.0:514             0.0.0.0:*               LISTEN      8751/rsyslogd      
tcp6       0      0 :::514                  :::*                    LISTEN      8751/rsyslogd

Wazu agent

Now, it's time to change to the Wazuh agent machine, please append this line to /etc/rsyslog.conf:

*.* @@192.168.1.143:514

Where 192.168.1.143 is the Wazuh manager IP (rsyslog server IP).

# set rsyslog enable
# systemctl restart rsyslog

Testing

Now let's see a debug message sent from client and received on the server:

On the server machine (Wazuh manager), execute the following command:

# tailf /var/log/oneGiantHeapOfLogs.log

Where oneGiantHeapOfLogs.log is the file we configured on first step.

On the client machine (Wazuh agent), execute the following command:

# logger "Hello this is a test message"

On the tailf command output, you should see:

Apr 16 05:44:43 osboxes osboxes: Hello this is a test message

Where osboxes is my hostname for the Wazuh agent, and the date is the message date.

Once you guys have this test passed with a similar result, we will continue dig into Wazuh itself, to handle these logs ok?

Hope it helps, have a nice day.

Best regards,

Jesús