Alert for Silent Log Sources

[XYZ Company]

Hello Team,

I am very impressed with the capabilities of Wazuh so far and am using Wazuh over enterprise level SIEM. Cheers!

Since, we have more than 10 log sources (agents in different linux OS, Win OS, firewall and router logs via syslog and so on) forwarded to Wazuh manager, how can I find out if one of the log source is not forwarding its logs? or how can I be notified if one of the source stopped sending logs due to some config errors? 

Now my question is:

- Do you have any rule to trigger when any log sources becomes silent all of sudden?

OR

- Do you have any scripts capability to check every 10 mins and alert if any log source has stopped sending logs to the manager?

Looking forward to quick response. Thank You.

[Jonathan Martín Valera]

Hello,

The quick answer is that if the device is sending the information through a wazuh-agent (installed), then you can check its status (it should be active) in the output of this command /var/ossec/bin/agent_control -l. For example:

# /var/ossec/bin/agent_control -l

Wazuh agent_control. List of available agents:
   ID: 000, Name: manager (server), IP: 127.0.0.1, Active/Local
   ID: 001, Name: agent3, IP: any, Active

List of agentless devices:

Here we can find information about its status, but we can’t be sure that it is sending logs/events of the desired module… (see https://documentation.wazuh.com/current/user-manual/agents/agent-life-cycle.html), plus not all devices would be registered in this list, since you will have some that are sending via remote syslog.

So what can you do?

As you said, you could make a script that could check if the agents/devices are reporting to the wazuh-manager. There is no script already made for this, but I am going to tell you how you could do it. To do this, first you have to be clear about what you want to do.

• Are all agents/devices continuously reporting logs/events to the manager? First, you have to set the time frame for which you want to generate that alert, that is, the amount of time in which the agent/device has not reported anything to the wazuh-manager.

• Next, we have to decide if what we want to check are the events or alerts generated. One thing is that the agent is reporting logs/events to the wazuh-manager, and another thing is that they generate alert (relation: one event can 0..1 alert). In your case, I imagine that you want to check that the agent/device is reporting only to the wazuh-manager. This is important since it is going to decide in which file we are going to look for the information.

Once we have this information, the next thing is to activate the information source where we are going to look for the events of all the agents/devices. All the event logging received by the manager can be stored in two files: archives.log and archives.json. By default, this storage is disabled because it can cause disk usage and intensive storage in case the wazuh-manager receives a lot of events.

We are going to enable the logging of events in json format, to do so edit the following in the /var/ossec/etc/ossec.conf file

<logall_json>no</logall_json>

to

<logall_json>yes</logall_json>

and then, restart the wazuh-manager

systemctl restart wazuh-manager

From now on, every event received by the manager will be logged in the /var/ossec/logs/archives/archives.json file. For example, you may find events with the following information:

{"timestamp":"2021-10-26T07:58:20.226+0000","agent":{"id":"000","name":"manager"},"manager":{"name":"manager"},"id":"1635235100.0","full_log":"Starting rootcheck scan.","decoder":{"name":"rootcheck"},"data":{"title":"Starting rootcheck scan."},"location":"rootcheck"}

What you can do is to create a script, which will iterate through a list with the identifier of your devices/agents (for example agent id if it is registered, or by the IP directly that can be used in the name), read the information from this file and make the difference between the timestamp of the current time and the last event received from the agent/device. In any case, you can generate a custom output that will then be processed in the wazuh-manager.

Up to this point, we have a script that is able to know if any agent has not reported, but this process is not automatic. We can make the wazuh-manager run every x time the script, process its output and generate an alert if desired, or directly the script logs its output in a file that monitors the wazuh-manager using the logcollector module (https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/localfile.html).

Here is an example of how to configure the execution of commands periodically https://documentation.wazuh.com/current/user-manual/capabilities/command-monitoring/command-configuration.html#check-if-the-output-changed.

The execution of such a command will generate an event in the wazuh-manager. You can create custom decoders and rules to generate alerts from that event https://documentation.wazuh.com/current/user-manual/ruleset/custom.html.

To summarize, the general idea is as follows:

• Enable event logging in the wazuh-manager.
• Create a script that allows to identify if an agent/device has not reported in a time interval (difference of timestamps with the last logged event of that agent/device).
• Configure the wazuh-manager to run this script periodically, process its output and generate an alert if necessary (you will probably have to create custom rules for this). You also have the possibility that if the script instead of displaying the information on the screen logs it to a file, you can configure the wazuh-manager to monitor that file, although in this case nobody will run the script automatically unless you schedule a cronjob.

I hope this information is helpful, maybe it is a little complex to understand right now because there is too much information. If you have any doubts on any point, please let us know :)

Best regards.

​