Integration of threat intelligence information in Wazuh.

[InfoSec]

Use Case:

- All security-related logs are currently collected through Wazuh (network firewall, DNS lookups, Web proxies, O/S audit logs, database logs, and application logs).

- Need to automate the process of creating and maintaining up-to-date CDB lists, built from reliable sources of threat intelligence, and create alert rules in Wazuh.

- Can anyone provide practical guidance to automate the generation and periodic maintenance of reputation CDB lookup lists from threat intelligence sources?

[Cristóbal López]

Hi Infosec,

The automation of a CDB list can be performed with active responses. I propose the next use case:

We want to create an alert that to trigger when an IP address, which at some point has attempted a brute force attack against us, is trying to connect again. The rule can be:

----------------------------
<rule id="4709" level="15">
    <if_sid>5700</if_sid>
  <list field="srcip" lookup="match_key">etc/lists/IP-blacklist</list>
  <description>An IP included in the black list is trying to connect with the host.</description>
  <group>authentication_failed,pci_dss_10.2.4,pci_dss_10.2.5,</group>
</rule>
----------------------------

You must place the next block in ossec.conf to enable the blacklist.

----------------------------
<ruleset>
    <list>etc/lists/IP-blacklist</list>
</ruleset>
----------------------------

We can automate the blacklist configuration of attacking IPs with the following script:

---------------------------- block-IP.sh
#!/bin/bash

IP=$3
BL_DIR="/var/ossec/etc/lists/IP-blacklist"

echo "$IP:Brute force attack" >> $BL_DIR
----------------------------

This script would be launched with an active response to the brute force rule, indicating the following in ossec.conf:

----------------------------
<command>
    <name>block-IP</name>
    <executable>block-IP.sh</executable>
    <timeout_allowed>no</timeout_allowed>
    <expect>srcip</expect>
</command>

<active-response>
    <command>block-IP</command>
    <location>server</location>
    <rules_id>5712</rules_id>
</active-response>
----------------------------

Finally, we must schedule a CDB list compilation and a restart of the manager to apply the configuration. We can do it with wodle command:

----------------------------
<wodle name="command">
  <disabled>no</disabled>
  <tag>test</tag>
  <command>/bin/bash /root/update_CDB.sh</command>
  <interval>1d</interval>
  <ignore_output>yes</ignore_output>
  <run_on_start>yes</run_on_start>
</wodle>
----------------------------

Where update_CDB.sh can be:

----------------------------
#!/bin/bash

/var/ossec/bin/ossec-makelists
/var/ossec/bin/ossec-control restart
----------------------------

Can this use case be applied to your need? If not, can you specify what you need?

Best regards,
Cristobal Lopez.

[yol...@saitechnology.com]

Hi Cristobal,

This use case is very good.

I will ask now: what about feed the CDB.list with the latest Virus
and/or malware hashes?

I know that wazuh 3.0 have the integration with VirusScan.

But some idea for previous Wazuh versions, to feed and keep updated the
CDB.list regarding to hash or md5 for virus?


Thanks and Regards
Yolanda

> --
> You received this message because you are subscribed to the Google
> Groups "Wazuh mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wazuh+un...@googlegroups.com.
> To post to this group, send email to wa...@googlegroups.com.
> Visit this group at https://groups.google.com/group/wazuh.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/wazuh/05b8c60e-e147-4d8f-a358-715ff044cdec%40googlegroups.com
> [1].
> For more options, visit https://groups.google.com/d/optout.
>
>
> Links:
> ------
> [1]
> https://groups.google.com/d/msgid/wazuh/05b8c60e-e147-4d8f-a358-715ff044cdec%40googlegroups.com?utm_medium=email&utm_source=footer

[Borja Arroba]

Hi Yolanda,

I have been investigating what you are proposing and I am sorry to tell you that it is not possible. The best solution for your case would be to update to the new available versions and configure the integration with VirusTotal.

Let me explain how easy would be integration with vti in new agents and manager:

1º Sing up in virustotal web page.

At www.virustotal.com, register and get public API-key in settings section.

2º Go to the manager and modify ossec.conf:

- Add in ossec.conf integration tag with your public API-Key.

<integration>

  <name>virustotal</name>

  <api_key>API-KEY-FROM-VTI-AT-www.virustotal.com</api_key>

  <group>syscheck</group>

  <alert_format>json</alert_format>

</integration>

- Enable integrator daemon and reload with this commands:

/var/ossec/bin/ossec-control enable integrator 

/var/ossec/bin/ossec-control reload

Note: you should be keep in mind that VirusTotal's functionalities of public API-Key has limitations such as: 

The request ratio to no more than four requests of per minute, and

low priority access of requests done for the VirusTotal engine.

3º In agent:

- Monitor any folder with FIM adding in the <syscheck> tag on ossec.conf in any agent:

<directories check_all="yes" realtime="yes" report_changes="yes">/path-folder</directories>

Note: Remember you can't add file with realtime option, folder required. You can schedule "realtime" in specific files with "restrict" option.

- Use this command to reload daemons:

/var/ossec/bin/ossec-control reload

And now it's all on ready to run.

4º Checking alerts:

When check a virus file: 

Rule: 87105 (level 12) -> 'VirusTotal: Alert - /home/test/suspiciusfile - 17 engines detected this file'

{"virustotal": {"permalink": "https://www.virustotal.com/file/51732cd023793f45f1ca7737b7844e339b73523a45dc4d13e9c3dfa5dde9be7a/analysis/1512387296/", "sha1": "91e22909dba884c2e0b71f76ff825df90ea32d1a", "malicious": 1, "source": {"alert_id": "1519554946.71382", "sha1": "91e22909dba884c2e0b71f76ff825df90ea32d1a", "file": "test/suspiciusfile", "md5": "d74d8b51a4291c3bd80058aa540e4368"}, "positives": 17, "found": 1, "total": 59, "scan_date": "2017-12-04 11:34:56"}, "integration": "virustotal"}

No alerts: 

** Alert 1519555008.74088: - virustotal,

2018 Feb 25 11:36:48 (agentubuser) any->virustotal

Rule: 87103 (level 3) -> 'VirusTotal: Alert - No records in VirusTotal database'

{"virustotal": {"found": 0, "malicious": 0, "source": {"alert_id": "1519555007.73597", "sha1": "e67490a19c868c602d099fbecf61e71ccf2235dc", "file": "/home/test/normalfile", "md5": "386633098d8dfaaa7c315de168d47256"}}, "integration": "virustotal"}

Note: While it's monitoring, you can see another alerts, if you has wrong key in configuration or you pass the limit of request, in alerts you can see both messages:

Rule: 87102 (level 3) -> 'VirusTotal: Error: Check credentials'

Rule: 87101 (level 3) -> 'VirusTotal: Error: Public API request rate limit reached'

Best regards.

Borja Arroba.

> an email to wazuh+unsubscribe@googlegroups.com.

[yol...@saitechnology.com]

Hi Cristobal,

Do you have a chance to looking into this email?


What about feed the CDB.list with the latest Virus and/or malware

[Cristóbal López]

Hi Yolanda,

As Borja says, integration with Virustotal is not compatible with Wazuh < 3.0. Therefore, and for many other advantages, I recommend that you upgrade to the latest release.

That integration is an example of threat intelligence, where we use Virustotal as a feed. However, you can build the CDB list based on other sources of threat intelligence, which fits your needs, such as explained in this article.

To maintain the CDB lists you can use scripts that periodically update the content of the feed (see wodle command), or use active responses as in the use case I proposed above. These scripts could ask ElasticSearch for information about the data you want to keep on the CBD list.

Best regards,
Cristobal Lopez.

>> an email to wazuh+unsubscribe@googlegroups.com.

[InfoSec]

Hi Cristobal,

I have a slightly different use case.

I would like to check the reputation of IP addresses whenever non-RFC 1918 IP information is present in an event (decoded using the classic decoder -- e.g. contains fields srcip &/or dstip, or decoded using dynamic decoders -- contains fields src_ip &/or dst_ip), separately for source and destination IP.

The rules would look something like:

<!-- AlienVault IP Reputation Integration -->
<group name="ipreputation,">
  <rule id="100100" level="10">
    <list field="srcip" lookup="address_match_key">etc/lists/blacklist-alienvault</list>
    <description>Source IP in black list.</description>
  </rule>

  <rule id="100101" level="10">
    <list field="src_ip" lookup="address_match_key">etc/lists/blacklist-alienvault</list>
    <description>Source IP in black list.</description>
  </rule>

  <rule id="100102" level="10">
    <list field="dstip" lookup="address_match_key">etc/lists/blacklist-alienvault</list>
    <description>Dest IP in black list.</description>
  </rule>

  <rule id="100103" level="10">
    <list field="dst_ip" lookup="address_match_key">etc/lists/blacklist-alienvault</list>
    <description>Dest IP in black list.</description>
  </rule>
</group>

How would the above rules behave in case an event does not contain one or more of: srcip, dstip, src_ip, or dst_ip -- which will always be the case in one or more of the rules above?

If it works (nothing says that it will), what about the performance impact if Wazuh collects 'permit' events from busy Internet-facing firewalls, and decodes srcip and dstip from these events?

Can you suggest a more efficient way of achieving the objective?

[Cristóbal López]

Hi Infosec,

The CDB lists will not be checked unless a decoder has extracted the target field. However, srcip is too common for be checked for all the events of your systems.

I recommend you to create a parent alert that filters the events you want to check with these lists.

Best regards,
Cristobal Lopez.

[InfoSec]

How would we go about creating parent lists without causing an exponential growth in the number of rules, as IP addresses may be present (and decoded) in a large number of types of events.

[Cristóbal López]

Hi Infosec,

The rule where you use the CDB list would be daughter of a filter rule which prevents srcip from being compared in all logs. For this, use <if_sid> tag. For identify the parent of a rule, use the if_sid tag.

Best regards,
Cristobal Lopez.

[Cristóbal López]

I would like to clarify something about the case of the use of my first message. I used the following code block:

<wodle name="command">
  <disabled>no</disabled>
  <tag>test</tag>
  <command>/bin/bash /root/update_CDB.sh</command>
  <interval>1d</interval>
  <ignore_output>yes</ignore_output>
  <run_on_start>yes</run_on_start>
</wodle>

There does not make sense to set the run_on_start tag to yes as that would invoke the script that restarts Wazuh right at startup.

#!/bin/bash

/var/ossec/bin/ossec-makelists
/var/ossec/bin/ossec-control restart

If you use a system with systemd, it is recommended to call systemctl wazuh-manager restart instead of the ossec-control script.

Best regards,
Cristobal Lopez.

[Dimitry Baranov]

Hi Cristobal,

Please advise whether the ipset 2 cdb list conversion script (python), referenced from the article you mentioned, could still be found somewhere?

Since the article publication the linked iplist-to-cdblist.py became unavailable unfortunately.

Kind regards,

Dimitry A. Baranov

четверг, 1 марта 2018 г., 17:18:39 UTC+3 пользователь Cristóbal López написал:

[Eva Lopez]

Hi Dimitry

I have reported this problem, we will solve as soon as possible. The script you search is the following.

Kind regards, Eva

#!/usr/bin/env python

#
# Convert IP list to CDB list
# Copyright (C) 2016 Wazuh Inc.
#
# This program is a free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public
# License (version 2) as published by the FSF - Free Software
# Foundation.
#

import re
from sys import exit, argv

try:
    if len(argv) != 3:
        print("Bad arguments. Try: iplist-to-cdblist.py input output")
        exit(1)

    ip_regex = re.compile("^((?:[0-9]{1,3}\.){3}[0-9]{1,3})(?:/(\d{1,2}){0,1}|)")
    first_time = True
    cdir_conversion = {"32": 4, "24": 3, "16": 2, "8": 1}

    fo = open(argv[2], 'w')

    with open(argv[1]) as f:
        for line in f:
            match = ip_regex.match(line.rstrip('\r\n'))

            if not match:  # Read just lines that start with an IP
                continue

            ip = match.group(1)
            mask = match.group(2)

            if mask:  # Convert allowed masks (32, 24, 16, 8)
                ip = ip.split('.')
                if mask in cdir_conversion:
                    ip = '.'.join(ip[:cdir_conversion[mask]])
                    if mask != "32":
                        ip += "."

                else:
                    continue

            ip += ":"  # CDB List format

            if first_time:
                fo.write(ip)
                first_time = False
            else:
                fo.write("\n" + ip)

    fo.close()

    print("[{0}] -> [{1}]".format(argv[1], argv[2]))

except Exception as e:
    print("Error:\n{0}\nExiting...".format(e))
exit(1)

[Supragya Karki]

An additional query here:

How can I configure this rule to check for all the black-listed IPs in every logs parsed to wazuh manager? 

  <rule id="200011" level="15">      

    <if_sid>5700</if_sid>

    <list field="srcip" lookup="address_match_key">etc/lists/blacklist-alienvault</list>

    <description>High Level Alert! An IP included in the black list is trying to connect with the host!</description>   

  </rule>

The above given rule gives me desired alert only when I try to connect via ssh but doesn't check for other groups or logs except ssh logs. When I tried by removing the <if_sid> part, it didn't give me the alert I wanted even when blacklisted IP tried to connect via ssh. Rather I get normal alert of authentication success and failure.

Help me with this info.

>> an email to wazuh+un...@googlegroups.com.

​

[Supragya Karki]

Waiting for the response Sir!

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/1d76afbf-1df6-40a2-8c86-c5eefdeca4can%40googlegroups.com.

[piyus...@gmail.com]

Hi,

not an expert here but you might try other sid across which you want to check blocked IPs as 5700 will only work for ssh 

[Supragya Karki]

Thank you but I actually want to check against the list of blocked IPs for each and every logs that is parsed into wazuh. Every logs, not just for one.

[XYZ Company]

Hello Wazuh Team,

I am eager to know the reply for the query "Supragya Karki" has addressed, as I am also facing the same thing. I want to create a rule to check the black-listed IPs from CDB list in every logs (might be src IP or dst IP or incoming or outgoing IPs etc ) which are parsed to wazuh manager from any log sources.