How to manage Logstash data filters

[Cyprien Chapelle]

Hello,

I installed Wazuh with this link: https://packages.wazuh.com/4.3/wazuh-install.sh

This works fine, however I have a few questions as to how it works.

I would like to be able to add filters on the data before it is sent to Opensearch. It seems to me that it is done on Logstash, but with this installation I am a little lost. For example, I would like to add a tag, for events corresponding to false positives. I would also like to change the name of some fields.

Finally, how do you configure a second index? Especially when I have data coming from other tools, I would like to add an index specific to this tool.

[Federico Gustavo Galland]

Hi there Cyprien,

You can customize the filebeat template available at:

/usr/share/filebeat/module/wazuh/archives/ingest/pipeline.json

to customize how data is stored to the Wazuh Indexer.

However, it would make more sense (to me anyways) to filter the data on the decoding/ruleset matching stages. You can also change the name of most fields over there, and you don't risk breaking the whole indexing system.

A good starting point for writing your own decoders/rules is here:

https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/

You can write to us anytime with questions, though.

For creating new indices, you can follow OpenSearch's documentation on the subject:

https://opensearch.org/docs/latest/opensearch/index-templates/

I hope this helps.

Regards,

Fede

[Cyprien Chapelle]

Hello Frederico,

Thank you for your help.

I see how to do it with Filebeat, but I thought you could use Logstash's filtering which allows you to be more precise.

Have a good day.

[Federico Gustavo Galland]

You are welcome Cyprien.

I'm wondering what is the specific use-case you are after, since the default way of adding fields to alerts is through the decoding stage.

Another question I'm asking myself here is how you would realize they are false positives, and if you found a way to do that, why would you want those false positives at all?

Maybe a little context will come in handy.

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/PNGiZaSOMsc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/06fb67a8-676a-4b8f-be05-8cf55fe1aa1fn%40googlegroups.com.

--

Federico Galland Pre-sales Engineer +1 (844) 349 2984 wazuh.com federico...@wazuh.com f-galland

[Cyprien Chapelle]

Ok, I will explain all this to you more properly.

First, a bit of context:

I work on a Proxmox server where I installed a Firewall PfSense router with three interfaces (LAN, DMZ, WAN) with different "user" VMs and as well as servers (web and bdd).

So I installed Wazuh and Suricata to monitor my machines and my network. Not having much time for this project, I preferred to install Wazuh by the script provided, although in other projects I had the opportunity to install everything manually (with ELK). That's why I'm a little lost, I can't find the configuration files I used to. Indeed, in previous projects, I had used Logstash to code in Ruby various filters such as deleting fields, modifying them, filtering logs according to files (with REGEX), adding tags or information to certain specific alerts, etc.

For example, on Logstash, I had created a regex that filtered if the alerts came from Suricata or from Wazuh and thus returned the output to two different indexes.

In this project, I don't really have any fields to modify or other bugs to fix on Logstash for the moment, but I still wanted to know how to do it if necessary. As far as false positives go, that's a pretty complicated question for me. Indeed, currently, my false positives are alerts of the type "Suricata: Alert -GPL ICMP_INFO PING *NIX" corresponding to a known IP address. There are also alerts of the "Host-based abnormaly detection event" type which is redundant. Of course, I can't "remove" these false positives by filtering the rule (somehow) because I can't risk missing an alert that might turn out to be true. My idea is therefore to create a tag that would be added automatically to each selected alert, in order to identify potential false positives but while being able to view and analyze them, if necessary. Nothing goes by the wayside.

For example, add a "PFP" tag for "Potential False Positive" on alerts of the type "Suricata: Alert -GPL ICMP_INFO PING *NIX" for defined IPs. However, this recursive tag addition, cannot be done via Filebeat? I know very little about Filebeat unlike Logstash.

With Logstash, I would have made a condition on the corresponding field, to verify that it is equal to "Suricata: Alert -GPL ICMP_INFO PING *NIX" and a second condition that would look in a white list of IPs (defined in prior) if the source IP of the alert is registered or not.

Do you see the idea?

Thanks for reading.

Cyprien.

[Federico Gustavo Galland]

I do see what you mean regarding your use-case.

If I gather your requirements correctly, most of them can be achieved through the use of the Wazuh ruleset itself.

For example, the rule associated with the alert you bring up has the ID 86601.

We can create a child to that alert that makes it fire whenever the "alert.signature" field matches your example: -GPL ICMP_INFO PING *NIX

To do so, you would create a new xml file under /var/ossec/etc/rules/

With the following content:

<group name="ids,suricata,custom,possible_false_positive,">

    <rule id="100500" level="3">
        <if_sid>86601</if_sid>
        <field name=" alert.signature  ">GPL ICMP_INFO PING</field>

        <description>Suricata: Alert - $(alert.signature)</description>

        <info type="text">Possible False Possitive</info>
        <options>no_full_log</options>
    </rule>

</group>

Which is making this new rule, ID: 100500, a child of the previous one that only fires under your desired conditions. You could modify the alert description as well, but I opted to add an "info" tag adding some text which could help you filter these out from the Wazuh Dashboard.

This is actually our preferred supported way of doing pretty much all the things you mention with regards to manipulating alert data, save for saving to a separate index.

I hope this helps!

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/acc8b811-42b3-4b65-8d2a-2be8a760eb50n%40googlegroups.com.

[Cyprien Chapelle]

Ok, thank you very much, I understand the idea. By how can I specify a particular IP (or multiple IP)?

Can I use the srcip domain for this?

Finally, regarding wazuh-indexer/wazuh-dashboard/filebeat etc.

If I understood correctly, wazuh-manager creates the alerts, these alerts are sent to Filebeat, which sends it a json to wazuh-indexer (OpenSearch?) and then sends the data to wazuh-dashboard (OpenSearch Dashboard), is that?

[Cyprien Chapelle]

It doesn't work, but I think I know why. The corresponding alert is an alert coming from Suricata, Wazuh directly retrieves the eve.json, where the Suricata alerts are located. So if I have to add a rule, it's directly on Suricata right?

So I have a problem on this point (because it is difficult to configure rules like above on Suricata), it would really be necessary that the filtering that I want (for adding tags) is in post-processing, Filebeat is not able to do this you think?

If not, is it possible to configure a Logstash between Filebeat and wazuh-indexer?

[Alonso Cárdenas Márquez]

Hi Cyprien

Take a look at the following filebeat and logstash config file (https://people.freebsd.org/~acm/ports/wazuh/). It could give you a good idea about logstash config.

I am testing wazuh on FreeBSD and I have had to adapt some configuration for it works on FreeBSD. Just now I'm using a similar configuration like you need

wazuh-manager -> filebeat -> logstash -> wazuh-indexer (opensearch) 

Btw, you will need install logstash >= 8.x and logstash-output-opensearch plugin into logstash

Greetings

ACM

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/94951c46-8e3f-42ff-aa7a-7f03ff6406d2n%40googlegroups.com.

[Cyprien Chapelle]

Thank you very much, I will take a look at this.

From Logstash, you create an index named wazuh-alerts-4- ...

But where do we access the default indexes of wazuh-indexer? In fact, if I add Logstash, I will create my own index on Opensearch like you, so I would like to know where the default indexes are configured to modify or delete them.

[Alonso Cárdenas Márquez]

Hi Cyprien

I'm not an expert because I'm just beginning with wazuh but if I'm not mistaken index is created at wazuh-indexer (opensearch) data directory (by default /var/lib/wazuh-indexer). You must look into wazuh-indexer yml config file. It should be at /etc/wazuh-indexer/opensearch.yml in any GNU/Linux distro. Don't forget read careful Federico's emails because he gave you info about how indexes could be defined/modified. Somethings can be done using Wazuh API (For example remove indexes)

Greetings

ACM

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/859be1b1-3f3e-4dd0-86c2-5dd612bee3b1n%40googlegroups.com.