Enrich events with external data

[Marco S]

Hello,

I'm evaluating all the possible ways to enrich data before it's indexed. Our scenario consists in enriching firewall and other events with external data (could be MISP data, but we're not closed to other possibilities). So far what I've seen:

- Since Wazuh uses Opensearch, it seems there's no "enrich" processor for ingest nodes as it's an Elasticsearch proprietary feature;

- Logstash seems not to be part of Opensearch >2.2 anymore; The solution I'm looking for should work for Wazuh 4.4 too;

- Wazuh itself doesn't seem to have a feature to enrich events in preindexing that would scale for high EPS use cases (in our case potentially thousands of EPS);

Is Wazuh 4.4 bringing any useful feature in that regard? Can you please advise?

Thank you!

Best regards,

Marco

[Jesus Linares]

Hello Marco,

One option to consider for enriching data before it's indexed in Wazuh is to use Logstash. You can configure Wazuh to send its data to Logstash, where you can use various input, filter, and output plugins to enrich the data. If you're using Wazuh 4.3.x, you can use Logstash OSS with the elasticsearch output plugin, and it will work with Opensearch. For 4.4.x, you will need to use Logstash OSS 7.13.4 with the Opensearch output plugin.

In addition to Logstash, another option to consider is using Filebeat processors. Filebeat processors allow you to filter and enhance data before it's sent to the indexer. This can be a good option depending on your specific use case. Check out the official guide: https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.htm.

Finally, it's important to keep in mind that the Wazuh ruleset is a way of enriching data. You can create rules with different groups, such as PCI DSS, MITRE, etc. You can also match events with CDB lists, which can be created using OSINT data. Here is an example: https://wazuh.com/blog/using-osint-to-create-cdb-lists/.

I hope this information helps you in your evaluation of data enrichment options for Wazuh. Let me know if you have any further questions.

[Marco S]

Hello Jesus,

 

thank you for your detailed answer.

Also thank you for clearing up the usage and role of Logstash in this context.

As for the other proposals I see a few caveats:

- Filebeat doesn’t seem to have a processor that matches the use case (looking up for matches in some kind of external table/resource/list) to enrich data before sending it to indexing, it seems to have basic string manipulation use cases.

- As far as I have understood, CDB lists can be used to create specific rules that fire up once there’s a match without doing an enrichment (no further fields are created in the matched events). Without even considering the difficulties in maintaining CDB lists up-to-date with relevant data, I’m not sure that the Wazuh component that uses them would be able to keep up with high EPS. Can you please either confirm or dismiss this?

- In case we used Logstash, enrichment would be possible but the enriched data wouldn’t be available to Wazuh rules for alerting, correct? So a combination of Logstash enrichment and CDB lists (always considering the previous point) would be needed to achieve both.

 

Please let me know if you need further details from my side, I’ll gladly provide them.

 

Thanks for your time, have a nice day.

 

Best regards,

Marco

[Jesus Linares]

Hello,

> Wazuh component that uses them would be able to keep up with high EPS. Can you please either confirm or dismiss this?

Regarding your first question, Wazuh customers with high EPS have successfully used CDB lists without issues.

> In case we used Logstash, enrichment would be possible but the enriched data wouldn’t be available to Wazuh rules for alerting, correct? So a combination of Logstash enrichment and CDB lists (always considering the previous point) would be needed to achieve both.

Yes, it is correct. If you use Logstash for enrichment, the enriched data won't be available to the Wazuh ruleset engine for alerting (since it was already generated and sent to Logstash).

In the end, what we use for enrichment is CDB list and Filebeat/Logstash, with their limitations.

A possible workaround is to enrich the data before sending it to Wazuh. I'm not sure if you can do that in your deployment. It sounds doable if you are forwarding logs from network devices or similar via syslog. You can have this configuration:

• [Source] -> [Rsyslog -> Enrichment -> file <- Wazuh agent] -> [Wazuh manager]

or you could use the same server:

• [Source] - [Rsyslog -> Enrichment -> File <- Wazuh manager]

Also, rsyslog allows you to use "rsyslog templates" and format the message.

That said, we are working on a new engine with more features and I will forward this issue to the dev team. Thank you for your feedback.

Let me know if you need more help.