Wazuh or Winlogbeat?

[boyhi...@yahoo.com]

Figured this was the best place to ask knowing there's tons of use cases with SO out there.

For our current infrastructure we're pulling server logs via Beats, but we're looking to pull client logs as well. We're wondering if we should go with Wazuh or stick with beats.

When it comes down to it, w/Wazuh it seems the biggest benefit IMO is the HIDs rules.

Beats appears to parse much nicer (I'm sure with some work I could get Wazuh there as well).

I am using both on a test server currently and the biggest difference (outside of HIDS) is that Beats parse pretty clean, and the Wazuh logs appear to mostly come in unparsed blobs. Again, I'm sure with some work I could modify the current logstash confs to fix that.

If it makes any difference we're running Sysmon everywhere with the Ion threat intel configs.

Thoughts? Pros, Cons? What's everyone else doing?

[Philip Robson]

I used a system that used ossec in the past, issue I had was rolling the agent out and keeping it rolled out.

Now I use windows event forwarding, group policy to make sure servers and workstations pump the logs to the wef  server. From that box I use winlogbeats send the relevant events along with sysmon and auto runs.

Upside is gpo keeps the machines pushing events, downside is you then have to create your own alerts using elastalert.

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

[Kevin Branch]

Hi,

The issue of Wazuh's sub-level parsing of Windows log events was resolved starting with Wazuh 3.8.0.  The Wazuh agent can now collect Windows events as JSON objects with native Windows event fields and names intact, much like Winlogbeat has been doing for some time.  Wazuh no longer uses static decoders to try to extract key fields from messy textual Windows log events.  I was very happy when they achieved this.  SO first took advantage of this when it moved from Wazuh 3.7.2 to Wazuh 3.8.2 on 2/25/19.  By default, legacy logs SYSTEM and APPLICATION are collected using the old method, but it is very easy to switch 100% of Wazuh log collection to the JSON method.  I believe Wazuh just wants to catch up their HIDS ruleset support for the new eventchannel JSON field names a bit more before changing the defaults completely over in this area.

Wazuh has taken the Wazuh agent to Wazuh manager relationship a long ways since they forked legacy OSSEC.  Countless bugs, performance issues, and feature gaps have been resolved and entirely new improvements and expansions have been added.  I used OSSEC for years before Wazuh forked it and I am thankful for the many headaches that I no longer have as a result of the extensive work Wazuh has done and continues to do.  It is a solid tool being used reliably in business environments involving a handful of agents up to huge companies with tens of thousands of agents.

Kevin Branch

Wazuh Trainer

See the "Windows events are collected in native JSON" section in https://documentation.wazuh.com/3.x/release-notes/release_3_8_0.html#wazuh-core

[Josh]

Kevin! Thanks for the information. The event in this instance that I compared was a Sysmon event.

The beats event parsed and pulled out the data properly, Wazuhs logs definitely are improved from how they use to be, but still need quite a bit of work it seems. They left a lot of information unparsed.

Although I guess talking it out in this case, it's probably partially an issue with the sysmon parsing in logstash combined with how it's presented.

[Kevin Branch]

Here is how the Sysmon-specific fields were parsed out of an example event in Kibana on my Wazuh 3.8.2 setup (standalone Wazuh setup, not Security Onion).  

Maybe SO's logstash config to handle Sysmon events needs to be updated to handle the new JSON EventChannel incoming format for Sysmon events.  Actually, Logstash should not need to parse this new format much as it is essentially pre-parsed, leaving mostly some normalization and enrichment for Logstash to do.  Can you share an example of what your poorly-parsed Sysmon event looks like in Kibana?  Did it arrive from a system running Wazuh Agent 3.8.x?

[Wes Lambert]

Most of the above is legitimate.

We are planning to have this updated -- changes are currently in the..wait for it...pipeline... :)

https://github.com/Security-Onion-Solutions/security-onion/issues/1469

But, honestly, it shouldn't break anything except for visualizations IIRC, although we do have to rename some of the fields back to some of the values we were using previously, which can occur in the existing mutate block in 6501_ossec_sysmon.conf

Thanks,

Wes

--

https://twitter.com/therealwlambert

https://securityonion.net/

[Josh]

Kevin - for example. The following screenshot is from Wazuh.

Where in beats it's

t image_path C:\Program Files (x86)\Nmap\nmap.exe

If this is something that SO is working on then maybe we'll either A. Hold out, or roll out Wazuh and work with what we have for now.

On topic of rolling out. Any clear indicator that gets logged when a host hasn't connected, failed to connect, or hasn't submitted logs in some period of time?

I know you can do a flatline in elastalert, but was hoping ossec does some kind of tracking.

[Kevin Branch]

Hi Josh,

With Sysmon 9.0 and Wazuh 3.8.2 using eventchannel json log collection, all that stuff in your image_path picture is broken down into separate fields.  I do remember it used to look like you showed there and that bugged me because I wanted the filename separated out from the other metadata of the image.   

You example looks to me like a Sysmon event collected with a pre-3.8.x version of Wazuh agent that is parsed via SO Logstash rules.

You can query Wazuh manager about what agents are connected vs disconnected without looking at Elasticsearch at all.  The Wazuh Kibana app also shows that information including agent coverage percentage over time, but that is not ready on SO yet.  Try "/var/ossec/bin/agent_control -l"

With one of my clients, I use a script-generated set of Elastalert flatline rules for all agents that they want to especially notice if they drop out of line.

Kevin

[Josh]

Kevin - since the last post I have upgraded my agent, and there are def more fields! But now data is being parsed weird.

For a reason I've yet to determine, the "n" was removed from nmap, haha. It's fine in beats.

data.EventChannel.EventData.Image C:\Program Files (x86)\Nmap\map.exe

[Kevin Branch]

You have got to be kidding :)  Someone need to turn off the "remove random character from field" function somewhere.

I've dug though lots of data.EventChannel.EventData.Image values and have yet to see missing characters.  Of course if they were missing I don't suppose I'd see them...

I suggest you find the actual JSON record corresponding to your "nmap-minus-n" event at the source in /var/ossec/logs/archives/archives.json and see if the character is missing there. 

If it is present there, then I presume something unexpected is going on in SO's archives.json pipeline from syslog-ng to Logstash, or in Logstash itself.  Please do let us know what you find.

Thanks,

Kevin

[Josh]

I appreciate the help so far. I did grep "map.exe" in that archives.json and the character is indeed missing there.

"Image":"C:\\Program Files (x86)\\Nmap\\map.exe"

[Kevin Branch]

I'll see if I can reproduce that.  In the mean time are you sure you are using Wazuh 3.8.2 on both the Windows agent side and on the SO server side.  You could run this on the SO server to confirm the Wazuh manager version in use there: "/var/ossec/bin/manage_agents -V"

On Wed, Mar 13, 2019 at 5:12 PM 'Josh' via security-onion <securit...@googlegroups.com> wrote:

I appreciate the help so far. I did grep "map.exe" in that archives.json and the character is indeed missing there.

"Image":"C:\\Program Files (x86)\\Nmap\\map.exe"

[Kevin Branch]

I was able to reproduce it and after a touch of research I found there is indeed an issue of certain Windows paths having for example "\n" removed by Wazuh due to being misinterpreted as white space instead as a part of the text of the path (like "\nmap.exe" becoming "map.exe").   The issue was found 13 days ago and 8 days ago they rolled the fix into Wazuh 3.9.0.  I would not expect to see a 3.8.3 version appear on account of this issue, so plan on waiting till 3.9.0 is released by Wazuh and incorporated by SO.  Once 3.9.0 is released by Wazuh, you could experiment with Wazuh 3.9.0 agent reporting to the Wazuh Manager 3.8.2 still on SO and it would probably work OK though the official recommendation is to always have your Wazuh manager version equal to or higher than the version(s) of your Wazuh agents.

https://github.com/wazuh/wazuh/issues/2715

Looking at the 3.9.0 project board (https://github.com/wazuh/wazuh/projects/23) it appears they have made massive progress on the upcoming next version of Wazuh but I have not heard how soon they expect to actually release 3.9.0.

Kevin

[Kevin Branch]

Wazuh was removing "\n", "\t" and "\r" as part of the process of stripping white space from Windows log messages during parsing and missed as least some cases where such substrings can be part of the field values in a non-escaped way, such as with Windows file paths.

Now that I'm looking for it, I do find the issue popping up elsewhere in my dataset, like with "c:\windows\system32\askhostw.exe" missing the t in taskhostw.

Kevin

[Josh]

Well not good, but good to know I'm not alone and there's a fix in the pipeline.

I'd really prefer to use Wazuh, but items like this would really hurt detection efforts for any kind of alerting of even malicious processes.

I guess we can hold out a bit and see what comes of it all.