Using Sysmon To Enrich Security Onion’s Host-Level Capabilities

[DefensiveDepth]

Newly published paper: Using Sysmon to Enrich Security Onion's Host-Level Capabilities:
http://defensivedepth.com/2015/03/27/using-sysmon-to-enrich-security-onions-host-level-capabilities/

Of particular note, I wrote an OSSEC decoder and a number of rules for Sysmon Event ID 1: Process Created, as well as ELSA parsers for both the Process Created and Network Connection Detected events. They can be found on Github... Feel free to tweak, contribute back, send feedback, etc:
https://github.com/defensivedepth?tab=repositories

Doug has already created the following issues to get some of this funcitionality integrated into the SO core:
https://github.com/Security-Onion-Solutions/security-onion/issues/706
https://github.com/Security-Onion-Solutions/security-onion/issues/707

Keep in mind that there may be issues with the current stable release of OSSEC (2.8) as the <eventchannel> bug is unfixed--

I believe the bug fix is slated to be released with 2.9...(https://github.com/ossec/ossec-hids/issues/224)

-Josh

[Doug Burks]

Great work, Josh!

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[Mike Pilkington]

That's fantastic!  Thanks Josh!

[Damon Rouse]

This is GREAT!

[Justin Henderson]

Do you know what would need modified to get this to parse correctly if the log came from a different source other than ossec? I am trying to modify your code to parse sysmon logs that come from evtsys with no luck. The problem is for large scale deployment of systems that are constantly changing it is far easier to collect logs with a traditional syslog setup similar to Snare or evtsys due the agent management overhead of ossec.

When testing the sysmon logs are coming in and classified as a program of sysmon and class of WINDOWS. The eventid is correct and the source is set to Process Create. However, all the other fields are not coming in parsed out so no reports can be run such as by Image or CommandLine.

Do you know what would be needed to allow parsing to work from evtsys instead of ossec?

[Doug Burks]

On Fri, May 15, 2015 at 9:51 AM, Justin Henderson
<jhend...@tekrefresh.com> wrote:
> Do you know what would need modified to get this to parse correctly if the log came from a different source other than ossec? I am trying to modify your code to parse sysmon logs that come from evtsys with no luck. The problem is for large scale deployment of systems that are constantly changing it is far easier to collect logs with a traditional syslog setup similar to Snare or evtsys due the agent management overhead of ossec.

Hi Justin,

What exactly do you mean by "agent management overhead of ossec"? I
recently helped an organization design an automated deployment of
OSSEC agents to hundreds of devices. I like OSSEC because it has
encrypted log transport (unlike cleartext syslog).

[Justin Henderson]

I love Ossec and have it deployed on servers but wasn't sure how to deploy it to a large amount of devices with non-static IPs. I'm looking to deploy to approximately 15,000 devices. Over 10 K of those are DHCP.

Do you think Ossec will be able to handle that properly? I'm not familiar with the method to do unattended installs to devices without static IPs. That is why I was planning on just using EvtSys for workstations.

Thoughts? I'm not planning on collecting everything, just certain events. If this is achievable on my workstations with Ossec I'd prefer that over EvtSys.

[Doug Burks]

Please see:
http://ossec-docs.readthedocs.org/en/latest/manual/agent/agent-dhcp-nat.html

On Mon, May 18, 2015 at 11:52 AM, Justin Henderson

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.

[mmfirm...@gmail.com]

This is awesome work! I'm wondering if anyone has created a step-by-step process for sysmon integration into SO? The paper was mostly a high level overview on the subject.

I'm just getting my feet wet with Windows Log Forwarding and sysmon and would like to do some testing.

[Justin Henderson]

Thanks Doug! This is exactly what I needed. I wasn't aware of this change to Ossec.

[James Taylor]

I have deployed it to about 150 host I think I needed to modify the OSSEC and ELSA parsers for SYSMON_PROCESS to work with SYSMON 3. Network connections seem to work ok as is.

We did something like this via group policy to setup the even forwarding.

http://blogs.technet.com/b/otto/archive/2008/07/08/quick-and-dirty-enterprise-eventing-for-windows.aspx

Not sure if you have more specific questions.

[DefensiveDepth]

@James I have not deployed 3.0 yet.. If edits were needed for the parser, would you mind pushing your changes upstream for ELSA & OSSEC(https://github.com/defensivedepth/Sysmon_ELSA_Parsers & https://github.com/defensivedepth/Sysmon_OSSEC/blob/master/Sysmon_OSSEC-Decoders.xml)

The current version of the OSSEC decoder has already been merged into the OSSEC project for release in 2.9, and I want to make sure that it is usable for Sysmon 3.0...

@mmfirm For specific instructions - As James already mentioned, use GPOs to setup event forwarding per the blog post, and deploy Sysmon to your workstations using whatever software deployment tool that you have in your environment... Anything specific beyond that?

Thanks,

-Josh

[James Taylor]

Josh, Next week I should have some time to verify that they actually needed modified I may have been dealing with other issues if that is the case I will try to push those changes to you or let you know if they are fine as is. I don't know much about this stuff don't want to cause extra issues.

Josh have you thought about parsers for any other events like createremotethread or imageload?


Thanks

[Justin Henderson]

I am using sysmon 3 and I believe things will need slight modifications in order to parse sysmon 3. Some of the layout such as in event ID 1 has changed.

--
You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/BtXoridbflQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.

[DefensiveDepth]

Saw the PR, (thanks!) will look into it as soon as possible, but it will probably be a couple weeks while I deploy 3.0 in my primary environment...

Yes, I have thought about alot of other parsers, but haven't been able to prioritize the time at this point... If you have never written parsers yourself, it can take a little bit of investment to get your head wrapped around it, but it is worth it!

Thanks

-Josh

[James Taylor]

Great Josh I think this is really valuable stuff for host detection again glad you wrote the paper. I am pulling a lot of valuable data from it already.

I am going to add the createremotethread parsers this week for testing. If I have success I will get them to you.

[Mike Pilkington]

Hi Josh,

You mentioned that 2.8 has an eventchannel bug.  I tried 2.8 with eventchannel and nothing showed up in ELSA. I then switched to eventlog and the logs are arriving in ELSA, but they're unparsed.  Should I use an older version of OSSEC with eventchannel instead?  I'm using sysmon 3.1 and all updates applied to SO via soup (as of earlier this week). Here are a couple sample events from ELSA:

Info

Sat Sep 19 02:31:38

2015 Sep 19 02:31:38 (WIN72) 192.168.100.150->WinEvtLog 2015 Sep 19 02:31:36 WinEvtLog: Microsoft-Windows-Sysmon/Operational: Information(1): no source: SYSTEM: NT AUTHORITY: WIN7SIFT: Process Create:

host=127.0.0.1 program=ossec_archive class=WINDOWS eventid=1 srcip=127.0.0.1 source=WIN7SIFT user= domain= share_name= share_path= share_target=

Info

Sat Sep 19 02:31:38

2015 Sep 19 02:31:38 (WIN72) 192.168.100.150->WinEvtLog 2015 Sep 19 02:31:36 WinEvtLog: Microsoft-Windows-Sysmon/Operational: Information(5): no source: SYSTEM: NT AUTHORITY: WIN7SIFT: Process terminated:

host=127.0.0.1 program=ossec_archive class=WINDOWS eventid=5 srcip=127.0.0.1 source=WIN7SIFT user= domain= share_name= share_path= share_target=

Thanks,

Mike

--
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.

[DefensiveDepth]

Yes, that is the bug, which was fixed 12/14 for 2.9. I posted yesterday in the OSSEC group, asking if we can merge the fix into the current stable (2.8) , feel free to comment in the thread.

For testing purposes, you can use the 2.9 pre-release that have it fixed.

https://groups.google.com/forum/#!topic/ossec-list/JA9x4uzDg1g

-Josh

[Mike Pilkington]

Thanks Josh.  I switched the ossec-agent.exe binary to a 2.9 beta, but still no luck.  I also changed the OSSEC config back to eventchannel.  Here's what it shows in the ossec agent config:

  <localfile>

    <location>Microsoft-Windows-Sysmon/Operational</location>

    <log_format>eventchannel</log_format>

  </localfile>

The binary I tried is 2.9.0-beta02, found a few posts down from the top of this page: https://github.com/ossec/ossec-hids/releases.

After the switch, I do get the events coming in ELSA, but they're still unparsed.  Any ideas what I've missed?

Sat Sep 19 20:04:43

2015 Sep 19 20:04:43 (WIN73) 192.168.100.150->WinEvtLog 2015 Sep 19 20:04:40 WinEvtLog: Microsoft-Windows-Sysmon/Operational: Information(1): no source: SYSTEM: NT AUTHORITY: WIN7SIFT: Process Create:

host=127.0.0.1 program=ossec_archive class=WINDOWS eventid=1 srcip=127.0.0.1 source=WIN7SIFT user= domain= share_name= share_path= share_target=

Sat Sep 19 20:04:43

2015 Sep 19 20:04:43 (WIN73) 192.168.100.150->WinEvtLog 2015 Sep 19 20:04:40 WinEvtLog: Microsoft-Windows-Sysmon/Operational: Information(5): no source: SYSTEM: NT AUTHORITY: WIN7SIFT: Process terminated:

host=127.0.0.1 program=ossec_archive class=WINDOWS eventid=5 srcip=127.0.0.1 source=WIN7SIFT user= domain= share_name= share_path= share_target=

Thanks,

Mike

-Josh

[Mike Pilkington]

Actually it looks like I sent example logs in my last email from before I made the switch.  After more testing, I realized new sysmon logs are not coming in after all.  I then checked the OSSEC log on the agent and saw this message:

   WARN: eventchannel not available on this version of OSSEC

I looked at compiling the latest version on Ubuntu using mingw, as described here: http://dcid.me/blog/2009/06/compiling-the-windows-agent-from-a-linux-system/ & here: http://ossec-docs.readthedocs.org/en/latest/manual/installation/compile-ossec-mingw.html.  It looks like a straightforward process, except neither the 2.9 beta 3 or 4 tar packages had the gen-win.sh file (and several others) needed to generate the Windows package.  So then I went to get beta 2 and noticed the pre-compiled version for download...but that's what gives me the error above.

Before I troubleshoot more, can you tell me which exact version you've successfully used with your sysmon parsers?  And if you have any suggestions on downloading or compiling that version, I'd appreciate it.

Thank you,

Mike

[DefensiveDepth]

Mike,

Yes, the pre-compiled 2.9 beta does not work for some reason - The binary that I use in dev and testing that works is from 1/15, which you are welcome to download here:

https://onedrive.live.com/redir?resid=300C2D5A3AA2CF4D!22186&authkey=!AHxyOgaUFQUB5hw&ithint=folder%2csha1

-Josh

[James Taylor]

Mike, I don't know much about this but to build the winagent I followed the dcid blog on ubuntu 14.04 vm installed mingw-w64 and nsis downloaded the latest beta snapshot moved into the src directory and ran #make TARGET=winagent

That built the setup I have been using in a VERY narrow production deployment, but like I said no clue if that is correct in anyway.

[Mike Pilkington]

Thanks for the feedback.  The version that Josh made available worked great for testing.  I'm now getting Sysmon parsed in ELSA. 

Thanks,

Mike