eventchannel decoder testing
Craig
<localfile>
<location>Microsoft-Windows-Sysmon/Operational</location>
<log_format>eventchannel</log_format>
</localfile>
I am using the default sysmon decoder included on my server:
<decoder name="Sysmon-EventID#1">
<type>windows</type>
<prematch>INFORMATION\(1\)</prematch>
<regex offset="after_prematch">Image: (\.*) \s*CommandLine: \.* \s*User: (\.*) \s*LogonGuid: \S* \s*LogonId: \S* \s*TerminalSessionId: \S* \s*IntegrityLevel: \.*HashType: \S* \s*Hash: (\S*) \s*ParentProcessGuid: \S* \s*ParentProcessID: \S* \s*ParentImage: (\.*) \s*ParentCommandLine:</regex>
<order>status,user,url,data</order>
</decoder>
I modified the default sysmon rule so that I would capture all process creates by setting the level to 1:
<rule id="184700" level="1">
<if_sid>18100</if_sid>
<description>Sysmon - Process Create Event</description>
</rule>
I would think that i would now see all process creates in my alerts.log but unfortunately I don't see any sysmon events at all. Any idea on the best way to troubleshoot this? Thank you!
dan (ddp)
on the server, and watch the archives.log.
When you find a sysmon log, you can use that with ossec-logtest to try
and figure out what is going on.
>
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ossec-list" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ossec-list+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
lostinthetubez
Delving into Sysmon event log parsing reveals just how monumental a task it is to parse out useful information from Windows event logs. The challenge is that nearly each and every Event ID has a different log format, which essentially means that almost every Event ID needs its own decoder... I may be waxing a little dramatic here, but the point is that to properly parse Windows logs, the original decoder needs to be made more generic and LOTS more child decoders need to be developed. At least, that is the approach I took, personally. Maybe I’m totally off base. Been testing it for a few months and it seems to work OK, but I haven’t done any auditing to see if I’ve broken anything. Anyway, here’s what I did and what works for me at the moment. If you go this route, you’ll need to comment out the original windows decoder in /var/ossec/etc/decoder.xml (and whatever else sysmon-related might have made it in there since last I looked; I’m not running the latest beta). I put these in local_decoder.xml:
<decoder name="windows">
<type>windows</type>
<prematch>^\d\d\d\d \w\w\w \d\d \d\d:\d\d:\d\d WinEvtLog: |^WinEvtLog: </prematch>
</decoder>
<decoder name="windows-defaultlogs">
<parent>windows</parent>
<type>windows</type>
<use_own_name>true</use_own_name>
<prematch offset="after_parent">^Application: |^Security: |^System: </prematch>
<regex offset="after_parent">^\.+: (\w+)\((\d+)\): (\.+): </regex>
<regex>(\.+): \.+: (\S+): </regex>
<order>status, id, extra_data, user, system_name</order>
<fts>name, location, user, system_name</fts>
</decoder>
<decoder name="windows-sysmon-eventID1">
<parent>windows</parent>
<type>windows</type>
<prematch offset="after_parent">^Microsoft-Windows-Sysmon/Operational: INFORMATION\(1\)</prematch>
<regex offset="after_parent">^Microsoft-Windows-Sysmon/Operational: (\w+)\((\d)\): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: (\S+): Process Create: \.+ ProcessId: \d+ Image: (\.*) CommandLine: \.*</regex>
<regex> User: (\.*) LogonGuid: \S* LogonId: \S* TerminalSessionId: \d* IntegrityLevel: \w* Hashes: \w+=(\w*) ParentProcessGuid: \S* ParentProcessID: \S* ParentImage: (\.+)</regex>
<order>status, id, system_name, dstuser, srcuser, url, extra_data, extra_data</order>
</decoder>
Put this in your local_rules.xml:
<rule id="184600" level="1">
<if_sid>18101</if_sid>
<id>^1$</id>
<description>Sysmon Process Launch Event</description>
</rule>
If you haven’t done so already, it is always helpful to enable logall mode when you’re working on new decoders. This will retain a copy of every single log line sent to your OSSEC manager. In your ossec.conf on the manager, put <logall>yes</logall> in a global tag somewhere and restart the service. You will now have a record of all logs sent to the manager, not just those that generate alerts. The current day’s archival logs are in /var/ossec/logs/archives/archives.log. Note that in order to run these particular logs through ossec-logtest, you’ll need to remove a prepended bit of text. So, edit a log entry like this:
2016 Jul 29 08:33:17 (hostname) 100.200.123.123->WinEvtLog 2016 Jul 29 08:36:08 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: hostname.subdomain.domain.tld: Process Create: UtcTime: 2016-07-29 15:36:08.268 ProcessGuid: {A560AB96-77E8-579B-0000-0010B7B17E50} ProcessId: 50292 Image: C:\Program Files (x86)\KeePass Password Safe 2\KeePass.exe CommandLine: "C:\Program Files (x86)\KeePass Password Safe 2\KeePass.exe" CurrentDirectory: C:\Program Files (x86)\KeePass Password Safe 2\ User: domain\username LogonGuid: {A560AB96-40DE-578E-0000-00209886AB02} LogonId: 0x2AB8698 TerminalSessionId: 1 IntegrityLevel: Medium Hashes: SHA1=5F5AC91EB83EFB6C4171AFF9EC1ED98EBA1C6A6C ParentProcessGuid: {A560AB96-40E0-578E-0000-0010285AAC02} ParentProcessId: 7540 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE
to become:
2016 Jul 29 08:36:08 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: hostname.subdomain.domain.tld: Process Create: UtcTime: 2016-07-29 15:36:08.268 ProcessGuid: {A560AB96-77E8-579B-0000-0010B7B17E50} ProcessId: 50292 Image: C:\Program Files (x86)\KeePass Password Safe 2\KeePass.exe CommandLine: "C:\Program Files (x86)\KeePass Password Safe 2\KeePass.exe" CurrentDirectory: C:\Program Files (x86)\KeePass Password Safe 2\ User: domain\username LogonGuid: {A560AB96-40DE-578E-0000-00209886AB02} LogonId: 0x2AB8698 TerminalSessionId: 1 IntegrityLevel: Medium Hashes: SHA1=5F5AC91EB83EFB6C4171AFF9EC1ED98EBA1C6A6C ParentProcessGuid: {A560AB96-40E0-578E-0000-0010285AAC02} ParentProcessId: 7540 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE
before copy/pasting into ossec-logtest. This is the best way to go about testing an eventchannel log. You get to see exactly what is decoded and which rules are triggered.
--
Craig
Jesus Linares
2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE2016 Jul 29 22:32:25 (WIN7-X64-PC1) 172.16.213.5->WinEvtLog 2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE
**Phase 1: Completed pre-decoding.
full event: '2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE'
hostname: 'ip-10-0-0-10'
program_name: '(null)'
log: '2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE'
**Phase 2: Completed decoding.
decoder: 'windows'
status: 'C:\Users\administrator\Desktop\svchost.exe'
dstuser: 'HACKME\Administrator'
url: 'C019D10F80409FC4C7D45EBFA48B0076'
extra_data: 'C:\Windows\explorer.exe'
**Phase 3: Completed filtering (rules).
Rule id: '184666'
Level: '12'
Description: 'Sysmon - Suspicious Process - svchost.exe'
**Alert to be generated.
You can find decoders for all sysmon events here.
Jesus Linares
2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE
**Phase 1: Completed pre-decoding.
full event: '2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE'
hostname: 'ip-10-0-0-10'
program_name: '(null)'
log: '2016 Jul 29 22:32:24 WinEvtLog: Microsoft-Windows-Sysmon/Operational: INFORMATION(1): Microsoft-Windows-Sysmon: SYSTEM: NT AUTHORITY: Win7-x64-PC1.hackme.local: Process Create: UtcTime: 2016-07-30 03:32:24.846 ProcessGuid: {67C360F4-1FC8-579C-0000-001017F41E00} ProcessId: 3988 Image: C:\Users\administrator\Desktop\svchost.exe CommandLine: "C:\Users\administrator\Desktop\svchost.exe" CurrentDirectory: C:\Users\administrator\Desktop\ User: HACKME\Administrator LogonGuid: {67C360F4-1C55-579C-0000-00206BBC0600} LogonId: 0x6bc6b TerminalSessionId: 1 IntegrityLevel: High Hashes: MD5=C019D10F80409FC4C7D45EBFA48B0076 ParentProcessGuid: {67C360F4-1C57-579C-0000-001092EC0600} ParentProcessId: 3056 ParentImage: C:\Windows\explorer.exe ParentCommandLine: C:\Windows\Explorer.EXE'
**Phase 2: Completed decoding.
decoder: 'windows'
status: 'C:\Users\administrator\Desktop\svchost.exe'
dstuser: 'HACKME\Administrator'
url: 'C019D10F80409FC4C7D45EBFA48B0076'
extra_data: 'C:\Windows\explorer.exe'
**Phase 3: Completed filtering (rules).
Rule id: '184666'
Level: '12'
Description: 'Sysmon - Suspicious Process - svchost.exe'On Monday, August 1, 2016 at 9:46:31 AM UTC+2, Jesus Linares wrote:
Hi Craig,the raw event is:
Craig
Jesus Linares
dan (ddp)
> So, interesting. I guess that is my question. In my testing (using 2.9RC2),
> the decoder below won't recognize the log entry unless I keep the header
> from archives.log (you can see the output in my post above). If I remove the
> header, the decoder doesn't work. What version were you running with your
> testing of my log entry?
> <decoder name="Sysmon-EventID#1">
> <type>windows</type>
> <prematch>INFORMATION\(1\)</prematch>
> <regex offset="after_prematch">Image: (\.*) \s*CommandLine: \.* \s*User:
> (\.*) \s*LogonGuid: \S* \s*LogonId: \S* \s*TerminalSessionId: \S*
> \s*IntegrityLevel: \.*HashType: \S* \s*Hash: (\S*) \s*ParentProcessGuid: \S*
> \s*ParentProcessID: \S* \s*ParentImage: (\.*) \s*ParentCommandLine:</regex>
> <order>status,user,url,data</order>
> </decoder>
>
>
lostinthetubez
Craig,
Hm... I just now noticed your exact symptoms while playing with a test OSSEC server that was created from a relatively recent git clone of the repository (cloned within the last month or two?). Take a look at your original output of ossec-logtest, under “Prepended Data Removed”. Look at the parsed “log:” field. Ossec-logtest is stripping “2016 Jul 29 22:32:24 WinEvtLog:” before processing it against the decoders. It isn’t supposed to be doing this. At least, this was not the behavior under 2.8.3. I do not have time to test the latest build from the repo to see if this problem has been fixed, though you might give that a whirl if you have the luxury of time. If you just want to make things work correctly, build your OSSEC server from the last known-good release, which is 2.8.3, or just follow Jesus’ suggestion and try out the Wazuh build.
Craig Mitchell
--
---
You received this message because you are subscribed to a topic in the Google Groups "ossec-list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ossec-list/X0uBpH1nEDE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ossec-list+...@googlegroups.com.
lostinthetubez
I’d give Wazuh a whirl, if I were you. They’ve got decoders and rules for sysmon, as well as eventchannel working (or I assume they do, if they have that stuff setup for sysmon).
My current decoder/rule development server and agents have been around long enough that I don’t recall what sort of Frankenstein mixture of agent and server versions I am running. Now that I think back on it, I’m fairly certain I had to track down a fixed version of the Windows agent in the listserv archives to make 2.8.3 work.
Jesus Linares
- Create a backup of your decoder.xml
- Replace "windows decoder" copying from line 174 to 417 of this file (https://github.com/wazuh/ossec-rules/blob/master/rules-decoders/ossec/decoders/windows_decoders.xml#L174)
- Restart ossec
Jesus Linares
Ossec-logtest is stripping “2016 Jul 29 22:32:24 WinEvtLog:” before processing it against the decoders. It isn’t supposed to be doing this. At least, this was not the behavior under 2.8.3...
dan (ddp)
<lostint...@gmail.com> wrote:
> I’d give Wazuh a whirl, if I were you. They’ve got decoders and rules for
> sysmon, as well as eventchannel working (or I assume they do, if they have
> that stuff setup for sysmon).
>
>
>
> My current decoder/rule development server and agents have been around long
> enough that I don’t recall what sort of Frankenstein mixture of agent and
> server versions I am running. Now that I think back on it, I’m fairly
> certain I had to track down a fixed version of the Windows agent in the
> listserv archives to make 2.8.3 work.
>
lostinthetubez
> -----Original Message-----
> From: ossec...@googlegroups.com [mailto:ossec...@googlegroups.com]
> On Behalf Of dan (ddp)
> Sent: Wednesday, August 3, 2016 5:52 AM
> To: ossec...@googlegroups.com
> Subject: Re: [ossec-list] eventchannel decoder testing
>
> <lostint...@gmail.com> wrote:
> > I’d give Wazuh a whirl, if I were you. They’ve got decoders and rules for
> > sysmon, as well as eventchannel working (or I assume they do, if they have
> > that stuff setup for sysmon).
> >
> >
> >
> > My current decoder/rule development server and agents have been
> around long
> > enough that I don’t recall what sort of Frankenstein mixture of agent and
> > server versions I am running. Now that I think back on it, I’m fairly
> > certain I had to track down a fixed version of the Windows agent in the
> > listserv archives to make 2.8.3 work.
> >
>
> Do you have more information on this? It doesn't sound familiar.
https://groups.google.com/forum/#!topic/ossec-list/o1SXX5Wk0A0
I will try to do more testing with the current 2.9 beta code in a week or two, see if I can validate any of the issues I observed earlier in the thread.