ExtraHop CEF format data to wazuh dashboard

[Aizat Yaacob]

Hi All, Thanks for your time, i am new to Wazuh, and need your help.

I have ExtraHop Appliance and configured it to send logs using CEF format to Wazuh.

I can see the log enter into /var/ossec/logs/archives/archives.json

below are an example of the CEF format of EH logs:

{

"timestamp":"2021-10-04T07:20:52.456+0000",

"rule":{"level":2,"description":"Unknown problem somewhere in the system.","id":"1002","firedtimes":5,"mail":false,"groups":["syslog","errors"],"gpg13":["4.3"]},

"agent":{"id":"000","name":"wazuh-manager"},

"manager":{"name":"wazuh-manager"},

"id":"1633332052.824463",

"full_log":"709 <14>2021-10-04T15:19:30.141+08:00 EDA CEF:0|ExtraHop|Reveal(x)|7.8|1|Daily Summary: Credentials Received over HTTP|6|cn1=127 cn1Label=detectionID cn2=60 cn2Label=riskScore cs1=https://192.168.0.111/extrahop/#/detections/detail/127 cs1Label=detectionURL cs2=sec,sec.caution cs2Label=category rt=2021-10-04T07:19:30.087Z end=2021-10-04T07:19:30.087Z start=2021-10-04T07:19:30.087Z src=00:0C:29:C3:9C:96 msg=Over the past day, servers received HTTP data with passwords or basic authentication headers. If the unencrypted HTTP data is accessible to an attacker, they can collect sensitive information. Confirm that the SSL/TLS protocol or HTTP Strict Transport Security (HSTS) policy is enabled for these servers.",

"decoder":{},

"location":"192.168.0.111"

}

So I learn about decoders, I developed a decoder according to Wazuh documentation. and below are the decoder file that i have made.

<decoder name="ExtraHop">

  <parent>ExtraHop</parent>

  <regex>^(\d+)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|</regex>

  <order>CEFversion,Vendor,Product,ProductVersion,EventID,EventName,EventSeverity</order>

</decoder>

<!--

cn1=127 

cn1Label=detectionID 

cn2=60 

cn2Label=riskScore 

cs1=https://192.168.0.111/extrahop/#/detections/detail/127 

cs1Label=detectionURL 

cs2=sec,sec.caution 

cs2Label=category 

rt=2021-10-04T07:19:30.087Z 

end=2021-10-04T07:19:30.087Z 

start=2021-10-04T07:19:30.087Z 

src=00:0C:29:C3:9C:96 

msg=

-->

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cn1=(\.*)\s\w+=|cn1=(\.*)$</regex>

   <order>cn1</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cn1Label=(\.*)\s\w+=|cn1Label=(\.*)$</regex>

   <order>cn1Label</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cn2=(\.*)\s\w+=|cn2=(\.*)$</regex>

   <order>cn2</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cn2Label=(\.*)\s\w+=|cn2Label=(\.*)$</regex>

   <order>cn2Label</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cs1=(\.*)\s\w+=|cs1=(\.*)$</regex>

   <order>cs1</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cs1Label=(\.*)\s\w+=|cs1Label=(\.*)$</regex>

   <order>cs1Label</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cs2=(\.*)\s\w+=|cs2=(\.*)$</regex>

   <order>cs2</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>cs2Label=(\.*)\s\w+=|cn2Label=(\.*)$</regex>

   <order>cs2Label</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>rt=(\.*)\s\w+=|rt=(\.*)$</regex>

   <order>rt</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>end=(\.*)\s\w+=|end=(\.*)$</regex>

   <order>end</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>start=(\.*)\s\w+=|start=(\.*)$</regex>

   <order>start</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>source=(\.*)\s\w+=|source=(\.*)$</regex>

   <order>source</order>

</decoder>

<decoder name="ExtraHop">

   <parent>ExtraHop</parent>

   <regex>msg=(\.*)\s\w+=|msg=(\.*)$</regex>

   <order>msg</order>

</decoder>

The question here is, how can I show the data from ExtraHop to the Wazuh dashboard?

Thanks in advance~!

[Mariano Koremblum]

Hi Aizat,

You have some errors regarding the decoders.

• 1st: The parent/father decoder should not have a parent label (the first one)
• 2nd: The parent decoder should have a prematch or program_name label to first match a log.

Also, in the case of the logs that you are using, they start with 709 <14> and that is not a default log format, so it can not be pre-decoded. But we can use these codes to set the prematch as follows:

<decoder name="ExtraHop">
  <prematch>^\d+ \<\d+></prematch>
</decoder>

Then, to extract the timestamp, we can use a sibling decoder (with a name different to “timestamp”, as it is a reserved name):

<!-- 2021-10-04T15:19:30.141+08:00 -->

<decoder name="ExtraHop">
  <parent>ExtraHop</parent>

  <regex>\<\d+>(\d+-\d+-\d+\w+\d+:\d+:\d+.\d+\p\d+:\d+) </regex>
  <order>log_timestamp</order>
</decoder>

And then your others decoders work as expected, just removed the “^“:

<decoder name="ExtraHop">
  <parent>ExtraHop</parent>

  <regex>(\d+)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|</regex>

When I tested it with wazuh-logtest I got the following output:

╰─# /var/ossec/bin/wazuh-logtest                  
Starting wazuh-logtest v4.2.4
Type one log per line

709  <14>2021-10-04T15:19:30.141+08:00 EDA  CEF:0|ExtraHop|Reveal(x)|7.8|1|Daily Summary: Credentials Received over  HTTP|6|cn1=127 cn1Label=detectionID cn2=60 cn2Label=riskScore  cs1=https://192.168.0.111/extrahop/#/detections/detail/127  cs1Label=detectionURL cs2=sec,sec.caution cs2Label=category  rt=2021-10-04T07:19:30.087Z end=2021-10-04T07:19:30.087Z  start=2021-10-04T07:19:30.087Z src=00:0C:29:C3:9C:96 msg=Over the past  day, servers received HTTP data with passwords or basic authentication  headers. If the unencrypted HTTP data is accessible to an attacker, they  can collect sensitive information. Confirm that the SSL/TLS protocol or  HTTP Strict Transport Security (HSTS) policy is enabled for these  servers.

**Phase 1: Completed pre-decoding.
    full event:  '709 <14>2021-10-04T15:19:30.141+08:00 EDA  CEF:0|ExtraHop|Reveal(x)|7.8|1|Daily Summary: Credentials Received over  HTTP|6|cn1=127 cn1Label=detectionID cn2=60 cn2Label=riskScore  cs1=https://192.168.0.111/extrahop/#/detections/detail/127  cs1Label=detectionURL cs2=sec,sec.caution cs2Label=category  rt=2021-10-04T07:19:30.087Z end=2021-10-04T07:19:30.087Z  start=2021-10-04T07:19:30.087Z src=00:0C:29:C3:9C:96 msg=Over the past  day, servers received HTTP data with passwords or basic authentication  headers. If the unencrypted HTTP data is accessible to an attacker, they  can collect sensitive information. Confirm that the SSL/TLS protocol or  HTTP Strict Transport Security (HSTS) policy is enabled for these  servers.'

**Phase 2: Completed decoding.
    name: 'ExtraHop'
    CEFversion: '0'
    EventID: '1'
    EventName: 'Daily Summary: Credentials Received over HTTP'
    EventSeverity: '6'
    Product: 'Reveal(x)'
    ProductVersion: '7.8'
    Vendor: 'ExtraHop'
    cn1: '127'
    cn1Label: 'detectionID'
    cn2: '60'
    cn2Label: 'riskScore'
    cs1: 'https://192.168.0.111/extrahop/#/detections/detail/127'
    cs1Label: 'detectionURL'
    cs2: 'sec,sec.caution'
    cs2Label: 'category'
    end: '2021-10-04T07:19:30.087Z'
    log_timestamp: '2021-10-04T15:19:30.141+08:00'
     msg: 'Over the past day, servers received HTTP data with passwords or  basic authentication headers. If the unencrypted HTTP data is accessible  to an attacker, they can collect sensitive information. Confirm that  the SSL/TLS protocol or HTTP Strict Transport Security (HSTS) policy is  enabled for these servers.'
    rt: '2021-10-04T07:19:30.087Z'
    start: '2021-10-04T07:19:30.087Z'

**Phase 3: Completed filtering (rules).
    id: '1002'
    level: '2'
    description: 'Unknown problem somewhere in the system.'
    groups: '['syslog', 'errors']'
    firedtimes: '1'
    gpg13: '['4.3']'
    mail: 'False'

I would strongly recommend you to read the following Wazuh documentation pages in order to have a better understanding of how to elaborate custom rules and decoders:

• https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/
• https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/sibling-decoders.html
• https://wazuh.com/blog/sibling-decoders-flexible-extraction-of-information/
• https://documentation.wazuh.com/current/user-manual/capabilities/wazuh-logtest/how-it-works.html
• https://documentation.wazuh.com/current/development/wazuh-logtest.html
• https://documentation.wazuh.com/current/user-manual/reference/tools/wazuh-logtest.html#wazuh-logtest

I hope that my answer helps you!

Best Regards,

Mariano Koremblum

​

[Mariano Koremblum]

For some weird reason, the previous message’s XML visualization is not the best, so here it is:

<decoder name="ExtraHop">
  <prematch>^\d+ \<\d+></prematch>
</decoder>

<decoder name="ExtraHop">
  <parent>ExtraHop</parent>
  <regex>\<\d+>(\d+-\d+-\d+\w+\d+:\d+:\d+.\d+\p\d+:\d+) </regex>
  <order>log_timestamp</order>
</decoder>

<decoder name="ExtraHop">
  <parent>ExtraHop</parent>
  <regex>(\d+)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|(\.*)\|</regex>
  <order>CEFversion,Vendor,Product,ProductVersion,EventID,EventName,EventSeverity</order>
</decoder>

<!--
cn1=127 
cn1Label=detectionID 
cn2=60 
cn2Label=riskScore 
cs1=https://192.168.0.111/extrahop/#/detections/detail/127 
cs1Label=detectionURL 
cs2=sec,sec.caution 
cs2Label=category 
rt=2021-10-04T07:19:30.087Z 
end=2021-10-04T07:19:30.087Z 
start=2021-10-04T07:19:30.087Z 
src=00:0C:29:C3:9C:96 
msg=
-->

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cn1=(\.*)\s\w+=|cn1=(\.*)$</regex>
   <order>cn1</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cn1Label=(\.*)\s\w+=|cn1Label=(\.*)$</regex>
   <order>cn1Label</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cn2=(\.*)\s\w+=|cn2=(\.*)$</regex>
   <order>cn2</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cn2Label=(\.*)\s\w+=|cn2Label=(\.*)$</regex>
   <order>cn2Label</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cs1=(\.*)\s\w+=|cs1=(\.*)$</regex>
   <order>cs1</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cs1Label=(\.*)\s\w+=|cs1Label=(\.*)$</regex>
   <order>cs1Label</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cs2=(\.*)\s\w+=|cs2=(\.*)$</regex>
   <order>cs2</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>cs2Label=(\.*)\s\w+=|cn2Label=(\.*)$</regex>
   <order>cs2Label</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>rt=(\.*)\s\w+=|rt=(\.*)$</regex>
   <order>rt</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>end=(\.*)\s\w+=|end=(\.*)$</regex>
   <order>end</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>start=(\.*)\s\w+=|start=(\.*)$</regex>
   <order>start</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>source=(\.*)\s\w+=|source=(\.*)$</regex>
   <order>source</order>
</decoder>

<decoder name="ExtraHop">
   <parent>ExtraHop</parent>
   <regex>msg=(\.*)\s\w+=|msg=(\.*)$</regex>
   <order>msg</order>
</decoder>

​

[Aizat Yaacob]

Hi, thanks for your reply, will try to deploy the xml and also read more on decoders and rules.

Best,
Aizat