Don't see parsed fields in the Wazuh Dashbord

[MajorFudge]

Hello team,

I have an issue with data that comes from one of the agent.

Flow:

Windows server with Wazuh agent installed.

In the ossec.conf configuration I specify the localfile config to grab a DNS server logs:

<localfile>
    <location>%WINDIR%\SysNative\dns\log.txt</location>
    <log_format>syslog</log_format>
    <out_format>win_dns: $(log)</out_format>
  </localfile>

On the Wazuh server site I have a decoder:

<decoder name="windows_dns">
  <prematch>win_dns: </prematch>
  <regex type="pcre2">win_dns:\s(\d\d\.\d\d\.\d\d\d\d\s\d\d:\d\d:\d\d)\s(\d+)\s(\w+)\s+(\S+)\s(\w+)\s(\w+)\s(\S+)\s+(\S+)\s(\w|\s)\s(\w)\s\[(\S+)\s(\w|\s)\s{1,2}(\w+)\s+(\w+)\]\s(\w+)\s+(\S+)</regex>
  <order>timestamp, dns.thread.id, dns.context, dns.packet.id, dns.proto, dns.sendreceive, source.ip, dns.x.id, dns.queryresponse.id, dns.op.code, dns.flag.hex, dns.recursion.desired, dns.recursion.available, dns.code, dns.type, dns.domain.name</order>
</decoder>

My rule:

  <rule id="100003" level="5">
        <decoded_as>windows_dns</decoded_as>
        <description>Logs from the Windows DNS</description>
<!--        <options>no_full_log</options> -->
  </rule>

When I try to run rule test, I see that all works fine:

echo 'win_dns: 09.10.2023 18:09:55 1618 PACKET  000002AA7D1EBD40 UDP Rcv 10.6.4.11    8d6b   Q [0001   D   NOERROR] AAAA   (8)zbx-db01(4)mgmt(5)example(2)ru(0)' | sudo /var/ossec//bin/wazuh-logtest
Starting wazuh-logtest v4.4.4
Type one log per line


**Phase 1: Completed pre-decoding.
full event: 'win_dns: 09.10.2023 18:09:55 1618 PACKET  000002AA7D1EBD40 UDP Rcv 10.6.4.11    8d6b   Q [0001   D   NOERROR] AAAA   (8)zbx-db01(4)mgmt(5)example(2)ru(0)'

**Phase 2: Completed decoding.
name: 'windows_dns'
dns.code: 'NOERROR'
dns.context: 'PACKET'
dns.domain.name: '(8)zbx-db01(4)mgmt(5)example(2)ru(0)'
dns.flag.hex: '0001'
dns.op.code: 'Q'
dns.packet.id: '000002AA7D1EBD40'
dns.proto: 'UDP'
dns.queryresponse.id: ' '
dns.recursion.available: 'D'
dns.recursion.desired: ' '
dns.sendreceive: 'Rcv'
dns.thread.id: '1618'
dns.type: 'AAAA'
dns.x.id: '8d6b'
source.ip: '10.6.4.11'
timestamp: '09.10.2023 18:09:55'

**Phase 3: Completed filtering (rules).
id: '100003'
level: '5'
description: 'Logs from the Windows DNS'
groups: '['custom parser rules']'
firedtimes: '1'
mail: 'False'
**Alert to be generated.

I also see that it successful triggers the alert creation:

{"timestamp":"2023-10-09T18:11:15.358+0300","rule":{"level":5,"description":"Logs from the Windows DNS","id":"100003","firedtimes":26765,"mail":false,"groups":["custom parser rules"]},"agent":{"id":"018","name":"mgmt-dc01","ip":"10.6.3.11"},"manager":{"name":"wserver03-srv-infra"},"id":"1696864275.119181304","full_log":"win_dns: 09.10.2023 18:09:55 1618 PACKET  000002AA7D1EBD40 UDP Rcv 10.6.44.11    8d6b   Q [0001   D   NOERROR] AAAA   (8)zbx-db01(4)mgmt(5)nubes(2)ru(0)","decoder":{"name":"windows_dns"},"data":{"timestamp":"09.10.2023 18:09:55","dns":{"thread":{"id":"1618"},"context":"PACKET","packet":{"id":"000002AA7D1EBD40"},"proto":"UDP","sendreceive":"Rcv","x":{"id":"8d6b"},"queryresponse":{"id":" "},"op":{"code":"Q"},"flag":{"hex":"0001"},"recursion":{"desired":" ","available":"D"},"code":"NOERROR","type":"AAAA","domain":{"name":"(8)zbx-db01(4)mgmt(5)example(2)ru(0)"}},"source":{"ip":"10.96.44.111"}},"location":"C:\\Windows\\SysNative\\dns\\log.txt"}

The problem is that in the Wazuh dashboard I don't see parsed fields, I only see full logs:

{
  "_index": "wazuh-alerts-4.x-2023.10.09",
  "_id": "9sjqFIsBl3DRswTJRlTt",
  "_version": 1,
  "_score": null,
  "_source": {
    "input": {
      "type": "log"
    },
    "agent": {
      "ip": "10.6.3.11",
      "name": "mgmt-dc01",
      "id": "018"
    },
    "manager": {
      "name": "wserver02-srv-infra"
    },
    "rule": {
      "firedtimes": 181128,
      "mail": false,
      "level": 5,
      "description": "Logs from the Windows DNS",
      "groups": [
        "custom parser rules"
      ],
      "id": "100003"
    },
    "location": "C:\\Windows\\SysNative\\dns\\log.txt",
    "decoder": {
      "name": "windows_dns"
    },
    "id": "1696862978.3920412956",
    "full_log": "win_dns: 09.10.2023 17:48:17 0CEC PACKET  0000023D423F6530 UDP Snd 77.88.8.1       0be3   Q [0001   D   NOERROR] A      (3)dns(8)example(3)com(0)",
    "timestamp": "2023-10-09T17:49:38.516+0300"
  },
  "fields": {
    "timestamp": [
      "2023-10-09T14:49:38.516Z"
    ]
  },
  "sort": [
    1696862978516
  ]
}

{ "_index": "wazuh-alerts-4.x-2023.10.09", "_id": "9sjqFIsBl3DRswTJRlTt", "_version": 1, "_score": null, "_source": { "input": { "type": "log" }, "agent": { "ip": "10.96.3.11", "name": "mgmt-dc01", "id": "018" }, "manager": { "name": "wserver02-srv-infra" }, "rule": { "firedtimes": 181128, "mail": false, "level": 5, "description": "Logs from the Windows DNS", "groups": [ "custom parser rules" ], "id": "100003" }, "location": "C:\\Windows\\SysNative\\dns\\log.txt", "decoder": { "name": "windows_dns" }, "id": "1696862978.3920412956", "full_log": "win_dns: 09.10.2023 17:48:17 0CEC PACKET 0000023D423F6530 UDP Snd 77.88.8.1 0be3 Q [0001 D NOERROR] A (3)dns(8)msftncsi(3)com(0)", "timestamp": "2023-10-09T17:49:38.516+0300" }, "fields": { "timestamp": [ "2023-10-09T14:49:38.516Z" ] }, "highlight": { "full_log": [ "win_dns: 09.10.2023 17:48:17 0CEC PACKET 0000023D423F6530 UDP Snd 77.88.8.1 0be3 Q [0001 D NOERROR] A (3)@opensearch-dashboards-highlighted-field@dns@/opensearch-dashboards-highlighted-field@(8)msftncsi(3)com(0)" ] }, "sort": [ 1696862978516 ] }

{ "_index": "wazuh-alerts-4.x-2023.10.09", "_id": "0FvxFYsBe9E9sgqjPedT", "_version": 1, "_score": null, "_source": { "input": { "type": "log" }, "agent": { "ip": "10.96.3.11", "name": "mgmt-dc01", "id": "018" }, "manager": { "name": "wserver02-srv-infra" }, "rule": { "firedtimes": 115252, "mail": false, "level": 5, "description": "Logs from the Windows DNS", "groups": [ "custom parser rules" ], "id": "100003" }, "location": "C:\\Windows\\SysNative\\dns\\log.txt", "decoder": { "name": "windows_dns" }, "id": "1696880211.4543636953", "full_log": "win_dns: 09.10.2023 22:35:31 0D6C PACKET 000002AA7D03EC80 UDP Snd 10.98.72.11 4dda R U [05a8 REFUSED] SOA (2)72(2)98(2)10(7)in-addr(4)arpa(0)", "timestamp": "2023-10-09T22:36:51.980+0300" }, "fields": { "timestamp": [ "2023-10-09T19:36:51.980Z" ] }, "sort": [ 1696880211980 ] }

{ "_index": "wazuh-alerts-4.x-2023.10.09", "_id": "0FvxFYsBe9E9sgqjPedT", "_version": 1, "_score": null, "_source": { "input": { "type": "log" }, "agent": { "ip": "10.96.3.11", "name": "mgmt-dc01", "id": "018" }, "manager": { "name": "wserver02-srv-infra" }, "rule": { "firedtimes": 115252, "mail": false, "level": 5, "description": "Logs from the Windows DNS", "groups": [ "custom parser rules" ], "id": "100003" }, "location": "C:\\Windows\\SysNative\\dns\\log.txt", "decoder": { "name": "windows_dns" }, "id": "1696880211.4543636953", "full_log": "win_dns: 09.10.2023 22:35:31 0D6C PACKET 000002AA7D03EC80 UDP Snd 10.98.72.11 4dda R U [05a8 REFUSED] SOA (2)72(2)98(2)10(7)in-addr(4)arpa(0)", "timestamp": "2023-10-09T22:36:51.980+0300" }, "fields": { "timestamp": [ "2023-10-09T19:36:51.980Z" ] }, "sort": [ 1696880211980 ] }

[Laura Estefania Cepeda Tamayo]

Hello MajorFudge,

I'm reviewing your query on a lab, I'll be back with you soon.

Please let me know where exactly you are checking it in the dashboard.

Regards,

Laura Cepeda

[Laura Estefania Cepeda Tamayo]

Hi MajorFudge, hope you are having a great day,

Please let me know where exactly you are reviewing this information. Usuaully when there is an alert, you can go to the agent, then to the "Security Events", and then scroll down to the "Security Alerts" and select one, for example:

There you have three tabs, Table, JSON and Rule. The one that shows the log parsed is "Table" and "JSON" shows some coding:

Please share with me a screenshot of the Table tab so we can better review.

I'll be attentive to your response.

Kind regards,

Laura Cepeda

[MajorFudge]

Dear Laura, thank you for your reply.

I'm trying to find these events in the Wazuh Dashboard >> Discovery.

And I see only Full_log without any parsed fields.

If I try to check Wazuh > Agent > Agent 018 > Security Events:

It is completely emtpy and I don't event see 'Security alerts':

 

вторник, 10 октября 2023 г. в 16:28:48 UTC+2, Laura Estefania Cepeda Tamayo:

[Kirill Golubenko]

Hello, sorry for pushing. But do you have any updates regarding this issue?

Kind regards,

Kirill

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/L9CoKzWaTpE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/dbb842ba-24e2-49ab-9cea-77057c2a2c54n%40googlegroups.com.

[Laura Estefania Cepeda Tamayo]

Kirill, thanks for your response,

Discover and the Security Events would be different. From Discover, we would need to open the alert by clicking the arrow next to the date, in order to see the fields, please send me a screenshot of it and from the Table tab, it should look like this: 

You can also filter the agent there and the time. 

From the agent, is possible there were no alerts in the 24 hours, please change that field and also please send a screenshot of the Dashboard.

In this case we could also check if the agent is Active.

You can use:

/var/ossec/bin/agent_control -i <YOUR_AGENT_ID> | grep Status

And you can have more information here: 

Checking connection with the Wazuh manager

https://documentation.wazuh.com/current/user-manual/agents/agent-connection.html 

You can also check the archivess to see if somethinfg changed on that agent on:

/var/ossec/logs/archives/

I'll be aware of your response.

Kind regards,

Laura Cepeda

[MajorFudge]

Laura, on the top I sent you the JSON output of the Discover event.

It is the same thing that you have on your screenshot but in JSON format.

And as you can see, I don't see parsed date, I only see full message. That is the problem:

And I want to see that same fields that I see if run this test:

среда, 11 октября 2023 г. в 21:55:51 UTC+2, Laura Estefania Cepeda Tamayo:

[Kirill Golubenko]

Hello,

Any updates here?

Here is the screenshot from the Discovery:

As you can see, there is only a full_log without any parsed fields.

And when I try to test my rules and decoders, I see that parsing is actually works:

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/8006a12e-fb31-4b72-aaa4-1a9a50cfcc5cn%40googlegroups.com.

[Laura Estefania Cepeda Tamayo]

Hello Kirill, hope you are doing well, I apologize for the delay on the response,

I reviewed this on my lab and could see the fields decoded. Please check and share from the /var/ossec/logs/alerts/alerts.json the alert, you can use this to filter it:

grep -i '"id":"100003",' alerts.json 

You can find more information about the alerts.json here: https://documentation.wazuh.com/current/user-manual/capabilities/log-data-collection/how-it-works.html

Please share as well, the indexer information, in this case the indice according to the timestamp of the alert, you can do it from here: l https://documentation.wazuh.com/current/user-manual/wazuh-indexer/wazuh-indexer-indices.html

You can go to OpenSearch Plugins > Index Management and then to Indices. 

Thanks for your time,
Laura Cepeda