Custom Windows DNS Log in JSON Format Not Ingesting

[Bill Justesen]

I have a custom JSON log from a Windows server that should be ingesting into Wazuh. You can see my configuration below as well as a sample of what should be saving in the /var/ossec/logs/archives.json file. A new log file is created every minute, and can have anywhere from 20-150 entries. The Windows ossec.log file shows that it is analyzing the file, but entries from the file never appear in the archives. (Yes, I know that Wazuh limits analysis to 200 files, so I start purging at 150 files to ensure that Wauzh has time to ingest.)

I know the wazuh-archives-x.x-xxxx.xx.xx indexes are created...


..but I can't seem to search from them in the OpenSearch Dashboard Discover tool. It doesn't show them.


Any ideas why the contents aren't saved?

Wazuh Server
/etc/filebeat/filebeat.yml
filebeat.modules:
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: true

/var/ossec/etc/ossec.conf (portion of global directive)
  <global>
    <jsonout_output>yes</jsonout_output>
    <alerts_log>yes</alerts_log>
    <logall>no</logall>
    <logall_json>yes</logall_json>
  <global>

Windows Server
C:\Program Files (x86)\ossec-agent\ossec.conf
<ossec_config>
  <localfile>
    <location>C:\Logs\*.json</location>
    <log_format>json</log_format>
  </localfile>
</ossec_config>

C:\Program Files (x86)\ossec-agent\ossec.log
2022/11/04 06:29:15 wazuh-agent: INFO: (1950): Analyzing file: 'C:\DNSLogs\Microsoft-Windows-DNSServer-Analytical-20221004_163750.json'.

C:\DNSLogs\Microsoft-Windows-DNSServer-Analytical-20221004_163750.json
{"Timestamp":"2022-11-04T11:22:09.428Z","Message":"RECURSE_RESPONSE_IN","Flags":33152,"AD":0,"QTYPE":1,"TCP":0,"CacheScope":"Default","XID":14754,"ServerScope":".","AA":0,"InterfaceIP":"0.0.0.0","QNAME":"wd-prod-cp.trafficmanager.net","Source":"172.16.98.1","Port":0,"Id":261,"Version":0,"Qualifiers":null,"Level":4,"Task":2,"Opcode":0,"Keywords":"-9223372036854775776","RecordId":21,"ProviderName":"Microsoft-Windows-DNSServer","ProviderId":"eb79061a-a566-4698-9119-3ed2807060e7","LogName":null,"ProcessId":1844,"ThreadId":3836,"MachineName":"dc.domain.local","UserId":{"BinaryLength":12,"AccountDomainSid":null,"Value":"S-1-5-18"},"ActivityId":null,"RelatedActivityId":null,"LevelDisplayName":"Information","OpcodeDisplayName":"Info","TaskDisplayName":"RECURSE_QUERY"}
{"Timestamp":"2022-11-04T11:22:09.524Z","Message":"QUERY_RECEIVED","Source":"172.16.98.122","QTYPE":1,"TCP":0,"QNAME":"prd-collector-anon.ex.co","InterfaceIP":"172.16.98.254","Flags":256,"Port":50461,"RD":1,"XID":56510,"Id":256,"Version":0,"Qualifiers":null,"Level":4,"Task":1,"Opcode":0,"Keywords":"-9223372036854775807","RecordId":24,"ProviderName":"Microsoft-Windows-DNSServer","ProviderId":"eb79061a-a566-4698-9119-3ed2807060e7","LogName":null,"ProcessId":1844,"ThreadId":3836,"MachineName":"dc.domain.local","UserId":{"BinaryLength":12,"AccountDomainSid":null,"Value":"S-1-5-18"},"ActivityId":null,"RelatedActivityId":null,"LevelDisplayName":"Information","OpcodeDisplayName":"Info","TaskDisplayName":"LOOK_UP"}
{"Timestamp":"2022-11-04T11:22:09.525Z","Message":"QUERY_RECEIVED","Source":"172.16.98.103","QTYPE":1,"TCP":0,"QNAME":"prd-collector-anon.ex.co","InterfaceIP":"172.16.98.254","Flags":256,"Port":51369,"RD":1,"XID":46599,"Id":256,"Version":0,"Qualifiers":null,"Level":4,"Task":1,"Opcode":0,"Keywords":"-9223372036854775807","RecordId":26,"ProviderName":"Microsoft-Windows-DNSServer","ProviderId":"eb79061a-a566-4698-9119-3ed2807060e7","LogName":null,"ProcessId":1844,"ThreadId":3836,"MachineName":"dc.domain.local","UserId":{"BinaryLength":12,"AccountDomainSid":null,"Value":"S-1-5-18"},"ActivityId":null,"RelatedActivityId":null,"LevelDisplayName":"Information","OpcodeDisplayName":"Info","TaskDisplayName":"LOOK_UP"}
{"Timestamp":"2022-11-04T11:22:09.598Z","Message":"RECURSE_RESPONSE_IN","Flags":34176,"AD":0,"QTYPE":1,"TCP":0,"CacheScope":"Default","XID":14392,"ServerScope":".","AA":1,"InterfaceIP":"0.0.0.0","QNAME":"prd-collector-anon.ex.co","Source":"172.16.98.1","Port":0,"Id":261,"Version":0,"Qualifiers":null,"Level":4,"Task":2,"Opcode":0,"Keywords":"-9223372036854775776","RecordId":27,"ProviderName":"Microsoft-Windows-DNSServer","ProviderId":"eb79061a-a566-4698-9119-3ed2807060e7","LogName":null,"ProcessId":1844,"ThreadId":3836,"MachineName":"dc.domain.local","UserId":{"BinaryLength":12,"AccountDomainSid":null,"Value":"S-1-5-18"},"ActivityId":null,"RelatedActivityId":null,"LevelDisplayName":"Information","OpcodeDisplayName":"Info","TaskDisplayName":"RECURSE_QUERY"}
{"Timestamp":"2022-11-04T11:22:09.598Z","Message":"RESPONSE_SUCCESS","Zone":"Cache","Flags":33152,"AD":0,"QTYPE":1,"Destination":"172.16.98.103","RCODE":"0","XID":46599,"AA":0,"TCP":0,"Scope":"Default","InterfaceIP":"172.16.98.254","QNAME":"prd-collector-anon.ex.co","DNSSEC":"0","Port":51369,"Id":257,"Version":0,"Qualifiers":null,"Level":4,"Task":1,"Opcode":0,"Keywords":"-9223372036854775806","RecordId":28,"ProviderName":"Microsoft-Windows-DNSServer","ProviderId":"eb79061a-a566-4698-9119-3ed2807060e7","LogName":null,"ProcessId":1844,"ThreadId":3836,"MachineName":"dc.domain.local","UserId":{"BinaryLength":12,"AccountDomainSid":null,"Value":"S-1-5-18"},"ActivityId":null,"RelatedActivityId":null,"LevelDisplayName":"Information","OpcodeDisplayName":"Info","TaskDisplayName":"LOOK_UP"}
{"Timestamp":"2022-11-04T11:22:09.598Z","Message":"RESPONSE_SUCCESS","Zone":"Cache","Flags":33152,"AD":0,"QTYPE":1,"Destination":"172.16.98.122","RCODE":"0","XID":56510,"AA":0,"TCP":0,"Scope":"Default","InterfaceIP":"172.16.98.254","QNAME":"prd-collector-anon.ex.co","DNSSEC":"0","Port":50461,"Id":257,"Version":0,"Qualifiers":null,"Level":4,"Task":1,"Opcode":0,"Keywords":"-9223372036854775806","RecordId":29,"ProviderName":"Microsoft-Windows-DNSServer","ProviderId":"eb79061a-a566-4698-9119-3ed2807060e7","LogName":null,"ProcessId":1844,"ThreadId":3836,"MachineName":"dc.domain.local","UserId":{"BinaryLength":12,"AccountDomainSid":null,"Value":"S-1-5-18"},"ActivityId":null,"RelatedActivityId":null,"LevelDisplayName":"Information","OpcodeDisplayName":"Info","TaskDisplayName":"LOOK_UP"}

[Bill Justesen]

All right, I figured out the OpenSearch Dashboard issue and have it resolved. What is still unresolved, however, is the fact that none of the JSON files are ingested by Wazuh.

When I search the archives.json file from PuTTY, the DNS entries are completely missing.

[Bill Justesen]

SOLUTION: My script was saving the log files as UTF-16. When I converted them to UTF-8, the floodgates opened!