Suppress log send

[Harvey Ryu]

Hi,

I recently collected windows event log 4688, it's process creation log, the logs are like this:

EventData

SubjectUserSid

S-1-5-21-1329622929-2207338157-1914140016-1024

SubjectUserName

h

SubjectDomainName

CNEHFKSA

SubjectLogonId

0x7282d

NewProcessId

0x3028

NewProcessName

C:\Windows\System32\HOSTNAME.EXE

TokenElevationType

%%1938

ProcessId

0x34f8

CommandLine

hostname

TargetUserSid

S-1-0-0

TargetUserName

-

TargetDomainName

-

TargetLogonId

0x0

ParentProcessName

C:\Windows\System32\cmd.exe

MandatoryLabel

S-1-16-8192 it's useful for me to see exe like cmd or powershell and it's command line parameter, but there is also a lot of noisy, like svchost and SearchHost and a lot of windows process, I hope there is a way to filter which condition of a log not to send to wazuh manager or only send specific log, like this case I want to send event 4668 and also the ParatProcessName or NewProcessName matchs cmd.exe, because windows seems don't have a way to filter it, so when it comes to a large scale, it won't take a lot of network bandwidth, in order to achieve that, the agent must do the filter work, I know there is a <query> tag, but I haven't find it's syntax, I don't know the <match> tag in rule configuration can be used here? And suddenly a question came to my mind, if a agent can't communicate to manager, how many logs does it temporary cache till next successful connection. Or it mark last successful send is like "point A" and wait till next successful manager connection and send from "point A" to every new events.

[Juan Pablo Saez]

Hello Harvery,

it's useful for me to see exe like cmd or powershell and it's command line parameter, but there is also a lot of noisy, like svchost and SearchHost and a lot of windows process, I hope there is a way to filter which condition of a log not to send to wazuh manager or only send specific log, like this case I want to send event 4668 and also the ParatProcessName or NewProcessName matchs cmd.exe, because windows seems don't have a way to filter it, so when it comes to a large scale, it won't take a lot of network bandwidth, in order to achieve that, the agent must do the filter work

• Yes, 4668 ID events can get floody really quick. It is possible to filter alerts by checking event fields(i.e: as you said above checking the ParatProcessName or NewProcessName value). 
• On the agent side, you can enable or disable the polling of certain events through its ID. If you need more granular filtering it has to be performed on the manager side as the agent only has forwarding capabilities. 
  
• I understand the bandwidth usage can be a worry especially if you are working in cloud environment. I think you shouldn't worry about a big bandwidth usage as it is just text. Anyway if you experience a large consumption regarding Wazuh agents, let us know, please.
• I think a rule like the further one can suit your use case. You can place it on /var/ossec/etc/rules/local_rules.xml:
    <rule id="200002" level="5">

      <if_sid>60103</if_sid>

      <field name="win.system.eventID">4688</field>

      <field name="win.eventdata.parentProcessName">C:\\Windows\\System32\\cmd.exe</field>

      <description> New CMD process </description>

   </rule>
  
  

When you talk about a lot of 4688 events, are you talking about event flooding or alert flooding (An event is just a log line, an alert is an event that was checked against a rule and tripped the rule) ? Please, could you explain this a little deeper and paste here some example alerts?

And suddenly a question came to my mind, if a agent can't communicate to manager, how many logs does it temporary cache till next successful connection. Or it mark last successful send is like "point A" and wait till next successful manager connection and send from "point A" to every new events

The agent has a buffer where the events can wait if the manager isn't available. The further agent configuration block allows customizing the buffer characteristics

  <client_buffer>

    <disabled>no</disabled>

    <queue_size>5000</queue_size>

    <events_per_second>500</events_per_second>

  </client_buffer>

By default (the configuration above is the default one), the agent buffer can keep 5000 events and the events can be shipped to the manager with an upper limit rate of 500 events per second. A full buffer will drop incoming events. You can read more about in our Anti-flooding and events buffering mechanism capabilities document. 

I hope it helps. Count on me to tune Wazuh until it suits your needs.

Greetings, JP Sáez

[Harvey Ryu]

So, the agent collect log can only choose eventID, can't choose field like "win.eventdata.parentProcessName" equal to "cmd.exe" in the event, and if I choose to collect event 4688, all the 4688 event will be send to wazuh manager, right? I don't have a alert flood issue, just think the event may flood.
So is there a way to test windows agent's log are all collected to wazuh manager without missing, so I can do a stress test, like is there a file I can tail in linux to see the collected events. So I will know when to added a additional wazuh server to collect.

[Juan Pablo Saez]

Hello again Harvery,

So, the agent collect log can only choose eventID, can't choose field like "win.eventdata.parentProcessName" equal to "cmd.exe" in the event, and if I choose to collect event 4688, all the 4688 event will be send to wazuh manager, right? I don't have a alert flood issue, just think the event may flood.

Exactly, on the agent side, it is possible to silence certain Eventchannel events trought its event ID but not discriminate according to the content of the fields. So, yes, if you choose to collect event 4688, all the 4688 events will be sent to the Wazuh manager. The events are collected/disabled through an <localfile> option in the agent's ossec.conf file as in the example below:

  <localfile>

    <location>Security</location>

    <log_format>eventchannel</log_format>

    <query>Event/System[EventID != 5145 and EventID != 5156 and EventID != 5447 and

      EventID != 4656 and EventID != 4658 and EventID != 4663 and EventID != 4660 and   <!-- The events here are silenced as they are known flooders  -->

      EventID != 4670 and EventID != 4690 and EventID != 4703 and EventID != 4907 and

      EventID != 5152 and EventID != 5157]</query>

  </localfile>

Anyway, the Wazuh anti-flooding mechanism should prevent large bursts of events on an agent from negatively impacting the network or the manager. 

So is there a way to test windows agent's log are all collected to wazuh manager without missing, so I can do a stress test, like is there a file I can tail in linux to see the collected events. So I will know when to added a additional wazuh server to collect.

Events that reach the manager but do not trigger a rule are dropped. To store all events even if they do not match a rule, you should enable the <logall> option in the /var/ossec/etc/ossec.conf manager configuration file. This events will be stored at  /var/ossec/logs/archives/archives.(json|log). In this file, you will see all the events coming from your Wazuh agents plus the events from the manager.

Inside the archives.log file, the source of the events can be identified through its header. For example, this is a Windows agent in my lab environment: 

2019 Nov 04 09:53:36 (W10) 192.168.75.141->EventChannel {"win":{"system":{"providerName":"Microsoft-Windows-Security-Auditing","providerGuid":"{54849625-5478-4994-a5ba-3e3b0328c30d}","eventID":"4689","version":"0","level":"0","task":"13313","opcode":"0","keywords":"0x8020000000000000","systemTime":"2019-11-04T08:53:35.919292100Z","eventRecordID":"2678","processID":"4","threadID":"1376","channel":"Security","computer":"DESKTOP-D33AN6I","severityValue":"AUDIT_SUCCESS","message":"Se salió de un proceso."},"eventdata":{"subjectUserSid":"S-1-5-21-2635242741-4024004514-10403589-1001","subjectUserName":"Zenidd","subjectDomainName":"DESKTOP-D33AN6I","subjectLogonId":"0x194a5","status":"0x1","processId":"0x15cc","processName":"C:\\Windows\\System32\\backgroundTaskHost.exe"}}}

On the other hand, the statistical file for ossec-agentd is located at C:\Program Files (x86)\ossec-agent\ossec-agent.state. This file provides information about the agent as the number of generated events, last connection, agent status, and some other information you can find useful when doing a stress test.

For example:

# State file for ossec-agent

# Agent status:

# - pending:      waiting to get connected.

# - connected:    connection established with manager in the last 10 seconds.

# - disconnected: connection lost or no ACK received in the last 10 seconds.

status='connected'

# Last time a keepalive was sent

last_keepalive='2019-11-04 09:59:51'

# Last time a control message was received

last_ack='2019-11-04 09:59:51'

# Number of generated events

msg_count='6416'

# Number of messages (events + control messages) sent to the manager

msg_sent='6526'

Let me know if you need more information or guidance for your Wazuh environment. Greetings, 

JP Sáez

[Harvey Ryu]

Hi Juan,

I tried the <logall> function, it works, thank you, I'll fine tune my policy now.

[Juan Pablo Saez]

Hi again Harvey, 

I would like to point out that Archived logs are not automatically deleted by default. You can choose when to manually or automatically (e.g., cron job) delete logs according to your own requirements.

Let me know if you need further assistance. Regards, 

JP Sáez

[Harvey Ryu]

Hi Juan,

It's been a month since I asked about this question, I just found out that the agent side can do log filter,

https://documentation.wazuh.com/3.10/user-manual/reference/ossec-conf/localfile.html#query

like this:

<localfile>
  <location>Security</location>
  <log_format>eventchannel</log_format>

  <query>Event[System/EventID = 4624 and (EventData/Data[@Name='LogonType'] = 2 or EventData/Data[@Name='LogonType'] = 10)]</query>
</localfile>

I tried it and it's working.

And I also found this one:
https://documentation.wazuh.com/3.10/user-manual/capabilities/log-data-collection/how-to-collect-wlogs.html?highlight=querylist#filtering-events-from-windows-event-channel-with-queries

It directly uses windows filtering sytax:
<QueryList>
 <Query Id="0" Path="Security">
 <Select Path="Security">*[System[(EventID=4689)]] and (*[EventData[Data[@Name="ProcessName"]="c:\windows\system32\cmd.exe"]])</Select>
 </Query>
</QueryList>

If I put this into eventvwr.msc-->"Filter Current Log"-->"XML"-->"Edit query manually", it will apply successfully.

But when I put this to agent.conf:

 <localfile>
   <location>Security</location>
   <log_format>eventchannel</log_format>
   <query>

     <QueryList>
       <Query Id="0" Path="Security">
         <Select Path="Security">*[System[(EventID=4689)]] and (*[EventData[Data[@Name="ProcessName"]="c:\windows\system32\cmd.exe"]])</Select>
       </Query>
     </QueryList>
   </query>
 </localfile>

My rule is like this:
<rule id="100011" level="14">
 <if_sid>60103</if_sid>
 <field name="win.system.eventID">^4689$</field>
 <description>test4689</description>
</rule>

No alert will generated when I created new events, I don't know why.

[Harvey Ryu]

Hi JP,

After an hour from the previous post, I noticed that there is a option called:only-future-events

https://documentation.wazuh.com/3.10/user-manual/reference/ossec-conf/localfile.html?highlight=localfile#only-future-events

If I changed it to no, it'll collect log since last agent stop time, right?

So if I wrote a script or something, when the wazuh-agent can't communicate with the wazuh-server some times and some duration, like a laptop not in office, or the network cable is unplugged, the script stops wazuh-agent, next time the connection is good, it'll send logs since the agent stopped, right?

[Juan Pablo Saez]

Hello again Harvey,

No alert will generated when I created new events, I don't know why.

Your snippet is almost valid but there are characters that should be escaped:

 <localfile>
   <location>Security</location>
   <log_format>eventchannel</log_format>
   <query>
     \<QueryList>
       \<Query Id="0" Path="Security">
         \<Select Path="Security">*[System[(EventID=4689)]] and (*[EventData[Data[@Name="ProcessName"]="c:\\windows\\system32\\cmd.exe"]])\</Select>
       \</Query>
     \</QueryList>
   </query>
 </localfile>

After escaping the characters above and using your rule, I can see the related alert:

** Alert 1575365357.111750: mail  - local,syslog,sshd,
2019 Dec 03 10:29:17 (w10) 192.168.75.141->EventChannel
Rule: 100011 (level 14) -> 'test4689'
{"win":{"system":{"providerName":"Microsoft-Windows-Security-Auditing","providerGuid":"{54849625-5478-4994-a5ba-3e3b0328c30d}","eventID":"4689","version":"0","level":"0","task":"13313","opcode":"0","keywords":"0x8020000000000000","systemTime":"2019-12-03T09:29:17.067544800Z","eventRecordID":"47432","processID":"4","threadID":"96","channel":"Security","computer":"DESKTOP-D33AN6I","severityValue":"AUDIT_SUCCESS","message":"Se salió de un proceso."},"eventdata":{"subjectUserSid":"S-1-5-21-2635242741-4024004514-10403589-1001","subjectUserName":"Zenidd","subjectDomainName":"DESKTOP-D33AN6I","subjectLogonId":"0x6e03e","status":"0xc000013a","processId":"0x7e8","processName":"C:\\Windows\\System32\\cmd.exe"}}}
win.system.providerName: Microsoft-Windows-Security-Auditing
win.system.providerGuid: {54849625-5478-4994-a5ba-3e3b0328c30d}
win.system.eventID: 4689
win.system.version: 0
win.system.level: 0
win.system.task: 13313
win.system.opcode: 0
win.system.keywords: 0x8020000000000000
win.system.systemTime: 2019-12-03T09:29:17.067544800Z
win.system.eventRecordID: 47432
win.system.processID: 4
win.system.threadID: 96
win.system.channel: Security
win.system.computer: DESKTOP-D33AN6I
win.system.severityValue: AUDIT_SUCCESS
win.system.message: Se salió de un proceso.
win.eventdata.subjectUserSid: S-1-5-21-2635242741-4024004514-10403589-1001
win.eventdata.subjectUserName: Zenidd
win.eventdata.subjectDomainName: DESKTOP-D33AN6I
win.eventdata.subjectLogonId: 0x6e03e
win.eventdata.status: 0xc000013a
win.eventdata.processId: 0x7e8
win.eventdata.processName: C:\Windows\System32\cmd.exe

If I changed it to no, it'll collect log since last agent stop time, right?
So if I wrote a script or something, when the wazuh-agent can't communicate with the wazuh-server some times and some duration, like a laptop not in office, or the network cable is unplugged, the script stops wazuh-agent, next time the connection is good, it'll send logs since the agent stopped, right?

When <only-future-events> is set to no, the agent will collect the events missed in the time it was off. If the <only-future-events> is set to yes, it will only collect the events that occur since the agent is active. On the other hand, Wazuh agents have a queue where the events wait when the agent, for any reason, can’t communicate with the server. By default, this queue can hold 5000 events.

I hope this information is useful for you Harvey. Let me know if you need anything else. Greetings,

JP Sáez

[Harvey Ryu]

Hi JP,

Although I don't know why those places need to add back slash, but I do as your example, and it works!

Could you help me with this issue?

https://groups.google.com/forum/#!topic/wazuh/lczu_Sb_Xts