Not getting Windows custom log into Wazuh

[Michael Fdez M]

Hi,

I have a windows app with logs like this:

***

[2024-04-12 05:00:24.260]: -------------------------------------------------------
[2024-04-12 05:00:24.261]: BACKUP - Replication - v8.5.0  Job Name: Backup 'COMPANY_MSSQL-ARS'
[2024-04-12 05:00:24.262]: Selected Node: COMPANY_MSSQL-ARS
[2024-04-12 05:00:24.262]: Selected Options: Update notes with the latest backup results. | Enable Active Block Mapping™ (ABM). | Enable Change Block Tracking (CBT).  SpaceSavingTech: Incremental
[2024-04-12 05:00:24.263]: -------------------------------------------------------
[2024-04-12 05:00:24.264]: SystemTime-4/12/2024 5:00:24 AM
[2024-04-12 05:00:24.264]: The job has started.
[2024-04-12 05:00:24.382]: Retrieving inventory node for the job...
[2024-04-12 05:00:24.444]: The job has been saved.
[2024-04-12 05:00:24.447]: Backup task 06fd26d4-5aab-478e-af42-6762b94d8ebb for virtual machine COMPANY_MSSQL-ARS-564d1ac1-a4fd-987e-94a7-5a74267eb268 was created.
[2024-04-12 05:00:24.448]: Backup task 06fd26d4-5aab-478e-af42-6762b94d8ebb for virtual machine COMPANY_MSSQL-ARS-564d1ac1-a4fd-987e-94a7-5a74267eb268 was queued.
[2024-04-12 05:00:24.449]: Completed retrieving virtual machines and creating their tasks.
[2024-04-12 05:00:24.449]: Total tasks created: 1
[2024-04-12 05:00:24.453]: The job has been saved.
[2024-04-12 05:02:48.838]: The job Backup 'COMPANY_MSSQL-ARS' has completed with a status of Success
[2024-04-12 05:02:48.839]: All tasks have completed for the job.
[2024-04-12 05:02:48.842]: The job has been saved.
{{WHITE SPACE}}

***

The file has: 18 Lines (the latest one a empty).

So, in my agent config file i set:

 <localfile>
  <location>C:\Program Files\XXXX\XYZ\Logs\User\Jobs\Backup\backup*.log</location>
  <log_format>multi-line:18</log_format>
 </localfile>

However, i'm not getting the logs or alerts within Wazuh. i also enabled the logall and the archives file is not getting entries there....

The agent logs:

***

024/04/16 05:21:27 wazuh-agent: INFO: (6008): File integrity monitoring scan started.
2024/04/16 05:21:38 wazuh-agent: INFO: (6009): File integrity monitoring scan ended.
2024/04/16 05:22:49 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 05:22:56 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/04/16 06:00:48 wazuh-agent: INFO: (1957): New file that matches the 'C:\Program Files\ XXXX\XYZ \Logs\User\Jobs\Backup\backup*.log' pattern: 'C:\Program Files\ XXXX\XYZ \Logs\User\Jobs\Backup\Backup 'COMPANY_FS_USUARIOS'_20240416T060010_fc1e44a8-c9db-440f-b2ea-e55f2d18f44e.log'.
2024/04/16 06:00:48 wazuh-agent: INFO: (1957): New file that matches the 'C:\Program Files\ XXXX\XYZ \Logs\User\Jobs\Backup\backup*.log' pattern: 'C:\Program Files\ XXXX\XYZ \Logs\User\Jobs\Backup\Backup 'COMPANY_NGINX'_20240416T060010_157ef5fd-472f-40e2-ac12-a36ef1593bb4.log'.
2024/04/16 06:22:56 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 06:23:03 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/04/16 07:23:04 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 07:23:11 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/04/16 08:23:12 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 08:23:19 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/04/16 09:23:20 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 09:23:27 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/04/16 10:23:28 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 10:23:36 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/04/16 11:23:37 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 11:23:44 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/04/16 12:23:45 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/04/16 12:23:52 wazuh-modulesd:syscollector: INFO: Evaluation finished.

***

Any idea?

Thanks in advance.

Michael.

[Alejandro Ruiz Becerra]

Hello Michael

There are a few causes why this could be happening.

- Are the logs always 18 lines long? If number of lines is dynamic, you should use multi-line-regex.
- Did you restart the manager to enable the `logall` setting? Changes to the configuration file require a restart.
- If so, did you check the `/var/ossec/logs/archives/archives.log` file in the `wazuh-manager` node.

In case you enabled `logall_json`, you should check the `/var/ossec/logs/archives/archives.json` instead.

[Michael Fdez M]

Hi Alejandro,

Thank you for your ideas.

Here's are the answers:

Are the logs always 18 lines long? If the number of lines is dynamic, you should use multi-line-regex.

Michael -> Yes, all the files with 18 lines.
- Did you restart the manager to enable the `logall` setting? Changes to the configuration file require a restart. You can

Michael -> Yes.

- If so, did you check the `/var/ossec/logs/archives/archives.log` file in the `wazuh-manager` node.

Michael -> And there's nothing there regarding the custom logs.

In case you enabled `logall_json`, you should check the `/var/ossec/logs/archives/archives.json` instead.

What else can I check?.

A few more info: Wazuh appliance: V. 4.7.3-

Thank you.

--
You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d592a893-a5a6-4296-9eba-112155ca5934n%40googlegroups.com.

[Alejandro Ruiz Becerra]

Can I take a look at `/var/ossec/logs/archives/archives.log` and `/var/ossec/logs/ossec.log` ??

[Michael Fdez M]

Hi Alejandro,

What do you suggest? where can i upload those files?.

Regards,

[Alejandro Ruiz Becerra]

As far as you don't share sensitive data, you can paste part of it here, as you did with the agent's logs above. We'd be interested on seeing the logs matching the date and time of the file creation. For example:

2024/04/16 06:00:48 wazuh-agent: INFO: (1957): New file that matches the 'C:\Program Files\ XXXX\XYZ \Logs\User\Jobs\Backup\backup*.log' pattern: 'C:\Program Files\ XXXX\XYZ \Logs\User\Jobs\Backup\Backup 'COMPANY_FS_USUARIOS'_20240416T060010_fc1e44a8-c9db-440f-b2ea-e55f2d18f44e.log'.

Manager logs around that date would be useful.

[Michael Fdez M]

Hi Alejandro,

Here are the logs regarding the agent:

2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_LAB_AWX'_20240417T210027_766faf64-2668-4f57-a659-c4d3b5a5cc5b.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MANTISPROD'_20240415T040058_a0dbe711-db11-4268-bf84-71f525d5dff1.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MANTISPROD'_20240416T040009_5cdd2b91-0776-4820-9984-c6a5dd2cfdd0.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MANTISPROD'_20240417T040019_427942ac-8215-44f5-a14c-c01255bdbdd8.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MANTISPROD'_20240418T040030_708c8813-c6fe-4680-a4dc-d47900e2d5b6.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MSSQL-ARS'_20240415T050058_2abbb38b-0b86-453d-ad67-ef2641f5edac.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MSSQL-ARS'_20240416T050009_59e3421d-2083-4370-a3e1-d37b250f03fe.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MSSQL-ARS'_20240417T050020_4703fbf3-a630-46c4-8a29-4a6e356d53aa.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_MSSQL-ARS'_20240418T050031_f578062d-8c5a-4d86-9a51-c5c474bb61a1.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_NGINX'_20240415T060059_f8be4c7a-8296-463a-b8d7-9b5eb5b0d0bf.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_NGINX'_20240416T060010_157ef5fd-472f-40e2-ac12-a36ef1593bb4.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_NGINX'_20240417T060020_0ed6cf7c-4db5-41cc-a237-d61c281af6d3.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'COMPANY_NGINX'_20240418T060031_0b0177ee-3b31-4e8b-ab41-416730f5b1f3.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBSYS_AD_LAB'_20240415T162700_9975b7a8-5254-4861-9f9d-abbc3de5e79b.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBSYS_AD_LAB'_20240417T210027_7cfe9c6a-36cd-4a4c-9d14-d8acb05f6c6b.log'.
2024/04/18 09:46:09 sca: INFO: Starting evaluation of policy: 'C:\Program Files (x86)\ossec-agent\ruleset\sca\cis_win2022.yml'
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBS_Desarrollo_Gyco'_20240415T200006_ec328a24-15b2-4197-a120-75854948e7bc.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBS_Desarrollo_Gyco'_20240416T200016_8dc0f665-28d7-4683-89a2-a48c7fa74dd2.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBS_Desarrollo_Gyco'_20240417T200026_efa152b8-678f-4629-8a69-2eba19dcf5b1.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBS_NGINX_INTERNO'_20240415T200006_530f9d1c-935f-4432-a75a-1ad102eff9e5.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBS_NGINX_INTERNO'_20240416T200016_67ea1a07-5509-4da2-96d4-91d5bfa9dfc8.log'.
2024/04/18 09:46:09 wazuh-agent: INFO: (1950): Analyzing file: 'C:\Program Files\XXXXX\YYYYYY\Logs\User\Jobs\Backup\Backup 'DBS_NGINX_INTERNO'_20240417T200026_d03b0283-e1c7-4cc8-8fbb-d066c35f7b64.log'.

The logs for archive (with logall set to yes), shows nothing.. just the typical  info related to Microsoft-Windows-Security-Auditing and stuff like that.

The logs ossec.log:

2024/04/18 09:44:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '013' vulnerabilities.
2024/04/18 09:44:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '013'
2024/04/18 09:44:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '014' vulnerabilities.
2024/04/18 09:44:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '014'
2024/04/18 09:44:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '015' vulnerabilities.
2024/04/18 09:44:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '015'
2024/04/18 09:44:56 wazuh-modulesd:vulnerability-detector: INFO: (5472): Vulnerability scan finished.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5431): Starting vulnerability scan.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: WARNING: (5575): Unavailable vulnerability data for the agent '000' OS. Skipping it.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '002' vulnerabilities.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '002'
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '004' vulnerabilities.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '004'
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '005' vulnerabilities.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '005'
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '006' vulnerabilities.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '006'
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '007' vulnerabilities.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '007'
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '008' vulnerabilities.
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5471): Finished vulnerability assessment for agent '008'
2024/04/18 09:49:56 wazuh-modulesd:vulnerability-detector: INFO: (5450): Analyzing agent '009' vulnerabilities.

Note: I tried to write the log to event logs (microsoft) and I see that the agent capture those logs and send them to the manager,. I do not see on the dashboards.. maybe this is the path...  what do i have to do in order to get them in the dashboard?.. maybe is easiest try this way?

Regards,

[Alejandro Ruiz Becerra]

Good morning Michael

I don't see anything notable in these logs. I will ask the team about this.

In the meantime, can you share your custom rule and decoder to parse your custom Windows app logs?

[Alejandro Ruiz Becerra]

In reply to your previous question. The alerts are not sent to the indexer (in other words, shown in the dashboard) unless it generates an alert.

About this:

I see that the agent capture those logs and send them to the manager

Do you mean that you see the logs in the alerts.json file? Where are you seeing that log?

[Alejandro Ruiz Becerra]

Hello again Michael!

Got some news. Enable the debug mode on log-collector and check the agents logs. We should be able to see what's going on.

[Alejandro Ruiz Becerra]

Allow me to extend my previous message.

The easiest and preferred method to enable debug mode to log-collector is to add logcollector.debug=2 to /var/ossec/etc/local_internal_options.conf.

A possible cause is that there is a new empty line at the end of the log files, causing logcollector to not detect lines properly.