How to view logs stored locally on the Wazuh server via the Wazuh dashboard?

[A Bobrov]

Good afternoon, gentlemen.
Please tell me about this issue.
We configured sending logs from CheckPoint to the Wazuh server.
The log on the server grows regularly, the data gets into it perfectly

serv2:# ls -lh /var/log/172.16.5.111.log
-rw-rw-rw- 1 syslog adm 12M Dec  4 09:24 /var/log/172.16.5.111.log

How can this local log be viewed and analyzed through the Wazuh dashboard? 

Thank you!!!

[A Bobrov]

addition,
on the local server, CISCO switches are specified in the Rsync settings
You can view logs for these devices in Wazuh dashboard.
№cisco
if $fromhost-ip startswith '10.14.3.103' then /var/log/10.14.3.103.log
& ~
№cisco
if $fromhost-ip startswith '10.9.0.1' then /var/log/10.9.0.1.log
& ~
№CheckPoint
if $fromhost-ip startswith '172.16.5.111' then /var/log/172.16.5.111.log
& ~

File permissions are almost the same

serv2:~# ls -lh /var/log/
total 514M
-rw-r--r--  1 syslog        adm             228K Dec  4 11:17 10.14.3.103.log
-rw-r--r--  1 syslog        adm             1.4M Dec  4 13:39 10.9.0.1.log
-rw-rw-rw-  1 syslog        adm              12M Dec  4 09:24 172.16.5.111.log

среда, 4 декабря 2024 г. в 09:55:45 UTC+3, A Bobrov:

[Md. Nazmur Sakib]

Hi A Bobrov,

Are these log files in the Wazuh server?

If yes, you can use localfile to forward these logs to Wazuh.


Add this configuration under <ossec_config>

  <localfile>

    <log_format>syslog</log_format>

    <location>/var/log/10.14.3.103.log</location>

  </localfile>

  <localfile>

    <log_format>syslog</log_format>

    <location>var/log/10.9.0.1.log</location>

  </localfile>

<localfile>

 <log_format>syslog</log_format>

 <location>/var/log/172.16.5.111.log</location>

</localfile>

Ref: https://documentation.wazuh.com/current/user-manual/capabilities/log-data-collection/monitoring-log-files.html

If the log files are on a different server. Install a Wazuh agent on that server and add the same localfile configuration to the agent’s ossec.conf


Next, to see alerts from your logs, you need to write decoders and rules if your logs are not triggering alerts on the Dashabord.

Check this document to learn more about writing decoders and rules:
https://documentation.wazuh.com/current/user-manual/ruleset/index.html

Let me know if you need any further information.

[A Bobrov]

Good afternoon, Nazmur, thank you for your answer.
You are absolutely right, Nazmur, in pointing to the file. I forgot to specify the settings from this file.
Currently added to the ossec.conf file

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/10.14.3.103.log</location>

  <out_format>10.14.3.103.log: $(log)</out_format>

</localfile>

<localfile>
  <log_format>syslog</log_format>

  <location>/var/log/10.9.0.1.log</location>
  <out_format>10.9.0.1.log: $(log)</out_format>

</localfile>

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/172.16.5.111.log</location>

  <out_format>172.16.5.111.log: $(log)</out_format>
</localfile>

CISCO 10.14.3.103 and 10.9.0.1 equipment are displayed in Dashabord,but CheckPoint 172.16.5.111 is not.
I'll take a look at the links you provided and let you know.
So the question arose, how, with essentially the same parameters, 2 devices I see, but one thing I don’t.

Thank you, Nazmur for your quick response!!!!

среда, 4 декабря 2024 г. в 12:48:19 UTC+3, Md. Nazmur Sakib:

[Md. Nazmur Sakib]

Restart the Wazuh manager and share the output of this command from your agent endpoint. 

cat /var/ossec/logs/ossec.log | grep -iE "wazuh-logcollector" 

With this command, we can verify if the files are monitored or not. 

If you see a log like this
2024/12/03 11:39:28 wazuh-Logcollector: INFO: (1950): Analyzing file: '/var/log/logfile" 

You might need to write decoders and rules to trigger alerts on the Dashboard. Check this document to learn more about writing decoders and rules: https://documentation.wazuh.com/current/user-manual/ruleset/index.html

Also, keep in mind that only new logs that are added to the log file will forwarded to Wazuh.

Let me know the update on the issue.

[A Bobrov]

Good afternoon, Nazmur!!!
Thank you for your answers and the time you find for this.
The end point is CheckPoint, then it dumps the logs to the Wazuh server

With the help of your links I moved forward a little. At these addresses, I found files for the decoder and rules
https://github.com/wazuh/wazuh/tree/master/ruleset/decoders
https://github.com/wazuh/wazuh/blob/master/ruleset/rules/0680-checkpoint-smart1_rules.xml

Created 2 files
/var/ossec/etc/decoders/172.16.5.111.xml
/var/ossec/etc/rules/172.16.5.111.xml

In them, for example, I placed data from files
0051-checkpoint-smart1_decoders.xml
0680-checkpoint-smart1_rules.xml
Along the way, I would like to say a huge thank you to the person who did such a great job creating these files!!!

I check with the utility
SStarting wazuh-logtest v4.9.1
Type one log per line

checkpoint-smart1

** Wazuh-Logtest: WARNING: (7612): Rule ID '64220' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64221' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64222' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64223' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64224' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64225' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64226' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64227' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64228' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64229' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64230' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64231' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64232' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64233' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64234' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64235' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64236' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64237' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64238' is duplicated. Only the first occurrence will be considered.
** Wazuh-Logtest: WARNING: (7612): Rule ID '64239' is duplicated. Only the first occurrence will be considered.

**Phase 1: Completed pre-decoding.
        full event: 'checkpoint-smart1'

**Phase 2: Completed decoding.
        No decoder matched.

No decoder matched! :((
By all laws, I should get some kind of conclusion.

And here are the questions:
1. In the main file, /var/ossec/etc/ossec.conf I will need to make changes

<!-- User-defined ruleset -->
  <decoder_dir>etc/decoders</decoder_dir>
  <rule_dir>etc/rules</rule_dir>
  <decoder_exclude>/var/ossec/etc/rules/172.16.5.111.xml</decoder_exclude>
</ruleset>

+ systemctl restart wazuh-manager

2. How will the new decoder and rule see the log from CheckPoint located at
/var/log/172.16.5.111.log ?
When I did not explicitly specify the path to the logs anywhere except /var/ossec/etc/ossec.conf

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/172.16.5.111.log</location>
  <out_format>172.16.5.111.log: $(log)</out_format>
</localfile>

I continue to study the manuals provided by you.
Thank you, Nazmur.

четверг, 5 декабря 2024 г. в 08:38:34 UTC+3, Md. Nazmur Sakib:

[A Bobrov]

Along the way, I found that decoders and rules are already available on the Wazuh desktop server

ls /var/ossec/ruleset/decoders/ | grep checkpoint
0050-checkpoint_decoders.xml
0051-checkpoint-smart1_decoders.xml

ls /var/ossec/ruleset/rules/ | grep check
0680-checkpoint-smart1_rules.xml

четверг, 5 декабря 2024 г. в 11:12:58 UTC+3, A Bobrov:

[A Bobrov]

the command output shows nothing
serv2:~# cat /var/ossec/logs/ossec.log | grep -iE "wazuh-logcollector"
serv2:~#

четверг, 5 декабря 2024 г. в 11:28:57 UTC+3, A Bobrov:

[A Bobrov]

Good afternoon, Nazmur!!!

The current situation is as follows:
1. the log from CheckPoint is transferred to the Wazuh server

serv2:~# ls -lh /var/log/172.16.5.111.log
-rw-rw-rw- 1 syslog adm 18M Dec  5 15:06 /var/log/172.16.5.111.log

Dec  5 15:06:05 172.16.5.111 time="1733400361" action="Drop" ifdir="inbound" ifname="eth0"
logid="0" loguid="{0x6751972b,0x3,0x6f0510ac,0x17fb2300}" origin="172.16.5.11"
originsicname="CN=gwtest,O=TESTSMS..u2xvp5" sequencenum="2" time="1733400361"
version="5" dst="172.16.63.255" inzone="External" layer_name="Network" layer_uuid="8a994dd3-993e-4c0c-92a1-a8630b153f4c" match_id="3" parent_rule="0" rule_action="Drop" rule_name="Cleanup rule" rule_uid="60e2929c-4371-402b-bf57-3f2efca62cad"
outzone="Local" product="VPN-1 & FireWall-1" proto="241" src="172.16.2.52" log_link="https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzMzOTgxMjBAQ0AyODM5Jm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D"

2. Files for decoding and rules are made. But, most likely, they may not be needed,
since these files are already on the system? How then should the log file be processed?

3. Here’s how to view the log file /var/log/172.16.5.111.log in the Wazuh dashboard, which is located on
on the same server?

четверг, 5 декабря 2024 г. в 12:50:11 UTC+3, A Bobrov:

[A Bobrov]

Good afternoon, Nazmur!

Following your recommendations, I checked the moment,
where you can see that wazuh-logcollector is monitoring log files.

serv2:~# cat /var/ossec/logs/ossec.log | grep -iE "wazuh-logcollector"

2024/12/06 08:49:28 wazuh-logcollector: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/12/06 08:49:41 wazuh-logcollector: INFO: Monitoring output of command(360): df -P
2024/12/06 08:49:41 wazuh-logcollector: INFO: Monitoring full output of command(360): netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d
2024/12/06 08:49:41 wazuh-logcollector: INFO: Monitoring full output of command(360): last -n 20
2024/12/06 08:49:41 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/ossec/logs/active-responses.log'.
2024/12/06 08:49:41 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/dpkg.log'.
2024/12/06 08:49:41 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/10.16.0.100.log'.
2024/12/06 08:49:41 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/10.14.3.103.log'.
2024/12/06 08:49:41 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/10.9.0.1.log'.
2024/12/06 08:49:41 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/172.16.5.111.log'.
2024/12/06 08:49:41 wazuh-logcollector: INFO: Started (pid: 1394030).
2024/12/06 08:49:44 wazuh-logcollector: INFO: (9203): Monitoring journal entries.

Maybe I'm making the wrong request in the Wazuh dashboard?
I'm looking for ip address 172.16.5.111.

четверг, 5 декабря 2024 г. в 17:33:17 UTC+3, A Bobrov:

Good afternoon, Nazmur!!!
We were able to obtain the data you requested

serv2:~# cat /var/ossec/logs/ossec.log | grep -iE "wazuh-logcollector"

2024/12/05 17:00:49 wazuh-logcollector: INFO: (1225): SIGNAL [(15)-(Terminated)] Received. Exit Cleaning...
2024/12/05 17:01:01 wazuh-logcollector: INFO: Monitoring output of command(360): df -P
2024/12/05 17:01:01 wazuh-logcollector: INFO: Monitoring full output of command(360): netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d
2024/12/05 17:01:01 wazuh-logcollector: INFO: Monitoring full output of command(360): last -n 20
2024/12/05 17:01:01 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/ossec/logs/active-responses.log'.
2024/12/05 17:01:01 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/dpkg.log'.
2024/12/05 17:01:01 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/10.16.0.100.log'.
2024/12/05 17:01:01 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/10.14.3.103.log'.
2024/12/05 17:01:01 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/10.9.0.1.log'.
2024/12/05 17:01:01 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/172.16.5.111.log'.
2024/12/05 17:01:01 wazuh-logcollector: INFO: Started (pid: 3397875).
2024/12/05 17:01:04 wazuh-logcollector: INFO: (9203): Monitoring journal entries.

CheckPoint <-----> 172.16.5.111
Cisco 10.16.0.100,10.14.3.103,10.9.0.1

can you tell me what request should be made in the Wazuh dashboard,
for CheckPoint, by ip 172.16.5.111 - we don’t find it.  :((

четверг, 5 декабря 2024 г. в 15:41:56 UTC+3, A Bobrov:

[Md. Nazmur Sakib]

The log you have shared does not match with any rules.

Can you enable archives.json log and share some sample logs?

For this, You can try the following steps:

For this, you can enable archive JSON format log from your manager's ossec.conf

<ossec_config>

  <global>

___________________

    <logall_json>yes</logall_json>

_______________

After making the changes make sure to restart the manager.

And share the output of this command

cat /var/ossec/logs/archives/archives.json | grep /var/log/172.16.5.111.log

Also, share any custom decoders and rules you have written for this log.

Looking forward to your update on the issue.

[A Bobrov]

Nazhmur, а как и с помощю чего Вы получили такой вывод лога? 

пятница, 6 декабря 2024 г. в 09:49:41 UTC+3, Md. Nazmur Sakib:

[A Bobrov]

Good afternoon, Nazmur!

I do it step by step

<logall_json>yes</logall_json>

restart wazuh-manager

command output

@serv2:~# cat /var/ossec/logs/archives/archives.json | grep /var/log/172.16.5.111.log

{"timestamp":"2024-12-06T08:32:50.177+0000","agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733473970.1710617566","full_log":"172.16.5.111.log: Dec  6 11:32:48 172.16.5.111 time=\"1733473963\" action=\"Drop\" ifdir=\"inbound\" ifname=\"eth0\" logid=\"0\" loguid=\"{0x6752b6af,0x0,0x6f0510ac,0x1857a371}\" origin=\"172.16.5.11\" originsicname=\"CN=gwtest,O=TESTSMS..u2xvp5\" sequencenum=\"1\" time=\"1733473963\" version=\"5\" dst=\"172.16.63.255\" inzone=\"External\" layer_name=\"Network\" layer_uuid=\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\" match_id=\"3\" parent_rule=\"0\" rule_action=\"Drop\" rule_name=\"Cleanup rule\" rule_uid=\"60e2929c-4371-402b-bf57-3f2efca62cad\" outzone=\"Local\" product=\"VPN-1 & FireWall-1\" proto=\"17\" s_port=\"1230\" service=\"123\" service_id=\"ntp-udp\" src=\"172.16.1.139\" log_link=\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM0NjM5MzRAQ0A4MTIxJm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D\"","decoder":{},"location":"/var/log/172.16.5.111.log"}
{"timestamp":"2024-12-06T08:32:50.177+0000","agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733473970.1710617566","full_log":"172.16.5.111.log: Dec  6 11:32:48 172.16.5.111 time=\"1733473963\" action=\"Drop\" ifdir=\"inbound\" ifname=\"eth0\" logid=\"0\" loguid=\"{0x6752b6af,0x1,0x6f0510ac,0x1857a371}\" origin=\"172.16.5.11\" originsicname=\"CN=gwtest,O=TESTSMS..u2xvp5\" sequencenum=\"2\" time=\"1733473963\" version=\"5\" dst=\"172.16.63.255\" inzone=\"External\" layer_name=\"Network\" layer_uuid=\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\" match_id=\"3\" parent_rule=\"0\" rule_action=\"Drop\" rule_name=\"Cleanup rule\" rule_uid=\"60e2929c-4371-402b-bf57-3f2efca62cad\" outzone=\"Local\" product=\"VPN-1 & FireWall-1\" proto=\"17\" s_port=\"137\" service=\"137\" service_id=\"nbname\" src=\"172.16.2.192\" log_link=\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM0NjM5MzRAQ0A4MTIyJm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D\"","decoder":{},"location":"/var/log/172.16.5.111.log"}
{"timestamp":"2024-12-06T08:32:54.178+0000","agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733473974.1711363607","full_log":"172.16.5.111.log: Dec  6 11:32:52 172.16.5.111 time=\"1733473967\" action=\"Drop\" ifdir=\"inbound\" ifname=\"eth0\" logid=\"0\" loguid=\"{0x6752b6b3,0x0,0x6f0510ac,0x1857a371}\" origin=\"172.16.5.11\" originsicname=\"CN=gwtest,O=TESTSMS..u2xvp5\" sequencenum=\"1\" time=\"1733473967\" version=\"5\" dst=\"172.16.63.255\" inzone=\"External\" layer_name=\"Network\" layer_uuid=\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\" match_id=\"3\" parent_rule=\"0\" rule_action=\"Drop\" rule_name=\"Cleanup rule\" rule_uid=\"60e2929c-4371-402b-bf57-3f2efca62cad\" outzone=\"Local\" product=\"VPN-1 & FireWall-1\" proto=\"17\" s_port=\"57055\" service=\"1947\" src=\"172.16.2.156\" log_link=\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM0NjM5MzRAQ0A4MTIzJm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D\"","decoder":{},"location":"/var/log/172.16.5.111.log"}

There was a problem with the decoder. Most likely I did it wrong.
I took as a basis the files that are already in the system

Created 2 files
/var/ossec/etc/decoders/172.16.5.111.xml
/var/ossec/etc/rules/172.16.5.111.xml

In them, for example, I placed data from files
0051-checkpoint-smart1_decoders.xml
0680-checkpoint-smart1_rules.xml

I'm a little confused at this point. :(

Thank you, Nazmur, for trying to help understand such a powerful system!

пятница, 6 декабря 2024 г. в 09:49:41 UTC+3, Md. Nazmur Sakib:

The log you have shared does not match with any rules.

[A Bobrov]

Dear, Nazmur
this is what I got, thanks to your advice and recommendations.

Starting wazuh-logtest v4.9.1
Type one log per line

{"timestamp":"2024-12-06T11:22:50.228+0000","agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733484170.2831507349","full_log":"172.16.5.111.log: Dec  6 14:22:50 172.16.5.111 time=\"1733484164\" action=\"Drop\" ifdir=\"inbound\" ifname=\"eth0\" logid=\"0\" loguid=\"{0x6752de88,0x1,0x6f0510ac,0x1857a371}\" origin=\"172.16.5.11\" originsicname=\"CN=gwtest,O=TESTSMS..u2xvp5\" sequencenum=\"1\" time=\"1733484164\" version=\"5\" dst=\"172.16.63.255\" inzone=\"External\" layer_name=\"Network\" layer_uuid=\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\" match_id=\"3\" parent_rule=\"0\" rule_action=\"Drop\" rule_name=\"Cleanup rule\" rule_uid=\"60e2929c-4371-402b-bf57-3f2efca62cad\" outzone=\"Local\" product=\"VPN-1 & FireWall-1\" proto=\"17\" s_port=\"137\" service=\"137\" service_id=\"nbname\" src=\"172.16.1.38\" log_link=\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM0NjM5MzRAQ0AyMTAxNSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi\"","decoder":{},"location":"/var/log/172.16.5.111.log"}

**Phase 1: Completed pre-decoding.
        full event: '{"timestamp":"2024-12-06T11:22:50.228+0000","agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733484170.2831507349","full_log":"172.16.5.111.log: Dec  6 14:22:50 172.16.5.111 time=\"1733484164\" action=\"Drop\" ifdir=\"inbound\" ifname=\"eth0\" logid=\"0\" loguid=\"{0x6752de88,0x1,0x6f0510ac,0x1857a371}\" origin=\"172.16.5.11\" originsicname=\"CN=gwtest,O=TESTSMS..u2xvp5\" sequencenum=\"1\" time=\"1733484164\" version=\"5\" dst=\"172.16.63.255\" inzone=\"External\" layer_name=\"Network\" layer_uuid=\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\" match_id=\"3\" parent_rule=\"0\" rule_action=\"Drop\" rule_name=\"Cleanup rule\" rule_uid=\"60e2929c-4371-402b-bf57-3f2efca62cad\" outzone=\"Local\" product=\"VPN-1 & FireWall-1\" proto=\"17\" s_port=\"137\" service=\"137\" service_id=\"nbname\" src=\"172.16.1.38\" log_link=\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM0NjM5MzRAQ0AyMTAxNSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi\"","decoder":{},"location":"/var/log/172.16.5.111.log"}'

**Phase 2: Completed decoding.
        name: 'json'
        agent.id: '000'
        agent.name: 'serv2'
        full_log: '172.16.5.111.log: Dec  6 14:22:50 172.16.5.111 time="1733484164" action="Drop" ifdir="inbound" ifname="eth0" logid="0" loguid="{0x6752de88,0x1,0x6f0510ac,0x1857a371}" origin="172.16.5.11" originsicname="CN=gwtest,O=TESTSMS..u2xvp5" sequencenum="1" time="1733484164" version="5" dst="172.16.63.255" inzone="External" layer_name="Network" layer_uuid="8a994dd3-993e-4c0c-92a1-a8630b153f4c" match_id="3" parent_rule="0" rule_action="Drop" rule_name="Cleanup rule" rule_uid="60e2929c-4371-402b-bf57-3f2efca62cad" outzone="Local" product="VPN-1 & FireWall-1" proto="17" s_port="137" service="137" service_id="nbname" src="172.16.1.38" log_link="https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM0NjM5MzRAQ0AyMTAxNSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi"'
        id: '1733484170.2831507349'
        location: '/var/log/172.16.5.111.log'
        manager.name: 'serv2'
        timestamp: '2024-12-06T11:22:50.228+0000'


But so far I haven’t found an event with id id: '1733484170.2831507349'
Most likely I didn't take something into account again.

пятница, 6 декабря 2024 г. в 13:48:48 UTC+3, A Bobrov:

[Md. Nazmur Sakib]

You need to test the log inside the full_log and write decoders and rules based on that.

172.16.5.111.log: Dec  6 14:22:50 172.16.5.111 time="1733484164" action="Drop" ifdir="inbound" ifname="eth0" logid="0" loguid="{0x6752de88,0x1,0x6f0510ac,0x1857a371}" origin="172.16.5.11" originsicname="CN=gwtest,O=TESTSMS..u2xvp5" sequencenum="1" time="1733484164" version="5" dst="172.16.63.255" inzone="External" layer_name="Network" layer_uuid="8a994dd3-993e-4c0c-92a1-a8630b153f4c" match_id="3" parent_rule="0" rule_action="Drop" rule_name="Cleanup rule" rule_uid="60e2929c-4371-402b-bf57-3f2efca62cad" outzone="Local" product="VPN-1 & FireWall-1" proto="17" s_port="137" service="137" service_id="nbname" src="172.16.1.38" log_link="https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM0NjM5MzRAQ0AyMTAxNSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi"

Bason on your JSON log, I can see it is not tripped by any decoder

You need to write decoders and rules for this rule.


I am sharing some sample decoders and rules

Decoders:

<decoder name="network_device">

  <prematch>^172.16.5.111.log: </prematch>

</decoder>

<decoder name="network_device_child">

  <parent>network_device</parent>

  <regex >\S+ (\w+\s*\d+\s*\d+:\d+:\d+) (\.+) \.+action="(\.+)"</regex>

  <order>logtimestamp, ip_address, action</order>

</decoder>

Rules:

<group name="network_device_rule,">

  <rule id="110000" level="3">

    <decoded_as>network_device</decoded_as>

    <description>Mikrotik-Event</description>

  </rule>

</group>

You can follow these documents for writing decoders and rules: https://documentation.wazuh.com/current/user-manual/ruleset/index.html

I hope you find this useful.

[A Bobrov]

Nazmur,
and one more addition
after searching
cat /var/ossec/logs/archives/archives.json | grep /var/log/172.16.5.111.log

the system issues events. I run the event through the script, the decoder processes it

{"timestamp":"2024-12-09T08:42:20.338+0000","rule":{"level":3,"description":"Mikrotik-Event","id":"110000","firedtimes":535,"mail":false,"groups":["network_device_rule"]},"agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733733740.1691879646","full_log":"172.16.5.111.log: Dec  9 11:42:18 172.16.5.111  {\"h_version\":5,\"action\":\"Drop\",\"flags\":395524,\"ifdir\":\"inbound\",\"ifname\":\"eth0\",\"logid\":0,\"loguid\":\"0x6756ad67,0x1,0x6f0510ac,0x183fa2e9\",\"origin\":\"172.16.5.11\",\"originsicname\":\"CN=gwtest,O=TESTSMS..u2xvp5\",\"sequencenum\":1,\"time\":1733733732,\"__policy_id_tag\":\"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]\",\"dst\":\"172.16.63.255\",\"inzone\":\"External\",\"outzone\":\"Local\",\"product\":\"VPN-1 & FireWall-1\",\"proto\":17,\"s_port\":51817,\"service\":1947,\"src\":\"172.16.1.67\",\"log_link\":\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMTYyNiZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi\",\"parent_rule_._._match_table\":[0],\"rule_action_._._match_table\":[\"Drop\"],\"rule_name_._._match_table\":[\"Cleanup rule\"],\"rule_uid_._._match_table\":[\"60e2929c-4371-402b-bf57-3f2efca62cad\"],\"match_id_._._match_table\":[3],\"layer_name_._._match_table\":[\"Network\"],\"layer_uuid_._._match_table\":[\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\"]}","decoder":{"name":"network_device"},"location":"/var/log/172.16.5.111.log"}

**Phase 1: Completed pre-decoding.
        full event: '{"timestamp":"2024-12-09T08:42:20.338+0000","rule":{"level":3,"description":"Mikrotik-Event","id":"110000","firedtimes":535,"mail":false,"groups":["network_device_rule"]},"agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733733740.1691879646","full_log":"172.16.5.111.log: Dec  9 11:42:18 172.16.5.111  {\"h_version\":5,\"action\":\"Drop\",\"flags\":395524,\"ifdir\":\"inbound\",\"ifname\":\"eth0\",\"logid\":0,\"loguid\":\"0x6756ad67,0x1,0x6f0510ac,0x183fa2e9\",\"origin\":\"172.16.5.11\",\"originsicname\":\"CN=gwtest,O=TESTSMS..u2xvp5\",\"sequencenum\":1,\"time\":1733733732,\"__policy_id_tag\":\"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]\",\"dst\":\"172.16.63.255\",\"inzone\":\"External\",\"outzone\":\"Local\",\"product\":\"VPN-1 & FireWall-1\",\"proto\":17,\"s_port\":51817,\"service\":1947,\"src\":\"172.16.1.67\",\"log_link\":\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMTYyNiZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi\",\"parent_rule_._._match_table\":[0],\"rule_action_._._match_table\":[\"Drop\"],\"rule_name_._._match_table\":[\"Cleanup rule\"],\"rule_uid_._._match_table\":[\"60e2929c-4371-402b-bf57-3f2efca62cad\"],\"match_id_._._match_table\":[3],\"layer_name_._._match_table\":[\"Network\"],\"layer_uuid_._._match_table\":[\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\"]}","decoder":{"name":"network_device"},"location":"/var/log/172.16.5.111.log"}'

**Phase 2: Completed decoding.
        name: 'json'
        agent.id: '000'
        agent.name: 'serv2'

        decoder.name: 'network_device'
        full_log: '172.16.5.111.log: Dec  9 11:42:18 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756ad67,0x1,0x6f0510ac,0x183fa2e9","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733733732,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":51817,"service":1947,"src":"172.16.1.67","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMTYyNiZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'
        id: '1733733740.1691879646'
        location: '/var/log/172.16.5.111.log'
        manager.name: 'serv2'
        rule.description: 'Mikrotik-Event'
        rule.firedtimes: '535'
        rule.groups: '['network_device_rule']'
        rule.id: '110000'
        rule.level: '3'
        rule.mail: 'false'
        timestamp: '2024-12-09T08:42:20.338+0000'

=============================================================================

and one more addition
after searching
cat /var/ossec/logs/archives/archives.json | grep /var/log/172.16.5.111.log
the system issues events. I run the event through the script, the decoder processes it
{"timestamp":"2024-12-09T08:47:56.666+0000","rule":{"level":3,"description":"Mikrotik-Event","id":"110000","firedtimes":1073,"mail":false,"groups":["network_device_rule"]},"agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733734076.1722542111","full_log":"172.16.5.111.log: Dec  9 11:47:55 172.16.5.111  {\"h_version\":5,\"action\":\"Drop\",\"flags\":395524,\"ifdir\":\"inbound\",\"ifname\":\"eth0\",\"logid\":0,\"loguid\":\"0x6756aeb8,0x0,0x6f0510ac,0x183fa2e9\",\"origin\":\"172.16.5.11\",\"originsicname\":\"CN=gwtest,O=TESTSMS..u2xvp5\",\"sequencenum\":1,\"time\":1733734068,\"__policy_id_tag\":\"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]\",\"dst\":\"255.255.255.255\",\"inzone\":\"External\",\"outzone\":\"Local\",\"product\":\"VPN-1 & FireWall-1\",\"proto\":17,\"s_port\":60521,\"service\":1947,\"src\":\"172.16.1.54\",\"log_link\":\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMjE2NSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi\",\"parent_rule_._._match_table\":[0],\"rule_action_._._match_table\":[\"Drop\"],\"rule_name_._._match_table\":[\"Cleanup rule\"],\"rule_uid_._._match_table\":[\"60e2929c-4371-402b-bf57-3f2efca62cad\"],\"match_id_._._match_table\":[3],\"layer_name_._._match_table\":[\"Network\"],\"layer_uuid_._._match_table\":[\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\"]}","decoder":{"name":"network_device"},"location":"/var/log/172.16.5.111.log"}

**Phase 1: Completed pre-decoding.
        full event: '{"timestamp":"2024-12-09T08:47:56.666+0000","rule":{"level":3,"description":"Mikrotik-Event","id":"110000","firedtimes":1073,"mail":false,"groups":["network_device_rule"]},"agent":{"id":"000","name":"serv2"},"manager":{"name":"serv2"},"id":"1733734076.1722542111","full_log":"172.16.5.111.log: Dec  9 11:47:55 172.16.5.111  {\"h_version\":5,\"action\":\"Drop\",\"flags\":395524,\"ifdir\":\"inbound\",\"ifname\":\"eth0\",\"logid\":0,\"loguid\":\"0x6756aeb8,0x0,0x6f0510ac,0x183fa2e9\",\"origin\":\"172.16.5.11\",\"originsicname\":\"CN=gwtest,O=TESTSMS..u2xvp5\",\"sequencenum\":1,\"time\":1733734068,\"__policy_id_tag\":\"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]\",\"dst\":\"255.255.255.255\",\"inzone\":\"External\",\"outzone\":\"Local\",\"product\":\"VPN-1 & FireWall-1\",\"proto\":17,\"s_port\":60521,\"service\":1947,\"src\":\"172.16.1.54\",\"log_link\":\"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMjE2NSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi\",\"parent_rule_._._match_table\":[0],\"rule_action_._._match_table\":[\"Drop\"],\"rule_name_._._match_table\":[\"Cleanup rule\"],\"rule_uid_._._match_table\":[\"60e2929c-4371-402b-bf57-3f2efca62cad\"],\"match_id_._._match_table\":[3],\"layer_name_._._match_table\":[\"Network\"],\"layer_uuid_._._match_table\":[\"8a994dd3-993e-4c0c-92a1-a8630b153f4c\"]}","decoder":{"name":"network_device"},"location":"/var/log/172.16.5.111.log"}'

**Phase 2: Completed decoding.
        name: 'json'
        agent.id: '000'
        agent.name: 'serv2'

        decoder.name: 'network_device'
        full_log: '172.16.5.111.log: Dec  9 11:47:55 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756aeb8,0x0,0x6f0510ac,0x183fa2e9","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733734068,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]","dst":"255.255.255.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":60521,"service":1947,"src":"172.16.1.54","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMjE2NSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'
        id: '1733734076.1722542111'
        location: '/var/log/172.16.5.111.log'
        manager.name: 'serv2'
        rule.description: 'Mikrotik-Event'
        rule.firedtimes: '1073'
        rule.groups: '['network_device_rule']'
        rule.id: '110000'
        rule.level: '3'
        rule.mail: 'false'
        timestamp: '2024-12-09T08:47:56.666+0000'

So in summary:
1. the log from another machine is processed by the encoder;
2. the log from /var/ossec/logs/archives/archives.json is processed, but a little differently;
3. and local log from /var/log/172.16.5.111.log
the system does not process No decoder matched
Dec  9 11:50:31 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756af54,0x1,0x6f0510ac,0x183fa2e9","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733734226,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.2","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMjM0NSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

**Phase 1: Completed pre-decoding.
        full event: 'Dec  9 11:50:31 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756af54,0x1,0x6f0510ac,0x183fa2e9","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733734226,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.2","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AxMjM0NSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'
        timestamp: 'Dec  9 11:50:31'
        hostname: '172.16.5.111'

**Phase 2: Completed decoding.
        No decoder matched.

I have not added decoders and rules yet. I'm studying.

понедельник, 9 декабря 2024 г. в 10:21:14 UTC+3, Md. Nazmur Sakib:

[A Bobrov]

Good afternoon, Nazmur!

Please tell me about this issue.  When trying to process a log located
on the server in wazuh the script does not see the system in the second phase of the decoder

**Phase 2: Completed decoding.
        No decoder matched.

**Phase 1: Completed pre-decoding.
        full event: 'Dec  9 09:59:22 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x67569547,0x2,0x6f0510ac,0x183fa2e9","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733727556,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":59823,"service":1947,"src":"172.16.2.96","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AzNDE4Jm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'
        timestamp: 'Dec  9 09:59:22'
        hostname: '172.16.5.111'

**Phase 2: Completed decoding.
        No decoder matched.

But at the same time, if in the script on the server in wazuh you specify the log that is collected by another machine from the same CheckPoint, then

**Phase 2: Completed decoding.
        name: 'json'

**Phase 1: Completed pre-decoding.
        full event: 'Dec  9 10:14:20 172.16.5.111 {"h_version": 5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x675698cb,0x1,0x6f0510ac,0x184422f8","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733728456,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.52","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0A0NzEzJm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'
        timestamp: 'Dec  9 10:14:20'
        hostname: '172.16.5.111'

**Phase 2: Completed decoding.
        name: 'json'

        __policy_id_tag: 'product=VPN-1 & FireWall-1[db_tag={304DA366-CF59-F943-947A-7C63006D98E7};mgmt=TESTSMS;date=1733490766;policy_name=Standard]'
        action: 'Drop'
        dst: '172.16.63.255'
        flags: '395524'
        h_version: '5'
        ifdir: 'inbound'
        ifname: 'eth0'
        inzone: 'External'
        layer_name_._._match_table: '['Network']'
        layer_uuid_._._match_table: '['8a994dd3-993e-4c0c-92a1-a8630b153f4c']'
        log_link: 'https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0A0NzEzJm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D'
        logid: '0'
        loguid: '0x675698cb,0x1,0x6f0510ac,0x184422f8'
        match_id_._._match_table: '[3]'
        origin: '172.16.5.11'
        originsicname: 'CN=gwtest,O=TESTSMS..u2xvp5'
        outzone: 'Local'
        parent_rule_._._match_table: '[0]'
        product: 'VPN-1 & FireWall-1'
        proto: '17'
        rule_action_._._match_table: '['Drop']'
        rule_name_._._match_table: '['Cleanup rule']'
        rule_uid_._._match_table: '['60e2929c-4371-402b-bf57-3f2efca62cad']'
        s_port: '137'
        sequencenum: '2'
        service: '137'
        service_id: 'nbname'
        src: '172.16.2.52'
        time: '1733728456'


How can this be, the local log decoder does not see, but the same log from another machine sees?
Thank you!

понедельник, 9 декабря 2024 г. в 07:49:20 UTC+3, A Bobrov:

[Md. Nazmur Sakib]

One log has the next line before 5 the log and one doesn't. Check this screenshot

[A Bobrov]

I may have made a mistake when copying, thank you  Nazmur  for your note.

I checked the log again with an extra space after Dec 9 14:30:19 172.16.5.111

Dec  9 14:30:19 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756d4c8,0x0,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733743813,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":50228,"service":1947,"src":"172.16.2.156","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyNDU2MSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}


Dec  9 14:30:19 172.16.5.111 {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756d4c8,0x0,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733743813,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":50228,"service":1947,"src":"172.16.2.156","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyNDU2MSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

With two spaces the decoder does not work, with one - ok. But this is where it comes from, this extra space in the local log.

понедельник, 9 декабря 2024 г. в 14:18:46 UTC+3, Md. Nazmur Sakib:

[A Bobrov]

Nazmur,
in continuation of the topic.
CheckPoint is configured to upload logs in json format.
On the Wazuh server with local log storage, I found this moment in the formation of the log.

One extra space is added after the IP address (Dec  9 13:50:24 172.16.5.111). As soon as you remove it, the log begins to be processed by the decoder

Log with extra space
Dec  9 13:50:24 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733741419,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.56","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}


**Phase 1: Completed pre-decoding.
        full event: 'Dec  9 13:50:24 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733741419,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.56","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'
        timestamp: 'Dec  9 13:50:24'

        hostname: '172.16.5.111'

**Phase 2: Completed decoding.
        No decoder matched.

Log with space removed

Dec  9 13:50:24 172.16.5.111 {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733741419,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.56","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

**Phase 1: Completed pre-decoding.
        full event: 'Dec  9 13:50:24 172.16.5.111 {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733741419,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.56","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'
        timestamp: 'Dec  9 13:50:24'

        hostname: '172.16.5.111'

**Phase 2: Completed decoding.
        name: 'json'

        __policy_id_tag: 'product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]'

        action: 'Drop'
        dst: '172.16.63.255'
        flags: '395524'
        h_version: '5'
        ifdir: 'inbound'
        ifname: 'eth0'
        inzone: 'External'
        layer_name_._._match_table: '['Network']'
        layer_uuid_._._match_table: '['8a994dd3-993e-4c0c-92a1-a8630b153f4c']'

        log_link: 'https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi'
        logid: '0'
        loguid: '0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0'

        match_id_._._match_table: '[3]'
        origin: '172.16.5.11'
        originsicname: 'CN=gwtest,O=TESTSMS..u2xvp5'
        outzone: 'Local'
        parent_rule_._._match_table: '[0]'
        product: 'VPN-1 & FireWall-1'
        proto: '17'
        rule_action_._._match_table: '['Drop']'
        rule_name_._._match_table: '['Cleanup rule']'
        rule_uid_._._match_table: '['60e2929c-4371-402b-bf57-3f2efca62cad']'
        s_port: '137'
        sequencenum: '2'
        service: '137'
        service_id: 'nbname'

        src: '172.16.2.56'
        time: '1733741419'


with two spaces after the IP address is not processed by the decoder
Dec  9 13:50:24 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733741419,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.56","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

with one spaces after the IP address - ОК
Dec  9 13:50:24 172.16.5.111 {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733741419,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.56","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

how to remove extra space in the local log?

понедельник, 9 декабря 2024 г. в 13:36:03 UTC+3, A Bobrov:

[Md. Nazmur Sakib]

I was wondering why there is one space and two space differences before the logs. Can you recheck if this is because of your <localfile> configuration?

If you are not able to remove the extra space by adjusting the localfile configuration I am afraid the JSON decoder will not work for your log. You might need to write custom decoders for your logs.


Decoders

Decoders Syntax

Regular Expression Syntax

I hope you find this information useful.

[A Bobrov]

Dear Nazmur good afternoon
checked <localfile>

ossec.conf
==========
<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/10.16.0.100.log</location>
  <out_format>10.16.0.100.log: $(log)</out_format>

</localfile>

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/10.14.3.103.log</location>
  <out_format>10.14.3.103.log: $(log)</out_format>
</localfile>

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/10.9.0.1.log</location>
  <out_format>10.9.0.1.log: $(log)</out_format>
</localfile>

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/172.16.5.111.log</location>
  <out_format>172.16.5.111.log: $(log)</out_format>
</localfile>

rsyslog.conf
================
if $fromhost-ip startswith '10.16.0.100' then /var/log/10.16.0.100.log
& ~

if $fromhost-ip startswith '10.14.3.103' then /var/log/10.14.3.103.log
& ~

if $fromhost-ip startswith '10.9.0.1' then /var/log/10.9.0.1.log
& ~

if $fromhost-ip startswith '172.16.5.111' then /var/log/172.16.5.111.log
& ~

In addition, Nazmur, logs were displayed in Wazuh dashboard!

For device 172.16.5.111, but as you can see there is an additional space there.
To check, I copy the log from Wazuh dashboard, remove the space, check with the script - everything is ok.

with space

===========

Dec 10 08:55:53 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6757d7e6,0x2,0x6f0510ac,0x1853a3b2","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733810148,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={A7576374-B7AA-EF47-A870-B2EF982212DB};mgmt=TESTSMS;date=1733746054;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.3.100","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM4MDg3ODNAQ0AyMTc0Jm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

without space

===========

Dec 10 08:55:53 172.16.5.111 {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6757d7e6,0x2,0x6f0510ac,0x1853a3b2","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":1,"time":1733810148,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={A7576374-B7AA-EF47-A870-B2EF982212DB};mgmt=TESTSMS;date=1733746054;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.3.100","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM4MDg3ODNAQ0AyMTc0Jm9yaWdfbG9nX3NlcnZlcl9pZD00MGRlNmE3NC1iYTU3LWM4NDgtOWZkZS0yMzMyNGFiN2E5NWI%3D","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

And the log itself processes rule.description Mikrotik-Event

Maybe I can clarify this point with CheckPoint support regarding the extra space in the json logs?

вторник, 10 декабря 2024 г. в 08:26:47 UTC+3, Md. Nazmur Sakib:

[A Bobrov]

in addition to the topic, here is a screenshot of the local log, with extra spaces

 

Maybe I should also look at the date format and other output parameters in rsyslog itself

вторник, 10 декабря 2024 г. в 08:26:47 UTC+3, Md. Nazmur Sakib:

I was wondering why there is one space and two space differences before the logs. Can you recheck if this is because of your <localfile> configuration?

[Md. Nazmur Sakib]

I believe this is happening because the original log has this extra space in the log. In this situation, I suggest you use a custom decoder as the default JSON decoder is not working for your log. This documents will be helpful for writing custom decoders

Decoders

Decoders Syntax

Regular Expression Syntax

I hope you find this helpful.

[A Bobrov]

Dear Nazmur, good afternoon!
Once again I thank you for your answers.
Continuing the topic.

I wrote to CheckPoint support and received a response
===============
Good afternoon, support!!!

We have a question regarding exporting logs in json format.
When receiving the Rsylog log from CheckPoint, an extra space appears in the log itself after the IP address (Dec 9 13:50:24 172.16.5.111)

Log format
==================

Dec  9 13:50:24 172.16.5.111  {"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x6756cb6d,0x2,0x6f0510ac,0x2be8d7c0","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":2,"time":1733741419,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={93D85D95-F4B5-674E-B811-FE55C36B099E};mgmt=TESTSMS;date=1733741103;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":137,"service":137,"service_id":"nbname","src":"172.16.2.56","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM3MjQ5NThAQ0AyMTM3MCZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

As a result of the appearance of an extra space after the IP address (Dec 9 13:50:24 172.16.5.111), the event cannot be processed by the SIEM system.
Is it possible to configure the settings for uploading the json log format in the Appliance CheckPoint configuration in such a way as to remove the extra space after the IP address?

================================

Good afternoon.

Try the following instructions:

https://sc1.checkpoint.com/documents/R81/WebAdminGuides/EN/CP_R81_LoggingAndMonitoring_AdminGuide/Topics-LMG/Log-Exporter-SIEM-specific-instructions.htm?tocpath=Log%20Exporter%7C_____6

Rsyslog
                                       
 By default, Rsyslog is not configured to use the RFC 5424 timestamp format.
Therefore, you should manually change the setting on the Rsyslog server for it to be compliant with the Log Exporter output format.
Edit the /etc/rsyslog.conf file:
vi /etc/rsyslog.conf
                                                   
Comment out this line (add the # character in the beginning), if it is not commented out already:
#"$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat"
                                                   
Add this line in the file:
$ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format
                                                   
Save the changes in the file and exit the editor.
                                                   
Restart the Rsyslog service:
service rsyslog restart                                                                        
===============================================================================================

My actions

They did it, but rolled it back

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
#$ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format

The log format immediately changed, and not entirely for the better. :(
I am attaching screenshots:
1. log output

the log format itself in the log file

made a rollback.

Along the way, I started looking for Rsyslog settings. So far I have made these settings.

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
#$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

$template myFileFormat,"%msg%\n"
$ActionFileDefaultTemplate myFileFormat

The file output has changed and the script can now decode the local log!!!

{"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x675951ff,0x0,0x6f0510ac,0x184d23ac","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":8,"time":1733906938,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={A7576374-B7AA-EF47-A870-B2EF982212DB};mgmt=TESTSMS;date=1733746054;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":138,"service":138,"service_id":"nbdatagram","src":"172.16.1.211","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM4OTg5NjNAQ0AxMDI0OSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

{"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x675951ff,0x0,0x6f0510ac,0x184d23ac","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":8,"time":1733906938,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={A7576374-B7AA-EF47-A870-B2EF982212DB};mgmt=TESTSMS;date=1733746054;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":138,"service":138,"service_id":"nbdatagram","src":"172.16.1.211","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM4OTg5NjNAQ0AxMDI0OSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}

**Phase 1: Completed pre-decoding.
        full event: '{"h_version":5,"action":"Drop","flags":395524,"ifdir":"inbound","ifname":"eth0","logid":0,"loguid":"0x675951ff,0x0,0x6f0510ac,0x184d23ac","origin":"172.16.5.11","originsicname":"CN=gwtest,O=TESTSMS..u2xvp5","sequencenum":8,"time":1733906938,"__policy_id_tag":"product=VPN-1 & FireWall-1[db_tag={A7576374-B7AA-EF47-A870-B2EF982212DB};mgmt=TESTSMS;date=1733746054;policy_name=Standard]","dst":"172.16.63.255","inzone":"External","outzone":"Local","product":"VPN-1 & FireWall-1","proto":17,"s_port":138,"service":138,"service_id":"nbdatagram","src":"172.16.1.211","log_link":"https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM4OTg5NjNAQ0AxMDI0OSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi","parent_rule_._._match_table":[0],"rule_action_._._match_table":["Drop"],"rule_name_._._match_table":["Cleanup rule"],"rule_uid_._._match_table":["60e2929c-4371-402b-bf57-3f2efca62cad"],"match_id_._._match_table":[3],"layer_name_._._match_table":["Network"],"layer_uuid_._._match_table":["8a994dd3-993e-4c0c-92a1-a8630b153f4c"]}'

**Phase 2: Completed decoding.
        name: 'json'

        __policy_id_tag: 'product=VPN-1 & FireWall-1[db_tag={A7576374-B7AA-EF47-A870-B2EF982212DB};mgmt=TESTSMS;date=1733746054;policy_name=Standard]'

        action: 'Drop'
        dst: '172.16.63.255'
        flags: '395524'
        h_version: '5'
        ifdir: 'inbound'
        ifname: 'eth0'
        inzone: 'External'
        layer_name_._._match_table: '['Network']'
        layer_uuid_._._match_table: '['8a994dd3-993e-4c0c-92a1-a8630b153f4c']'

        log_link: 'https://172.16.5.111/smartview/#external-nav%3DOpenLogCard&domain-id%3D41e821a0-3720-11e3-aa6e-0800200c9fde&args%3DbWFya2VyPUBBQEBCQDE3MzM4OTg5NjNAQ0AxMDI0OSZvcmlnX2xvZ19zZXJ2ZXJfaWQ9NDBkZTZhNzQtYmE1Ny1jODQ4LTlmZGUtMjMzMjRhYjdhOTVi'
        logid: '0'
        loguid: '0x675951ff,0x0,0x6f0510ac,0x184d23ac'

        match_id_._._match_table: '[3]'
        origin: '172.16.5.11'
        originsicname: 'CN=gwtest,O=TESTSMS..u2xvp5'
        outzone: 'Local'
        parent_rule_._._match_table: '[0]'
        product: 'VPN-1 & FireWall-1'
        proto: '17'
        rule_action_._._match_table: '['Drop']'
        rule_name_._._match_table: '['Cleanup rule']'
        rule_uid_._._match_table: '['60e2929c-4371-402b-bf57-3f2efca62cad']'

        s_port: '138'
        sequencenum: '8'
        service: '138'
        service_id: 'nbdatagram'
        src: '172.16.1.211'
        time: '1733906938'

in Wazuh dashboard you can now see the log.

How can you get output for this log in Wazuh dashboard?
what do I get when I check with a script?
Thank you again, Nazmur, for your advice and guidance!

среда, 11 декабря 2024 г. в 07:21:20 UTC+3, Md. Nazmur Sakib:

[A Bobrov]

Good afternoon, Nazmur!
In the end, we managed to set up receiving logs from CheckPoint in a format without extra spaces.
I was able to see the log in Wazuh dashboard.
To see the output of the event in the log, which is obtained using a script in the Wazuh dashboard,
I need to start studying the rules and decoders. Is everything right?
Thank you!!!

среда, 11 декабря 2024 г. в 14:43:44 UTC+3, A Bobrov:

[Md. Nazmur Sakib]

I am glad that you are able to resolve the issue. Thank you for sharing the update.

[A Bobrov]

Thank you, Nazmur again for your help in learning Wazuh!!!

Is it possible to view the event in detail from the log on the Wazuh toolbar?
A little different
How can I see the output of an event in the log on the panel?
Wazuh control, which can be seen using a script?
А small decoder and a rule I made. How to see the full event from the log in Wazuh toolbar?

четверг, 12 декабря 2024 г. в 15:05:50 UTC+3, Md. Nazmur Sakib: