Logs from TMG

Nataliia

unread,

Aug 26, 2022, 5:17:35 AM8/26/22

to Wazuh mailing list

Hi there!

I want to collect logs from Microsoft Forefront Threat Management Gateway and added location to localfile (logs are collecting to directory D:\Logs\ on the TMG server:

<log_format>syslog</log_format>

</localfile>

But in the archives.log I see only EventChannel logs, I don't see any file from D:\Logs

Even when I set location to definite file - D:\Logs\ISALOG_20220818_WEB_000.w3c - I still don't see any logs besides of EventChannel logs.

Help me to configure localfile correctly, please.

Mauricio Ruben Santillan

unread,

Aug 26, 2022, 1:02:17 PM8/26/22

to Wazuh mailing list

Hello!

Thanks for using Wazuh!

For starters, where have you set such module? It should be placed in your Agent's ossec.conf file (or sent to it using Centralized configuration).
Also, you should enable logcollector.debug on the Agent's C:\Program Files (x86)\ossec-agent\internal_options.conf by setting it to 2 (by default is 0) in order to get more information on the Wazuh Agent log file. Make sure to restart the Agent service for this change to impact. There's related information here.
Once done, try checking the Agent's log file C:\Program Files (x86)\ossec-agent\ossec.log. It should tell you if it is actually reading the file. Remember to set logcollector.debug back to 0 when done.

If the agent is actually reading the file, then you should see events from such files in your Manager's archives.json. By the way, did you make sure that you have enabled logall_json in your manager for it to feed the archives.json file?

Other than this, it would be very useful to check some samples of those log files. Would it be possible for you to share some of these files or samples from them?

Looking forward to your comments.

Nataliia

unread,

Aug 29, 2022, 11:53:23 AM8/29/22

to Wazuh mailing list

Hello!

When I configured localfile in my Agent's ossec.conf file and enabled logall option I saw logs in the archives.log file. Thank you!

Here is samples of logs:

2022 Aug 29 15:24:06 (sho-tmg01) any->\Logs\ISALOG_20220829_WEB_000.w3c 10.20.7.52 anonymous - 2022-08-29 15:23:57 SHO-TMG01 - graph.microsoft.com 40.126.32.161 443 0 4051 7400 SSL-tunnel - graph.microsoft.com:443 - Inet 0 Allow Web Access for All Users Req ID: 11bdb0ff Internal External 0x8 Allowed - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - 10.20.29.40 Feature disabled Web Proxy graph.microsoft.com 55892 -

2022 Aug 29 15:24:06 (sho-tmg01) any->\Logs\ISALOG_20220829_WEB_000.w3c 10.20.20.13 anonymous Calendar Connector 2022-08-29 15:23:57 SHO-TMG01 - wdm-k.wbx2.com 10.20.15.29 443 0 0 2887 SSL-tunnel - wdm-k.wbx2.com:443 - Inet 12233 Blocked P2P/File Sharing Req ID: 11bdc147 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy wdm-k.wbx2.com 30796 -

I have configured decoder and rules:

Decoder:

<decoder name="tmg-log">
<prematch>ISALOG</prematch>>
</decoder>

<decoder name="tmg-log">
<parent>tmg-log</parent>
<regex>(\d+.\d+.\d+.\d+)</regex>
<order>srcip</order>
</decoder>

<decoder name="tmg-log">
<parent>tmg-log</parent>
<regex>TMG\d+\s+-\s+(\S+)\s+(\d+.\d+.\d+.\d+)\s+(\d+)\s+\d+\s+\d+\s+\d+\s+(\S+)</regex>
<order>url,dstip,dstport,protocol</order>
</decoder>

<decoder name="tmg-log">
<parent>tmg-log</parent>
<regex>Inet\s+(\d+)\s+(\S+)</regex>
<order>id,rule_name</order>
</decoder>

Rules:

<group name="tmg">


<rule id="102000" level="0">
<decoded_as>tmg-log</decoded_as>
<description>TMG messages grouped.</description>
</rule>

<rule id="102001" level="5">
<if_sid>102000</if_sid>
<id>^12233</id>
<description>Access denied by rule - Blocked P2P/File Sharing</description>
</rule>

<rule id="102002" level="5">
<if_sid>102000</if_sid>
<id>^200|^0</id>
<description>Acces allowed by rule - Allow Web Access for All Users</description>
</rule>

</group>

In the rule test I see result:

**Phase 1: Completed pre-decoding.

full event: 2022 Aug 29 15:24:06 (sho-tmg01) any->\Logs\ISALOG_20220829_WEB_000.w3c 10.20.20.13 anonymous Calendar Connector 2022-08-29 15:23:57 SHO-TMG01 - wdm-k.wbx2.com 10.20.15.29 443 0 0 2887 SSL-tunnel - wdm-k.wbx2.com:443 - Inet 12233 Blocked P2P/File Sharing Req ID: 11bdc147 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy wdm-k.wbx2.com 30796 -

timestamp: 2022 Aug 29 15:24:06

hostname: -

program_name: -

**Phase 2: Completed decoding.

name: tmg-log

data: { "protocol": "SSL-tunnel",

"srcip": "10.20.20.13",

"dstip": "10.20.15.29",

"dstport": "443",

"id": "12233",

"url": "wdm-k.wbx2.com",

"rule_name": "Blocked" }

**Phase 3: Completed filtering (rules).

id: 102001

level: 5

description: Access denied by rule - Blocked P2P/File Sharing

groups: ["tmg"]

firedtimes: 5

gdpr: "-"

gpg13: "-"

hipaa: "-"

mail: "-"

mitre.id: "-"

mitre.technique: "-"

nist_800_53: "-"

pci_dss: "-"

tsc: "-"

**Alert to be generated.

But I don't see any log in the Discover.

Can you help me to see this logs in the Discover?

пятница, 26 августа 2022 г. в 20:02:17 UTC+3, mauricio....@wazuh.com:

Juan Carlos Tello

unread,

Sep 6, 2022, 8:26:41 AM9/6/22

to Nataliia, Wazuh mailing list

Hi Natalia,

When using <logall>, the entries added into the /var/ossec/logs/archives/archives.log file will include a header indicating the source of the event, specifically stating the collection timestamp, agent, binding IP and log file, which in your case is: 2022 Aug 29 15:24:06 (sho-tmg01) any->\Logs\ISALOG_20220829_WEB_000.w3c . This is not part of the log itself so the first decoder will not be able to find the ISALOG string it is looking for in its prematch.

The log that must be used for testing should instead be:

10.20.20.13 anonymous Calendar Connector 2022-08-29 15:23:57 SHO-TMG01 - wdm-k.wbx2.com 10.20.15.29 443 0 0 2887 SSL-tunnel - wdm-k.wbx2.com:443 - Inet 12233 Blocked P2P/File Sharing Req ID: 11bdc147 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy wdm-k.wbx2.com 30796 -

The first decoder can instead be:

Where you'll find that the log matches.

I assumed all logs started with the same string format, if this is not the case then the regular expression will need to be adapted.

An alternative to identify the logs for an easier pre-match is to collect it using the <out_format> option

<localfile>
<location>D:\Logs\*</location>
<log_format>syslog</log_format>

<out_format>TMG-LOG: $(log)</out_format>
</localfile>

This will allow you to capture the logs with the <prematch>^TMG-LOG</prematch>

I hope this helps, let us know if you have any more questions.

Best Regards,

Juan C. Tello

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/1dce774a-9a4b-4d3a-b308-f44e50686176n%40googlegroups.com.

Nataliia

unread,

Oct 6, 2022, 10:38:21 AM10/6/22

to Wazuh mailing list

Hello!

Hope you are well.

I added <out_format> and really log 10.20.20.13 anonymous Calendar Connector 2022-08-29 15:23:57 SHO-TMG01 - wdm-k.wbx2.com 10.20.15.29 443 0 0 2887 SSL-tunnel - wdm-k.wbx2.com:443 - Inet 12233 Blocked P2P/File Sharing Req ID: 11bdc147 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy wdm-k.wbx2.com 30796 - was decoded including data, which is described in my decoders. But when I try decoder test for logs such of this I see only decoder name, but don't see data:

10.20.140.138 ENT\Oleksii.Kozintsev - 2022-10-06 13:09:56 SHO-TMG01 - sls.update.microsoft.com 10.20.15.29 443 0 0 2891 SSL-tunnel - sls.update.microsoft.com:443 - Inet 12202 Block srv subnet inet Req ID: 17f9f9b8 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy sls.update.microsoft.com 56403 -

In this case decoder result is:

**Phase 1: Completed pre-decoding.

full event: 10.20.140.138 ENT\Oleksii.Kozintsev - 2022-10-06 13:09:56 SHO-TMG01 - sls.update.microsoft.com 10.20.15.29 443 0 0 2891 SSL-tunnel - sls.update.microsoft.com:443 - Inet 12202 Block srv subnet inet Req ID: 17f9f9b8 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy sls.update.microsoft.com 56403 -

timestamp: -

hostname: -

program_name: -

**Phase 2: Completed decoding.

name: -

data: "-"

**Phase 3: Completed filtering (rules).

id: 1002

level: 2

description: Unknown problem somewhere in the system.

groups: ["syslog","errors"]

firedtimes: 3

gdpr: "-"

gpg13: ["4.3"]

hipaa: "-"

mail: "-"

mitre.id: "-"

mitre.technique: "-"

nist_800_53: "-"

pci_dss: "-"

tsc: "-"

And when I test this log it's just no result found:

10.20.2.62 anonymous - 2022-10-06 13:09:57 SHO-TMG01 - config.edge.skype.com 13.107.42.16 443 0 906 6923 SSL-tunnel - config.edge.skype.com:443 - Inet 0 Allow Web Access for All Users Req ID: 17f9f9a6 Internal External 0x8 Allowed - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - 10.20.29.40 Feature disabled Web Proxy config.edge.skype.com 56185 -

вторник, 6 сентября 2022 г. в 15:26:41 UTC+3, Juan Carlos:

Juan Carlos Tello

unread,

Oct 14, 2022, 4:27:54 AM10/14/22

to Nataliia, Wazuh mailing list

Hi Natalia,

If you have used <out_format> please kindly share your configuration so we may help you while knowing the nature of the logs as received by the analysis engine.

I see that the new log you have provided is almost identical to the previous one with the exception of the first three words after the IP at the beginning.

To catch both types of messages you may use the following parent decoder:

Using this and the rest of the decoders and rules the result is:

**Phase 1: Completed pre-decoding.
	full event: '10.20.20.13 anonymous Calendar Connector 2022-08-29 15:23:57 SHO-TMG01 - wdm-k.wbx2.com 10.20.15.29 443 0 0 2887 SSL-tunnel - wdm-k.wbx2.com:443 - Inet 12233 Blocked P2P/File Sharing Req ID: 11bdc147 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy wdm-k.wbx2.com 30796 -'

**Phase 2: Completed decoding.
	name: 'tmg-log'
	dstip: '10.20.15.29'
	dstport: '443'
	id: '12233'
	protocol: 'SSL-tunnel'
	rule_name: 'Blocked'
	srcip: '10.20.20.13'
	url: 'wdm-k.wbx2.com'

**Phase 3: Completed filtering (rules).
	id: '102001'
	level: '5'
	description: 'Access denied by rule - Blocked P2P/File Sharing'
	groups: '["tmg"]'
	firedtimes: '1'
	mail: 'false'
**Alert to be generated.

---------------------------------------------------------------------------

**Phase 1: Completed pre-decoding.
	full event: '10.20.140.138 ENT\Oleksii.Kozintsev - 2022-10-06 13:09:56 SHO-TMG01 - sls.update.microsoft.com 10.20.15.29 443 0 0 2891 SSL-tunnel - sls.update.microsoft.com:443 - Inet 12202 Block srv subnet inet Req ID: 17f9f9b8 Internal External 0x0 Denied - - - - Allowed Malware Inspection Disabled Unknown - 0 - 0 - - Feature disabled Web Proxy sls.update.microsoft.com 56403 - '

**Phase 2: Completed decoding.
	name: 'tmg-log'
	dstip: '10.20.15.29'
	dstport: '443'
	id: '12202'
	protocol: 'SSL-tunnel'
	rule_name: 'Block'
	srcip: '10.20.140.138'
	url: 'sls.update.microsoft.com'

**Phase 3: Completed filtering (rules).
	id: '102000'
	level: '0'
	description: 'TMG messages grouped.'
	groups: '["tmg"]'
	firedtimes: '1'
	mail: 'false'

I hope you find this helpful, I recommend sharing both your configuration and a more thorough explanation of your sample logs if you're still facing issues.

Best Regards,

Juan C. Tello

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/0b2b9e2b-baaa-456e-b043-2ff8b14e0e0en%40googlegroups.com.

Reply all

Reply to author

Forward