Separating o365 logs

Андрей Смирнов

unread,

Aug 14, 2023, 7:32:40 AM8/14/23

to Wazuh mailing list

Hello!

For example, I have Country_A O365 logs connected to Wazuh using office365 module and Country_B connected the same way. Logs that are ingested in Wazuh doesn't have any unique value that can be used to separate them by different indexes, for example, wazuh-alert-country_a and wazuh-alert-country_b.

To achieve that I assume that I need to do something that will add a custom unique field to every log entry in Wazuh and then I can configure pipeline.json to match this field and store logs under different indexes.

But I am stuck the very first step - how to add custom unique field to every log that is ingested from O365, AWS or Azure using wodle's and modules. I would be very grateful if someone could help me with this or maybe there is other solution.

Diego Mendez Sakugawa

unread,

Aug 14, 2023, 1:20:10 PM8/14/23

to Wazuh mailing list

Hello!

It would be great if you could send us two different examples of a log for each index, so I can locally replicate the configuration that you need and assist you with it.

Please remember to obfuscate any sensitive data!

Looking forward to your feedback!

Andrej Smirnov

unread,

Aug 15, 2023, 3:02:42 AM8/15/23

to Wazuh mailing list

Good day Diego!

Here are two different log entries generated by o365 wodle. First one is DLP.ALL subscription, second is Audit.General.

They both come to one index - wazuh-alerts-4.x-<date>. What I need is that there is some unique field that is appended to the log that will tell that first log came from cpuntry_a office365 tenant and second came from country_b office365 tenant, so that I can then configure pipeline.json - if this field matches some value then store this log in wazuh-alerts-4.x-country-a and so on.

1. DLP.ALL

{
"_index": "wazuh-alerts-4.x-2023.08.15",
"_id": "MO2loBm",
"_score": 1,
"_source": {
"agent": {
"name": "wazuh-server",
"id": "000"
},
"manager": {
"name": "wazuh-server"
},
"data": {
"integration": "office365",
"office365": {
"ObjectId": "45a97a5f1ac8fd2",
"UserKey": "11111194278b9d",
"Operation": "DLPRuleMatch",
"OrganizationId": "2501531ee910",
"IncidentId": "ec68e85a72b2",
"Workload": "SharePoint",
"SensitiveInfoDetectionIsIncluded": "true",
"RecordType": "11",
"Version": "1",
"UserId": "USER@COUNTRY_A",
"CreationTime": "2023-08-15T06:26:35",
"SharePointMetaData": {
"UniqueID": "45aac8fd2",
"SiteCollectionGuid": "619120",
"FileName": ".pdf",
"FilePathUrl": "",
"FileOwner": "",
"From": "USER@COUNTRY_A",
"SensitivityLabelIds": [],
"ItemLastModifiedTime": "2023-08-15T06:26:07",
"SiteAdmin": [],
"IsVisibleOnlyToOdbOwner": "false",
"ItemLastSharedTime": "0001-01-02T00:00:00",
"SiteCollectionUrl": "",
"IsViewableByExternalUsers": "true",
"SharedBy": [
null
],
"FileID": "",
"ItemCreationTime": "2023-08-15T06:26:07",
"FileSize": "574317"
},
"PolicyDetails": [
{
"PolicyName": "General Data Protection Regulation (GDPR)",
"Rules": [
{
"ActionParameters": [
"GenerateIncidentReport:SiteAdmin"
],
"Actions": [],
"RuleId": "b84409dab8afd2",
"RuleMode": "TestWithoutNotifyUser",
"ManagementRuleId": "958aa981-284d-48d5-8b8b-f642943a2498",
"ConditionsMatched": {
"SensitiveInformation": [
{
"ClassifierType": "Content",
"SensitiveInformationDetections": {
"DetectedValues": [
{
"Value": "",
"Name": ""
},
],
"ResultsTruncated": false
},
"Confidence": 93,
"Count": 62,
"SensitiveInformationTypeName": "EU National Identification Number",
"SensitiveInformationDetailedClassificationAttributes": [],
"SensitiveType": "419f47a91b41"
},
{
"ClassifierType": "Content",
"SensitiveInformationDetections": {
"DetectedValues": [
{
"Value": "",
"Name": ""
},
],
"ResultsTruncated": false
},
"Confidence": 98,
"Count": 8,
"SensitiveInformationTypeName": "EU Social Security Number (SSN) or Equivalent ID",
"SensitiveInformationDetailedClassificationAttributes": [
{
"IsMatch": false,
"Confidence": 65,
"Count": 8
},
{
"IsMatch": true,
"Confidence": 75,
"Count": 8
},
{
"IsMatch": false,
"Confidence": 85,
"Count": 0
}
],
"SensitiveType": "d24e32a4-c0bb6303b95742d9"
},
{
"ClassifierType": "Content",
"SensitiveInformationDetections": {
"DetectedValues": [
{
"Value": "",
"Name": ""
},
],
"ResultsTruncated": false
},
"Confidence": 93,
"Count": 10,
"SensitiveInformationTypeName": "EU Tax Identification Number (TIN)",
"SensitiveInformationDetailedClassificationAttributes": [],
"SensitiveType": "e09c0c62748f5f"
}
],
"ConditionMatchedInNewScheme": true,
"OtherConditions": [
{
"Value": "IncludeExternalUsers",
"Name": "AccessScope"
}
]
},
"Severity": "Low",
"RuleName": "Low volume of content detected General Data Protection Regulati"
}
],
"PolicyId": "4e2603312031726745"
}
],
"Id": "391ac1de-82da-4ddb-af94-9ba5fb061025",
"Subscription": "DLP.All",
"UserType": "0"
}
},
"rule": {
"firedtimes": 12,
"mail": false,
"level": 3,
"hipaa": [
"164.312.a.1",
"164.312.b",
"164.312.c.1"
],
"pci_dss": [
"10.6.1",
"11.5"
],
"description": "Office 365: Data loss protection (DLP) events in SharePoint and OneDrive for Business.",
"groups": [
"office365",
"ComplianceDLPSharePoint"
],
"mitre": {
"technique": [
"Sharepoint"
],
"id": [
"T1213.002"
],
"tactic": [
"Collection"
]
},
"id": "91542"
},
"decoder": {
"name": "json"
},
"input": {
"type": "log"
},
"@timestamp": "2023-08-15T06:34:55.132Z",
"location": "office365",
"id": "1692081295.1626002426",
"timestamp": "2023-08-15T06:34:55.132+0000"
},
"fields": {
"@timestamp": [
"2023-08-15T06:34:55.132Z"
],
"timestamp": [
"2023-08-15T06:34:55.132Z"
]
}
}

2. Audit.General

{
"_index": "wazuh-alerts-4.x-2023.08.15",
"_id": "GOnw94KQh23VZc",
"_score": 1,
"_source": {
"agent": {
"name": "wazuh-server",
"id": "000"
},
"manager": {
"name": "wazuh-server"
},
"data": {
"integration": "office365",
"office365": {
"MessageVersion": "1691940",
"TeamGuid": "19:6cee12cb4e155c1e787",
"UserKey": "cbe7a4e721ab67",
"Operation": "ReactedToMessage",
"OrganizationId": "25015e0e0331ee910",
"ParentMessageId": "16920130553",
"TeamName": "",
"AppAccessContext": {
"UniqueTokenId": "kzwpxBgWic_AA",
"IssuedAtTime": "2023-08-15T06:05:24"
},
"ClientIP": "822",
"Workload": "MicrosoftTeams",
"ParticipantInfo": {
"HasOtherGuestUsers": "false",
"HasForeignTenantUsers": "false",
"HasGuestUsers": "false",
"HasUnauthenticatedUsers": "false"
},
"ChannelName": "General",
"RecordType": "25",
"Version": "1",
"UserId": "USER@COUNTRY_B",
"ChannelGuid": "19:6tacv2",
"CreationTime": "2023-08-15T06:43:11",
"ExtraProperties": [
{
"Value": "Euje",
"Key": "TimeZone"
},
{
"Value": "windows",
"Key": "OsName"
},
{
"Value": "10",
"Key": "OsVersion"
},
{
"Value": "us",
"Key": "Country"
},
{
"Value": "skeams",
"Key": "ClientName"
},
{
"Value": "270810",
"Key": "ClientVersion"
},
{
"Value": "7200",
"Key": "ClientUtcOffsetSeconds"
}
],
"Id": "d93f61fd3551f",
"MessageReactionType": "like",
"Subscription": "Audit.General",
"UserType": "0",
"AADGroupId": "eaf8be059591720",
"MessageId": "1692010230553"
}
},
"rule": {
"firedtimes": 57,
"mail": false,
"level": 3,
"hipaa": [
"164.312.b"
],
"pci_dss": [
"10.6.2"
],
"description": "Office 365: Events from Microsoft Teams.",
"groups": [
"office365",
"MicrosoftTeams"
],
"id": "91555"
},
"decoder": {
"name": "json"
},
"input": {
"type": "log"
},
"@timestamp": "2023-08-15T06:45:03.024Z",
"location": "office365",
"id": "16920810165",
"timestamp": "2023-08-15T06:45:03.024+0000"
},
"fields": {
"@timestamp": [
"2023-08-15T06:45:03.024Z"
],
"timestamp": [
"2023-08-15T06:45:03.024Z"
]
}
}

понедельник, 14 августа 2023 г. в 20:20:10 UTC+3, Diego Mendez Sakugawa:

Diego Mendez Sakugawa

unread,

Sep 11, 2023, 9:31:50 AM9/11/23

to Wazuh | Mailing List

Hello Andrej Smirnov,

My response from last week wasn't appropriately uploaded, I don't know what happened. I'm really sorry for the inconvenience.

To achieve what you're looking for, you will need to:

Modify the /usr/share/filebeat/module/wazuh/alerts/ingest/pipeline.json file by replacing the:

{
"date_index_name": {
"field": "timestamp",
"date_rounding": "d",
"index_name_prefix": "{{fields.index_prefix}}",
"index_name_format": "yyyy.MM.dd",
"ignore_failure": false
}
},

With:

{
"date_index_name": {
"field": "timestamp",
"date_rounding": "d",
"index_name_prefix": "countrya-",
"index_name_format": "yyyy.MM.dd",
"ignore_failure": false,
"if" : "if (ctx.data.office365.UserId.contains('COUNTRY_A')){return true;}"
}
},
{
"date_index_name": {
"field": "timestamp",
"date_rounding": "d",
"index_name_prefix": "countryb-",
"index_name_format": "yyyy.MM.dd",
"ignore_failure": false,
"if" : "if (ctx.data.office365.UserId.contains('COUNTRY_B')){return true;}"
}
},
{
"date_index_name": {
"field": "timestamp",
"date_rounding": "d",
"index_name_prefix": "{{fields.index_prefix}}",
"index_name_format": "yyyy.MM.dd",
"ignore_failure": false,
"if" : "if (ctx.data.office365.UserId.contains('COUNTRY_A')){return false;}"
}
},

Finally, set up the pipeline changes and restart the filebeat service:

filebeat setup --pipelines

systemctl restart filebeat

Hopefully, this is what you need. Sorry once again for the late response.

Looking forward to your feedback.

Best regards.

Andrej Smirnov

unread,

Oct 10, 2023, 4:22:48 AM10/10/23

to Wazuh | Mailing List

Hello Diego!

Thank you for the answer. This part is clear.

Might be that I have described my issue incorrectly, so I will try to clarify: the thing is that there are not COUNTRY_%1 values in the logs (I have added them to describe the issue, so that might have confused you). So the idea is that I want to somehow add a custom field to the log, that will be parsed to Wazuh and then by this custom field I will do what you have described.

Please let me know if there is a possibility to add custom field (ex. country: COUNTRY_A; country: COUNTRY_B;) to o365 logs using o365 wazuh module?

Regards,

Andrejs

понедельник, 11 сентября 2023 г. в 16:31:50 UTC+3, Diego Mendez Sakugawa:

Reply all

Reply to author

Forward