Wazuh to monitor Keycloak logs

[jayaraman M]

Hi Team,

I am little eager to know, if there are any existing decoders and rules exist with wazuh to monitor Keycloak logs to enable brute-force or DDOS detection.

Reagrds,

jai.

[Federico Gustavo Galland]

Hi Jayaraman,

I don't think we have built-in decoders or rules for keycloak, but we can check if any of the generic ones will catch any events if you share some sample log data with us.

You can sanitize domains and IP addresses beforehand if necessary.

If we don't have decoders and rules for it, I can give you a few hints on how those should be built (with code examples).

Looking forward to receiving those samples.

Regards,

Fede

[Federico Gustavo Galland]

Hi Jayaraman,

I built a few decoders and a rule based on the samples provided. The rule is set to trigger whenever "REFRESH_TOKEN_ERROR" is found within the "type" field.

Hope it helps!

Rules:

<group name="keycloak,community,">

  <rule id="100830" level="0">
    <decoded_as>keycloak-parent</decoded_as>
    <description>Keycloak events grouped</description>
  </rule>

  <rule id="100840" level="7">
    <if_sid>100830</if_sid>
    <field name="type">REFRESH_TOKEN_ERROR</field>
    <description>Keyclock $(facility): $(type)</description>
  </rule>

</group>

Decoders:

<decoder name="keycloak-parent">
  <prematch>iam.keycloak|org.keycloak</prematch>
</decoder>



<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>(\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d\p\d\d:\d\d)</regex>
  <order>timestamp</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>\d\d:\d\d:\d\d,\d\d\d (\S+)\s+[(\S+)] \((\S+)\)</regex>
  <order>facility,app-id,thread-id</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>type=(\S+),</regex>
  <order>type</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>clientId=(\S+),</regex>
  <order>clientId</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>userId=(\S+),</regex>
  <order>userId</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>ipAddress=(\S+),</regex>
  <order>ipAddress</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>error=(\S+),</regex>
  <order>error</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>grant_type=(\S+),</regex>
  <order>grant_type</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>refresh_token_type=(\S+),</regex>
  <order>refresh_token_type</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>refresh_token_id=(\S+),</regex>
  <order>refresh_token_id</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>email=(\S+),</regex>
  <order>email</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>federatedId=(\S+),</regex>
  <order>federatedId</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>roles=(\S+),</regex>
  <order>roles</order>
</decoder>

<decoder name="keycloak-child">
  <parent>keycloak-parent</parent>
  <regex>client_auth_method=(\S+)</regex>
  <order>client_auth_method</order>
</decoder>

On Tue, Oct 17, 2023 at 9:34 AM jayaraman M <prasan...@gmail.com> wrote:

Hi Fede,

Thanks a lot for the prompt response. 

Sure . Please find below the sample log lines for your reference.

Let me know, if i can use this idea, Actually from keycloak we are getting a WARN header for all kind of error messages, I taught of building a decoder to  catch the "type=REFRESH TOKEN ERROR" and order userid and ip address. Rule to say the user has a refresh token error in the below example log. And i can replicate this scenario for authentication issues as a bruteforce rule or multiple login request as a DDOS attack.

2023-10-17T12:34:02+02:00 2023-10-17 10:34:02,934 WARN  [org.keycloak.events] (executor-thread-1353) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=MSTeamsDevAdra, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:03+02:00 2023-10-17 10:34:03,097 WARN  [org.keycloak.events] (executor-thread-1443) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=MSTeamsDevAdra, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:07+02:00 2023-10-17 10:34:07,731 WARN  [org.keycloak.events] (executor-thread-1353) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=teams, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:07+02:00 2023-10-17 10:34:07,762 WARN  [org.keycloak.events] (executor-thread-1622) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=teams, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:07+02:00 2023-10-17 10:34:07,789 WARN  [org.keycloak.events] (executor-thread-1622) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=teams, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:07+02:00 2023-10-17 10:34:07,819 WARN  [org.keycloak.events] (executor-thread-1622) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=teams, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:07+02:00 2023-10-17 10:34:07,950 INFO  [com.telepo.iam.keycloak.provider.storage provider] (executor-thread-1443) getUserId: url: http://user-lookup/user/v1/bylogin/environment1/<User id> responseCode: 200
2023-10-17T12:34:07+02:00 2023-10-17 10:34:07,951 INFO  [com.telepo.iam.keycloak.provider.storage provider] (executor-thread-1443) getFedInfo: result: FederatedInfo {userId=<User id>, email=<Email ID>, federatedId=<Email ID>, roles=[agent, org_admin, crm_connect_user, sms_sender, webphone_user, teams_user, user]}
2023-10-17T12:34:07+02:00 2023-10-17 10:34:07,959 WARN  [org.keycloak.events] (executor-thread-1443) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=teams, userId=f:332e251e-df78-4431-a79f-f46833c95204:<User id>, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, refresh_token_type=Offline, refresh_token_id=1faa9072-9899-4d37-986d-e20422d5f0ce, client_auth_method=client-secret
2023-10-17T12:34:08+02:00 2023-10-17 10:34:08,092 INFO  [com.telepo.iam.keycloak.provider.storage provider] (executor-thread-1443) getUserId: url: http://user-lookup/user/v1/bylogin/environment1/<User ID> responseCode: 200
2023-10-17T12:34:08+02:00 2023-10-17 10:34:08,092 INFO  [com.telepo.iam.keycloak.provider.storage provider] (executor-thread-1443) getFedInfo: result: FederatedInfo {userId=<User ID>, email=<Email ID>, federatedId=<Email ID>, roles=[agent, org_admin, crm_connect_user, sms_sender, webphone_user, teams_user, user, supervisor]}
2023-10-17T12:34:08+02:00 2023-10-17 10:34:08,097 WARN  [org.keycloak.events] (executor-thread-1443) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=teams, userId=f:332e251e-df78-4431-a79f-f46833c95204:<User ID>, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, refresh_token_type=Offline, refresh_token_id=5316b922-1885-4cc8-b4d4-9531ae21648e, client_auth_method=client-secret
2023-10-17T12:34:13+02:00 2023-10-17 10:34:13,224 WARN  [org.keycloak.events] (executor-thread-1622) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=MSTeamsDevAdra, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:13+02:00 2023-10-17 10:34:13,389 WARN  [org.keycloak.events] (executor-thread-1353) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=MSTeamsDevAdra, userId=null, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, client_auth_method=client-secret
2023-10-17T12:34:17+02:00 2023-10-17 10:34:17,567 INFO  [com.telepo.iam.keycloak.provider.storage provider] (executor-thread-1353) getUserId: url: http://user-lookup/user/v1/bylogin/environment1/<User ID> responseCode: 200
2023-10-17T12:34:17+02:00 2023-10-17 10:34:17,567 INFO  [com.telepo.iam.keycloak.provider.storage provider] (executor-thread-1353) getFedInfo: result: FederatedInfo {userId=<User ID>, email=<Email ID>, federatedId=<Email ID>, roles=[agent, org_admin, crm_connect_user, sms_sender, webphone_user, teams_user, user, supervisor]}
2023-10-17T12:34:17+02:00 2023-10-17 10:34:17,574 WARN  [org.keycloak.events] (executor-thread-1353) type=REFRESH_TOKEN_ERROR, realmId=environment1, clientId=teams, userId=f:332e251e-df78-4431-a79f-f46833c95204:<User ID>, ipAddress=<ip address>, error=invalid_token, grant_type=refresh_token, refresh_token_type=Offline, refresh_token_id=46d5822c-2f27-4b20-979d-b18a83bce477, client_auth_method=client-secret

Regards,

jai.

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/pj3Y9DxFHEA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/0e109032-5046-4f5e-b803-3409c118eb26n%40googlegroups.com.

--

Federico Galland Pre-sales Engineer +1 (844) 349 2984 wazuh.com federico...@wazuh.com f-galland

[jayaraman M]

Hi Fede,

Hope you are doing great.

Thanks for the custom decoders and rules. 

1. I have added them to the local_rules.xml and local_decoders.xml files in wazuh server.
2. Also added a local_file to monitor my log file from keycloak(As of now I have made a copy of keycloak log file and stored it in one of my agent to check our rules are working fine)
3. Restarted both agents and server. 
4. There is no continuous log fed to that log file, as i copied a sample log and stored it in one of my agents. 
5. I could able to check the decoders and rules are working fine by testing rules in the toolset option in wazuh dashboard. 
6. But my problem is, i couldn't able to see any alerts in the security events of the dashboards. Did I miss something?.

Regards,

Jai.

[Federico Gustavo Galland]

Hi Jayaraman,

Can you test attaching new log lines to your monitored file on your Agent?

If you still cannot see any alerts, it may be necessary to turn <logall_json> to "yes" in the manager's /var/ossec/etc/ossec.conf

That will enable a file under /var/ossec/logs/archives/archives.json that keeps every received log (remember to disable it when not under use, as the file will grow quickly).

If everything else is set up properly, but the alerts are not showing, we should see the logs pop in there. If this is indeed the case, you can share the lines where those logs appear with me, as we may need to modify decoders/rules for it.

Let me know hot it goes!

Regards,

Fede

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/26661dcb-1ccb-4019-afd6-eddc41b5c3d5n%40googlegroups.com.

[Federico Gustavo Galland]

Hey Jai,

I tried enabling the log_all attribute.

This one enables the archives.log file, but not archives.json. In order to enable archives.json you need to set <logall_json> to yes. Remember to restart the wazuh-manager process afterwards.

 

When I checked the ossec.log in the agent, I caught an error message saying the logcollector cannot read the log file. Now I fixed the issue, the agent is able to read the logs now I confirmed it from the ossec.log file from the agent. But in the archives.json file in manager, there is no log from the specific monitored keycloak log file. But there are logs regarding the rootcheck, syslog monitoring from the syslog logs. 

If after correcting the above, you still cannot find the "keycloak" word under the archives.json file, it means that the file is not being parsed properly. You may want to share your agent's ossec.conf file with me to double check its syntax.

Looking forward to your reply,

Fede

[Federico Gustavo Galland]

Jai,

Take a look at the localfile block for keycloak:

  <localfile>
    <location><path>/keycloak.log</location>
    <log_format>syslog</log_format>
  </localfile>

We need to redefine the <path> tag here (which is not valid for wazuh's config) to the full path of this logfile. For example: /var/log/apache2/access.log

Let me know if that helps.

Fede.

On Fri, Oct 20, 2023 at 11:13 AM jayaraman M <prasan...@gmail.com> wrote:

Hi Federico,

I have set all the changes and still no logs are processed from the keycloak log. Please find below the agent conf file.

<!--

  Wazuh - Agent - Default configuration for darwin 22.6

  More info at: https://documentation.wazuh.com

  Mailing list: https://groups.google.com/forum/#!forum/wazuh

-->

<ossec_config>

  <client>

    <server>

      <address>managerip</address>

      <port>1514</port>

      <protocol>tcp</protocol>

    </server>

    <config-profile>darwin, darwin22, darwin22.6</config-profile>

    <notify_time>10</notify_time>

    <time-reconnect>60</time-reconnect>

    <auto_restart>yes</auto_restart>

    <crypto_method>aes</crypto_method>

  </client>

  <client_buffer>

    <!-- Agent buffer options -->

    <disabled>no</disabled>

    <queue_size>5000</queue_size>

    <events_per_second>500</events_per_second>

  </client_buffer>

  <!-- Policy monitoring -->

  <rootcheck>

    <disabled>no</disabled>

    <check_files>yes</check_files>

    <check_trojans>yes</check_trojans>

    <check_dev>yes</check_dev>

    <check_sys>yes</check_sys>

    <check_pids>yes</check_pids>

    <check_ports>yes</check_ports>

    <check_if>yes</check_if>

    <!-- Frequency that rootcheck is executed - every 12 hours -->

    <frequency>43200</frequency>

    <rootkit_files>etc/shared/rootkit_files.txt</rootkit_files>

    <rootkit_trojans>etc/shared/rootkit_trojans.txt</rootkit_trojans>

    <skip_nfs>yes</skip_nfs>

  </rootcheck>

  <!-- Osquery integration -->

  <wodle name="osquery">

    <disabled>yes</disabled>

    <run_daemon>yes</run_daemon>

    <log_path>/var/log/osquery/osqueryd.results.log</log_path>

    <config_path>/etc/osquery/osquery.conf</config_path>

    <add_labels>yes</add_labels>

  </wodle>

  <!-- System inventory -->

  <wodle name="syscollector">

    <disabled>no</disabled>

    <interval>1h</interval>

    <scan_on_start>yes</scan_on_start>

    <hardware>yes</hardware>

    <os>yes</os>

    <network>yes</network>

    <packages>yes</packages>

    <ports all="no">yes</ports>

    <processes>yes</processes>

    <!-- Database synchronization settings -->

    <synchronization>

      <max_eps>10</max_eps>

    </synchronization>

  </wodle>

  <sca>

    <enabled>yes</enabled>

    <scan_on_start>yes</scan_on_start>

    <interval>12h</interval>

    <skip_nfs>yes</skip_nfs>

  </sca>

  <sca>

    <policies>

      <policy enabled="yes">/Library/Ossec/etc/custom-sca-files/keywordcheck.yml</policy>

    </policies>

  </sca>

  

  <!-- File integrity monitoring -->

  <syscheck>

    <disabled>no</disabled>

    <!-- Frequency that syscheck is executed default every 12 hours -->

    <frequency>900</frequency>

    <scan_on_start>yes</scan_on_start>

    <!-- Directories to check  (perform all possible verifications) -->

    <directories>/etc,/usr/bin,/usr/sbin</directories>

    <directories>/bin,/sbin</directories>

    <directories><path>/wazuhcheck.txt</directories>

    <!-- <directories realtime="yes"><path>/Desktop</directories> -->

    <directories check_all="yes" report_changes="yes">wazuhcheck.txt</directories>

    

  <!-- Files/directories to ignore -->

    <ignore>/etc/mtab</ignore>

    <ignore>/etc/hosts.deny</ignore>

    <ignore>/etc/mail/statistics</ignore>

    <ignore>/etc/random-seed</ignore>

    <ignore>/etc/random.seed</ignore>

    <ignore>/etc/adjtime</ignore>

    <ignore>/etc/httpd/logs</ignore>

    <ignore>/etc/utmpx</ignore>

    <ignore>/etc/wtmpx</ignore>

    <ignore>/etc/cups/certs</ignore>

    <ignore>/etc/dumpdates</ignore>

    <ignore>/etc/svc/volatile</ignore>

    <!-- File types to ignore -->

    <ignore type="sregex">.log$|.swp$</ignore>

    <!-- Check the file, but never compute the diff -->

    <nodiff>/etc/ssl/private.key</nodiff>

    <skip_nfs>yes</skip_nfs>

    <skip_dev>yes</skip_dev>

    <skip_proc>yes</skip_proc>

    <skip_sys>yes</skip_sys>

    <!-- Nice value for Syscheck process -->

    <process_priority>10</process_priority>

    <!-- Maximum output throughput -->

    <max_eps>100</max_eps>

    <!-- Database synchronization settings -->

    <synchronization>

      <enabled>yes</enabled>

      <interval>5m</interval>

      <max_interval>1h</max_interval>

      <max_eps>10</max_eps>

    </synchronization>

  </syscheck>

  <!-- Log analysis -->

  <localfile>

    <log_format>full_command</log_format>

    <command>netstat -an | awk '{if ((/^(tcp|udp)/) && ($4 != "*.*") && ($5 == "*.*")) {print $1" "$4" "$5}}' | sort -u</command>

    <alias>netstat listening ports</alias>

    <frequency>360</frequency>

  </localfile>

  <localfile>

    <location>macos</location>

    <log_format>macos</log_format>

    <query type="trace,log,activity" level="info">(process == "sudo") or (process == "sessionlogoutd" and message contains "logout is complete.") or (process == "sshd") or (process == "tccd" and message contains "Update Access Record") or (message contains "SessionAgentNotificationCenter") or (process == "screensharingd" and message contains "Authentication") or (process == "securityd" and eventMessage contains "Session" and subsystem == "com.apple.securityd")</query>

  </localfile>

  <localfile>

    <location>/var/log/system.log</location>

    <log_format>syslog</log_format>

  </localfile>

  <localfile>

    <location><path>/keycloak.log</location>

    <log_format>syslog</log_format>

  </localfile>

  <!-- Active response -->

  <active-response>

    <disabled>no</disabled>

    <ca_store>etc/wpk_root.pem</ca_store>

    <ca_verification>yes</ca_verification>

  </active-response>

  <!-- Choose between "plain", "json", or "plain,json" for the format of internal logs -->

  <logging>

    <log_format>plain</log_format>

  </logging>

</ossec_config>

[Federico Gustavo Galland]

Jai,

It does need to be an absolute path, not a relative one, though.

On Fri, Oct 20, 2023 at 11:41 AM jayaraman M <prasan...@gmail.com> wrote:

Hi Fede,

I truncated the path as it has my sensitive information and masked it with the <path>. But in my configuration i have given the whole relative path there instead of <path>.

Regards,

jai.

[jayaraman M]

Hi Fede,

Sorry, I think I have confused you.The path for the keycloak file is mentioned as "/Users/myusername/Desktop/keycloak.log" in the <location> block under the <localfile> block in my agent configuration file. As I am posting the configuration file online for security purpose, just masked it to "<path>" keyword instead of the path of the keycloak log. Below is how my agent conf looks like for keycloak log inclusion.

<localfile>

    <location>/Users/user/Desktop/keycloak.log</location>

    <log_format>syslog</log_format>

  </localfile>

Regards,

jai. 

[Federico Gustavo Galland]

Hi Jai,

In case any other decoders are catching your logs before our custom one, we can do the following:

Replace your localfile section with the following:

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/keycloak.log</location>
    <out_format>keycloak_logs: $(log)</out_format>
  </localfile>

** Change the path to fit your own.

Modify your parent decoder to the following:

<decoder name="keycloak-parent">
  <prematch>^keycloak_logs:</prematch>
</decoder>

These modifications will prepend "keycloak_logs: " to every line from the log file. The parent decoder will then use that string to match the whole log.

Let me know if this fixes the issue.

Regards,

Fede.

On Mon, Oct 23, 2023 at 5:23 AM jayaraman M <prasan...@gmail.com> wrote:

Hi Fede,

Sorry, I think I have confused you.The path for the keycloak file is mentioned as "----" in the <location> block under the <localfile> block in my agent configuration file. As I am posting the configuration file online for security purpose, just masked it to "<path>" keyword instead of the path of the keycloak log. Below is how my agent conf looks like for keycloak log inclusion.

<localfile>

    <location>----</location>

    <log_format>syslog</log_format>

  </localfile>

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/67200bc4-952d-4d52-ad70-9d1f08d1cfb6n%40googlegroups.com.

[jayaraman M]

Hi Fede,

Hope you are doing great. Thanks for your response.

I made a blunder mistake, the keycloak log has a header at the starting of the file stating the "logcollected_app=keycloak", which i missed to delete before trying this test. This made the decoder to miss the matched log files i guess. But now i could successfully see the logs in the dashboard.

Thanks a lot for the help :) .

Regards,

jai.

[Federico Gustavo Galland]

Jai,

Glad you could solve it!

Regards,

Fede

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/7fc5ba00-2d08-45ae-b811-7c99710abedcn%40googlegroups.com.