Proposal: separate password and OTP brute-force protection (and ideally prevent OTP bypass attacks by default)

[Tom Tervoort]

Hello,

I work as a pentester and regularly encounter Keycloak deployments at our customers, where TOTP codes are used as a second authentication factor. What I'm noticing that we keep finding MFA bypass vulnerabilities at those customers, caused by the lack of brute-force restrictions on TOTP codes.

Since TOTPs are only six-digits long the chance of a random code matching whatever TOTP is valid at any given time is only 1 in a million. When the default "look around window" of 1 is used (and 3 in a million when the case when the default "look around window" of 1 is configured). In practice, this meant we could successfully bypass MFA within a few hours per account in each case where we had this finding.

This is not a new observation. We found this advisory from 2021 about this insecure-by-default configuration: https://herolab.usd.de/en/security-advisories/usd-2021-0016/. The recommendation was to enable Keycloak's brute force protection mechanism, which applies to both password and OTP-based logins. There are a few issues with this approach, however:

1. Ideally, we'd like to recommend enforcing a permanent lock-out after a large amount of invalid OTP's are entered. After all, at the point an attacker can submit these they have already compromised a user password. However, because the same policy applies to the password-based login as well this creates a Denial of Service vulnerability in which an unprivileged external attacker can lock out user accounts by simply submitting a number of invalid passwords.
2. It does not appear to be clearly documented that the same brute force protection mechanism applies to both passwords and OTPs.
3. End users expect that when they enable two-factor authentication, that a quite trivial bypass of that authentication factor is possible. In one case a customer originally made the decision not to use brute-force protection for passwords, because the presence of MFA made them accept the risk of password guessing.
4. When using TOTP, many administrators falsely assume that this is impractical because codes change every minute, not realizing that doesn't in the case of a probabilistic attack.
5. It is quite difficult to design a lockout policy that effectively mitigates this risk. For example: one customer had a policy of locking an account for 15 minutes after 15 invalid attempts. This would allow an attacker to perform an average of one OTP guess per minute; meaning it would take around 230 days to do 333.333 attempts. This may seem reasonable, but only applies when the attack targets a single user. If an attacker knows more than one user password, they can run multiple guessing attacks in parallel; effectively multiplying their request rate by the number of users whose password was compromised. If 10 passwords are known, for example, the expected guessing time needed to find an OTP of one of those users becomes less than a month with this policy. This scenario is not unreasonable for large organizations, as we often find dozens of valid passwords when carrying out password spraying or credential stuffing attacks against them.

My proposed solution for these issues would be separate brute-force protection measures used for passwords, from those used for OTPs. The OTP policy may be configurable, or even just hard-code a single reasonable policy. My recommendation for a default setting would be to enforce a permanent lockout when more than X invalid OTPs are submitted in a row. If X is somewhat high (say, 100) it is unlikely that a legitimate user who has issues with their authenticator app will manually enter that many codes before giving up. But X=100 would make malicious guessing attacks completely unfeasible, even when multiple user passwords are breached at once.

Now, I am not a Keycloak administrator myself, and please let me know if I am missing any new features or changes that already address this issue. But I was not able to find documentation on those.

Of course there may be other effective approaches to address the issues I mentioned as well. But I do think the current situation does pose a significant risk to Keycloak users. While it (T)OTP brute-force attacks (unlike some other types of MFA bypass techniques) are still relatively obscure and not widely exploited "in the wild" as far as I know, this could change at any point.

Sorry for the lengthy message. I hope this mailing list is an appropriate channel to raise issues such as these.

Best regards,

Tom Tervoort

[George McGinley Smith]

Can you just expand on this? In the original finding you linked to, it suggests that these kind of attacks are mitigated if brute force detection is enabled. 

Are you saying that you want separate policies so you can have more control or are you saying that there is not enough mitigation just using brute force detection?

Is there a suggested configuration for the current brute force detection to correctly mitigate against this kind of attack?

--
You received this message because you are subscribed to the Google Groups "Keycloak User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/keycloak-user/5bd8bf05-6986-42a1-ab39-2fcb8154f9f7n%40googlegroups.com.

[Tom Tervoort]

Hi George,

I'd say it's mostly a lack of control problem, due to the brute force protections being shared between password and OTP logins. It is certainly possible to mitigate this attack via current brute force policies, but it's difficult to find a policy with a right balance between mitigating a DoS attack against the password login and a probabilistic brute-force attack against the TOTP login. Ideally, I would want to be able to configure permanent lockouts for OTP and temporary lockouts for passwords. 

Another problem is that some users are just going to use the default settings, unaware that this creates an MFA bypass vulnerability. In fact, one of our customers' reasoning for not enabling brute-force protection was that they thought it was unneccessary because they "were using MFA anyway". The documentation also states "The downside of Keycloak brute force detection is that the server becomes vulnerable to denial of service attacks" so this may motivate users not to use it, especially when they are under the impression that an attacker who has brute-forced a password won't be able to proceed due to the MFA.

If I had to recommend a configuration right now, I'd go for something like this:

• Max Login Failures: 30 failures
• Strategy to increase wait time: Linear
• Wait Increment: 15 minutes
• Max Wait: 24 hours
• Failure Reset Time: 24 hours

With such a policy it wouldn't be very advantageous to keep guessing after hitting 30 failures, because you are better off waiting for the failure reset time to trigger, and with around 30 guesses per day actually getting a hit within reasonable time is very unlikely. DoS is still possible, of course, but would require an attacker to continiously keep sending requests after every lockout interval for each user: you can't just permanently lock everyone in a single go.

The default configuration (described on https://www.keycloak.org/docs/latest/server_admin/index.html#lockout-temporarily) doesn't seem to be that bad either: if my quick calculation is correct it allows you to do about 1400 TOTP guesses per day, which would make the average expected time to compromise a single user's TOTP around a year. But you can divide that expected time by the number of users whose password you have compromised, which can make the attack feasible in the shorter term if you've got a few dozen of those.

If it's not possible or desirable to split TOTP and password brute-force policies then I'd say making the default policy a bit stricter would be a good alternative as well; but then I think end users should be urged/forced to enable it whenever they use OTP as an MFA method. It may still be problematic for users with a low appetite for lockout DoS risks, however, although of course there are other mitigating measures they could take outside of Keycloak's scope.

Best regards,

Tom

Op do 29 jan 2026 om 11:48 schreef George McGinley Smith <geo...@gsgd.co.uk>:

You received this message because you are subscribed to a topic in the Google Groups "Keycloak User" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/keycloak-user/eOWfOty1fhs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to keycloak-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/keycloak-user/CACdwTErpkcN3qBt7KTWdSvqOT3-qppSg7wH-aiQT8d5OMA%3DXsQ%40mail.gmail.com.