Importing hashed password into Keycloak

[Bruno Parmentier]

Hi,

I would like to import Django user accounts with their hashed passwords into Keycloak 12, using the Admin REST API. When testing with one user, this user is successfully created but the password is invalid when trying to log-in.

My Django passwords have a salt and are hashed using PBKDF2 with SHA256, so should be compatible with Keycloak (?). They look like this (this is the hash for userpassword):

pbkdf2_sha256$180000$TAWdyXjZhOxy$Gdf7QXtuAA6S330T/w4Ul5LzcReOgDBx7/Ds6TFJWKs=

The format is algorithm$iterations$salt$hash.

I'm creating the user in Keycloak with the following request:

POST /auth/admin/realms/demo/users/

{
  "username": "us...@example.org",
  "enabled": true,
  "credentials": [
    {
      "credentialData": "{\"hashIterations\": 180000,\"algorithm\": \"pbkdf2-sha256\"}",
      "secretData": "{\"salt\": \"TAWdyXjZhOxy\",\"value\": \"Gdf7QXtuAA6S330T/w4Ul5LzcReOgDBx7/Ds6TFJWKs=\"}",
      "type": "password"
    }
  ]
}

As written above, the user is created but logging in with us...@example.org/userpassword as login/password fails and the logs look like a "normal" login error:

12:09:53,332 WARN  [org.keycloak.events] (default task-2) type=LOGIN_ERROR, realmId=demo, clientId=account-console, userId=b86fd5d9-7f29-4eff-9d3e-19be1607b12e, ipAddress=172.30.0.1, error=invalid_user_credentials, auth_method=openid-connect, auth_type=code, redirect_uri=http://localhost:8080/auth/realms/demo/account/#/, code_id=8bfc2f0c-dc48-4681-aa70-4b82117d39bf, username=us...@example.org, authSessionParentId=8bfc2f0c-dc48-4681-aa70-4b82117d39bf, authSessionTabId=H-zlSpmIjds

I'm suspecting an issue with the salt/hash length or encoding but I cannot see where exactly.

What could I be missing?

Regards,

Bruno Parmentier

[Garth]

I think you need to add a password policy so that your hashing iterations match, as the default is 27500, not 180000.

Go into the admin for the Realm you're using. Click on "Authentication"->"Password Policy". Click "Add policy..."->"Hashing Iterations". Set the value to 180000 and click "Save".

Please message back here if that works, as I'm sure importing passwords from Django is a use case others have.

> --
> You received this message because you are subscribed to the Google
> Groups "Keycloak User" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to keycloak-use...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/keycloak-user/114ae434-d384-4a81-92c7-f338bcfe834bn%40googlegroups.com <https://groups.google.com/d/msgid/keycloak-user/114ae434-d384-4a81-92c7-f338bcfe834bn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Bruno Parmentier]

Thanks for the suggestion. Unfortunately it still doesn't work.

I suppose this policy is used for new passwords, while the password validation function is using the hashIterations value that is stored with each password.

Here is an extract from my Keycloak database:

username

credential_data

secret_data

ad...@example.org

{"hashIterations":180000,"algorithm":"pbkdf2-sha256","additionalParameters":{}}

{"value":"f0PWE2EuMgBkH3cwv79I9ZM0yv0xVMo9JjT3MgTDrBMjZ/kT7+OpQxsoJIzFeACWIHYvHZJNl96ViTeaBBWY8Q==","salt":"PWhIp0vdMbSHrjDEeSTHCw==","additionalParameters":{}}

us...@example.org

{"hashIterations":180000,"algorithm":"pbkdf2-sha256","additionalParameters":{}}

{"value":"Gdf7QXtuAA6S330T/w4Ul5LzcReOgDBx7/Ds6TFJWKs=","salt":"TAWdyXjZhOxy","additionalParameters":{}}

The first user (ad...@example.org, password = adminpassword) was created in Keycloak while the other one was created using the API as described in my first message (us...@example.org, password = userpassword).

The only clear difference I notice is the length of the hash and the salt which are both half the size of the ones created by Keycloak.

I found an article where the author is using the same hash length, but with 27500 iterations and it works: https://dev.to/oskarspakers/import-pbkdf2-hashed-user-passwords-into-keycloak-571m

Could it be that 180000 iterations isn't supported by Keycloak?

Bruno Parmentier

[Garth]

Here are the 2 source files that seem relevant:
https://github.com/keycloak/keycloak/blob/master/server-spi-private/src/main/java/org/keycloak/credential/hash/Pbkdf2Sha256PasswordHashProviderFactory.java
https://github.com/keycloak/keycloak/blob/master/server-spi-private/src/main/java/org/keycloak/credential/hash/Pbkdf2PasswordHashProvider.java

It doesn't look like there is a restriction on hash iterations. However, there is a default param called `derivedKeySize`, that is set to "512". It gets used in this constructor:

https://docs.oracle.com/javase/7/docs/api/javax/crypto/spec/PBEKeySpec.html#PBEKeySpec(char[],%20byte[],%20int,%20int)

Which points to "keyLength - the to-be-derived key length.", which appears to be just the byte length of the encoded value.

It looks like you can't change that with a config param or a password policy, but you should be able to make a new hash provider that passes the appropriate constructor args. Next, I'd try:
1. running a simple main() using that class to see if you get the correct hash value by tweaking the `derivedKeySize` value.
2. if that works, implement a new `PasswordHashProvider` that uses the correct defaults. If you need guidance on how to build a keycloak extension that implements a new provider, check out this example: https://github.com/leroyguillaume/keycloak-bcrypt

Also, might be nice if one of the keycloak devs chimed in. I'm not 100% familiar with how these providers work, so I may be missing something obvious!

> > > us...@example.org/userpassword <http://us...@example.org/userpassword> as login/password fails and the logs look

> https://groups.google.com/d/msgid/keycloak-user/61073ee1-30f5-48ca-8b49-e6f269c12aa8n%40googlegroups.com <https://groups.google.com/d/msgid/keycloak-user/61073ee1-30f5-48ca-8b49-e6f269c12aa8n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Garth]

FYI, I did a quick main() on `Pbkdf2PasswordHashProvider` to test, and I get the correct value for your "us...@example.org" example when I change `derivedKeySize` to 256 from 512. I used:
- algorithm = pbkdf2-sha256
- hashIterations = 180000
- derivedKeySize = 256

Again, I think the only way to use that derivedKeySize value is to implement your own `PasswordHashProvider`, unless someone with more knowledge can drop a hint here. Let us know if you need help with that.

> https://groups.google.com/d/msgid/keycloak-user/c5ac8647-535a-4099-8055-63d95fbdd066%40www.fastmail.com.
>

[Bruno Parmentier]

OK, I found the issue! The Django salt has to be encoded in Base64 before importing it in Keycloak.

In Django, the salt is an ASCII string that that is passed as bytes to the hashlib.pbkdf2_hmac function (for references: https://github.com/django/django/blob/3.1.6/django/contrib/auth/hashers.py#L194 and https://github.com/django/django/blob/3.1.6/django/utils/crypto.py#L89).

In Keycloak, the salt is an array of 16 random bytes that is encoded and stored in Base64 (https://github.com/keycloak/keycloak/blob/12.0.2/server-spi/src/main/java/org/keycloak/models/credential/dto/PasswordSecretData.java#L39).

The thing is that the Django salt is a valid Base64 string as well, so Keycloak didn't raise any alarm when creating the account (e.g. if I put TAWdyXjZhOx* as salt, an exception is raised - and not catched, btw).

So if I re-use the request I put in my first message and I encode the salt TAWdyXjZhOxy in Base64 (= VEFXZHlYalpoT3h5), it works!

Regarding to what you suggested about the derivedKeySize attribute, this seems to be used when generating the hash, not when verifying it. In the verify method of Pbkdf2PasswordHashProvider, the key size is calculated based on the value of the hash (https://github.com/keycloak/keycloak/blob/12.0.2/server-spi-private/src/main/java/org/keycloak/credential/hash/Pbkdf2PasswordHashProvider.java#L96).

Thanks again for your help and for making me dig a bit more in Keycloak's source code, I already had a look but didn't go that far. Too bad the API documentation doesn't explain the format to use in secretData and credentialData.

Regards,

Bruno Parmentier

[Michael Ströder]

On 2/1/21 4:50 PM, Bruno Parmentier wrote:
> Thanks for the suggestion. Unfortunately it still doesn't work.

Recently I wrote Python code to generate PBKDF2 hashes usable in keycloak.

Make sure to use pbkdf2_hmac() with key-word argument dklen=64. With
default args it did not work for me.

Ciao, Michael.

[Bruno Parmentier]

Hi Michael,

I just tested pbkdf2_hmac() with dklen=64 and dklen=32 (the default if you use sha256 as hash_name, see https://docs.python.org/3/library/hashlib.html#hashlib.pbkdf2_hmac) and both hashes were working in Keycloak.

According to the Git history, hashes with key length different than 64 (512 bits) are supported since Keycloak 4 https://github.com/keycloak/keycloak/commit/71e0b006000bee4e325b8635daa321c53a84f3f1.

Working Python example (this is how passwords are hashed in Django 3.0 with PBKDF2PasswordHasher):

>>> import base64
>>> import hashlib

>>> pwd_hash = hashlib.pbkdf2_hmac('sha256', b'mypassword', b'mysalt', 180000)
>>> len(pwd_hash)
32

>>> base64.b64encode(pwd_hash)

b'4E9MZV5g4bp18QvTnliP3SfiYeL2h73V/tkmmH/L/EI='

>>> base64.b64encode(b'mysalt')
b'bXlzYWx0'

You can then create the user in Keycloak with the following request:

POST /auth/admin/realms/demo/users/

{
  "username": "us...@example.org",
  "enabled": true,
  "credentials": [
    {
      "credentialData": "{\"hashIterations\": 180000, \"algorithm\": \"pbkdf2-sha256\"}",

      "secretData": "{\"salt\": \"bXlzYWx0\", \"value\": \"4E9MZV5g4bp18QvTnliP3SfiYeL2h73V/tkmmH/L/EI=\"}",
      "type": "password"
    }
  ]
}

Note that the next time the user logs-in, the password might be re-hashed according to Keycloak's policy.

Regards,

Bruno Parmentier