Zxcvbn custom wordlist not taken into account

[Clément Gindrier]

Hello, 

I started to use "Zxcvbn" algorithm and I added a custom wordlist, and even though the logs are good, telling me it's imported, it seems Zxcvbn only check the default wordlist and not mine. I am stuck on this issue for a while, I need your help.
Do you have an idea what I missed?

I use PWM 2.0.6

- I added the wordlist in "Settings ⇨ Word Lists  ⇨ Word List File URL".

- In local, it's in "/home/.../wordlist260120.zip" in WSL.

- It's a zip file with a directory "wordlist260120" with a text file " wordlist260120.txt" in it (I also tested without directory)

- Word List Case Sensitivity is disabled
- Word List Word Size Check is at 0

The Word Count went from 800 000 to 24 000, which means it's replaced but my wordlist.



But the strength bar didn't react like the one when I add words directly in the code. Also, I displayed the words that the algorithm find, and new words are not found.

For example, this word is on the default list but not mine, it displays it when I type it:


But this one is only in my wordlist and is not detected:






The format of the file is like that:



Do you see something that I did wrong.

Thank you in advance for the help!

[Jason Rivard]

Are you sure wordlist checking is enabled on the user's policy?  use the 'admin -> user debug' page to check the effective password policy for the user has wordlist checking enabled.

[Clément Gindrier]

Hello Jason, thanks for the answer! 
I can see the setting "Policies ⇨ Password Policies ⇨ ...  ⇨ Enforce Word List" that is related to "wordlist checking" in user's policy. I tried to check and uncheck it for all users and it doesn't change anything for me :/

I don't understand because even with "Word List Case Sensitivity" or " Word List Word Size Check", no difference can be seen, it's like this is no taken into account. Even restarting Tomcat... 

But if I change rules in Password Policy, the new policy is taken into account (for example Minimum Strength, etc.)

I must forget something but I don't know what.

User's password policy shows that "Enable Word List" is true.


Logs show that the 24 223 words are well imported (which correspond exactly to my dictionary).
But config/manager/wordlists shows me 44 139 words are imported, which make no sense. I don't know why..



Do you have any idea?

[Jason Rivard]

You can try setting the log level to trace and see what happens when you mutate a password in the UI.  There's generally a bit of logging at trace then happens during a wordlist check...

[Jason Rivard]

Two other thoughts:

1) the import process shows type "raw" which means its using the literal word found byte by byte in the wordlist file inside the zip.   If your using hashes you need to prefix each word with "SHA1:" or whatever the hash algo is, then the value of the hash in hex.    

2) The wordlist check is always the last thing in the password policy to be validated, so if any other check fails it won't bother with wordlist checking.

Hope this helps.

[Clément Gindrier]

You got a point, "Responses Storage Hashing Method" is set with a hash method. I tried first to set the hashing method to plain text and with plain text wordlist and it was not working. I also tried to use the hashing method + ":" + hex(hash(word)). It wasn't working, but maybe it's something to do with parameters (salt, iter, ..).

The second point doesn't seem to be the problem.

I already tried the logs but I don't see much...


If it doesn't work I can still integrate my words into the library, but it's not very clean

[Jason Rivard]

The response set hash settings control the way that user response sets, "the secret question answers" for forgotten password are stored.  The wordlist hashing has no setting and is controlled only by the per-row value in the wordlist text file.  If the text file line starts with:

sha1:hexvalue then the word is treated as a hash, which means the password is hashed to that format and compared to the hashed value of that particular word in the wordlist.   if the "sha1:" prefix isn't on that particular line, than the word is imported as is and is compared to the user input in plaintext.  

[Clément Gindrier]

Thank you very much for the answer, I don't get how it can compare with hash if the word is part of the password. But anyway I tested with all the hashing methods, I also tried to only put the hexa of the word without hashing, but nothing has been working. 
I tested with one word that is not detected with the default wordlist, for example: zalau.

So for sha1 I have a zip file with a .txt file with sha1:43b6fd09f06b9f0e9b3c5059db9e3fec9dbf06ea

Same with sha256, sha512, md5....

But the word is never detected

In the logs I have a line with "secure.SecureService, using default ephemeral hash algorithm" and then a hashing method. And also I got WordTypes={"SHA1":{}}, or other depending of the hashing method I used in the wordlist.
I don't have other idea to test to making work :/

I will try in debug mode and go line by line to understand how it works, but I don't have many time left on the project sadly

[Jason Rivard]

Wordlist checking only checks the entire word in the wordlist, or segments of the user password based on the setting 'Settings ⇨ Word Lists ⇨ Word List Word Size Check', but the default zero will only check the entire word.   See the help text for that setting.  This behavior is the same regardless if the word in the wordlist is hashed or not hashed, thus only whole hashes are ever evaluated. 

Ephemeral hashing is unrelated to the wordlist, its used for encrypted data in the user's http session.   At the end of an import there should be a logging statement showing how many words were imported of each hash type.

Hope this helps..