FIPS Bouncy Castle Cannot create a credential store

[Nate]

I am unable to create a credential store on FIPS enabled RHEL8 java-11-openjdk-devel Bouncy Castle FIPS Security Provider. I get a JCEKS error. Here is my setup (see the  error at the end, Thanks!):

[root@local-rhel8 wildfly-26.1.3.Final]# fips-mode-setup --check
FIPS mode is enabled.

[root@local-rhel8 wildfly-26.1.3.Final]# grep BouncyCastle /usr/lib/jvm/java-11-openjdk/conf/security/java.security
fips.provider.1=org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider
fips.provider.2=org.bouncycastle.jsse.provider.BouncyCastleJsseProvider fips:BCFIPS

[root@local-rhel8 wildfly-26.1.3.Final]# grep bcfks /usr/lib/jvm/java-11-openjdk/conf/security/java.security
fips.keystore.type=BCFKS

Add the Bouncy Castle FIPS jars to the bootclasspath as found here: https://issues.redhat.com/browse/WFLY-16186
[root@local-rhel8 wildfly-26.1.3.Final]# export JAVA_OPTS="$JAVA_OPTS -Xbootclasspath/a:/shared/bcfips/bc-fips-debug-1.0.2.4.jar:/shared/bcfips/bctls-fips-1.0.17.jar:/shared/bcfips/bcpkix-fips-1.0.7.jar"

Start the server
[root@local-rhel8 wildfly-26.1.3.Final]# bin/standalone.sh  > /dev/null 2>&1 &

Add the Bouncy Castle FIPS modules
[root@local-rhel8 wildfly-26.1.3.Final]# bin/jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect
[standalone@localhost:9990 /] module add --name=org.bouncycastle.fips --resources=/shared/bcfips/bc-fips-debug-1.0.2.4.jar:/shared/bcfips/bctls-fips-1.0.17.jar:/shared/bcfips/bcpkix-fips-1.0.7.jar
[standalone@localhost:9990 /] /subsystem=elytron/provider-loader=bc:add(module=org.bouncycastle.fips)
{"outcome" => "success"}

[standalone@localhost:9990 /] /subsystem=elytron:write-attribute(name=initial-providers,value=bc)
{
    "outcome" => "success",
    "response-headers" => {
        "operation-requires-reload" => true,
        "process-state" => "reload-required"
    }
}

[standalone@localhost:9990 /] reload
[standalone@localhost:9990 /] exit


Create the keystore: SUCCESS
[root@local-rhel8 wildfly-26.1.3.Final]# keytool -genkeypair -alias tc -dname "cn=localhost" -keypass Password1! -keystore ~/bcfips.keystore -storepass Password1! -validity 720 -storetype BCFKS -providername BCFIPS -providerclass org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider -providerpath /shared/bcfips/bc-fips-debug-1.0.2.4.jar -v
Generating 2,048 bit RSA key pair and self-signed certificate (SHA256withRSA) with a validity of 720 days
        for: CN=localhost
[Storing /root/bcfips.keystore]

Create the secret key entry: SUCCESS
[root@local-rhel8 wildfly-26.1.3.Final]# keytool -genseckey -alias init -keypass Password1! -keyalg AES -keysize 256 -keystore ~/bcfips.keystore -storepass Password1! -storetype BCFKS -providername BCFIPS -providerclass org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider -providerpath /shared/bcfips/bc-fips-debug-1.0.2.4.jar -v
Generated 256-bit AES secret key
[Storing /root/bcfips.keystore]

List the keystore entries
[root@local-rhel8 wildfly-26.1.3.Final]# keytool -list -keystore ~/bcfips.keystore -storepass Password1! -storetype BCFKS -providername BCFIPS -providerclass org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider -providerpath /shared/bcfips/bc-fips-debug-1.0.2.4.jar
Keystore type: BCFKS
Keystore provider: BCFIPS

Your keystore contains 2 entries

init, Dec 3, 2023, SecretKeyEntry,
tc, Dec 3, 2023, PrivateKeyEntry,
Certificate fingerprint (SHA-256): 1E:BF:2A:9C:99:5B:76:30:7A:8F:3D:7D:AC:62:2B:AF:5C:28:80:EA:40:92:02:E4:06:E9:C5:C2:3A:7A:F3:6F

Create credential store: FAIL
[root@local-rhel8 wildfly-26.1.3.Final]# bin/elytron-tool.sh credential-store -c -a init -x Password1! -l ~/bcfips.keystore -p Password1! -u "keyStoreType=BCFKS;keyAlias=init;external=true;externalPath=~/initstore" --debug
Exception encountered executing the command:
org.wildfly.security.credential.store.CredentialStoreException: ELY09514: Unable to initialize credential store
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.credential.store.impl.KeyStoreCredentialStore.getKeyStoreInstance(KeyStoreCredentialStore.java:978)
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.credential.store.impl.KeyStoreCredentialStore.setupExternalStorage(KeyStoreCredentialStore.java:988)
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.credential.store.impl.KeyStoreCredentialStore.load(KeyStoreCredentialStore.java:866)
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.credential.store.impl.KeyStoreCredentialStore.initialize(KeyStoreCredentialStore.java:224)
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.credential.store.CredentialStore.initialize(CredentialStore.java:160)
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.tool.CredentialStoreCommand.execute(CredentialStoreCommand.java:410)
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.tool.ElytronTool.main(ElytronTool.java:85)
        at org.jboss.modules.Module.run(Module.java:353)
        at org.jboss.modules.Module.run(Module.java:321)
        at org.jboss.modules.Main.main(Main.java:604)
Caused by: java.security.KeyStoreException: JCEKS not found
        at java.base/java.security.KeyStore.getInstance(KeyStore.java:878)
        at org.wildfly.secu...@1.19.1.Final//org.wildfly.security.credential.store.impl.KeyStoreCredentialStore.getKeyStoreInstance(KeyStoreCredentialStore.java:974)
        ... 9 more
Caused by: java.security.NoSuchAlgorithmException: JCEKS KeyStore not available
        at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
        at java.base/java.security.Security.getImpl(Security.java:733)
        at java.base/java.security.KeyStore.getInstance(KeyStore.java:875)
        ... 10 more

[Prarthona Paul]

Hello Nate, 

Right off the bat, I see that your error messages say "JCEKS not found" and "JCEKS KeyStore not available" as opposed to "BCFKS not found" and "BCFKS KeyStore not available", which tells me that your properties are probably not being read properly. The default keystore type assigned is JCEKS if keyStoreType is not specified. Perhaps looking more into why this is happening would be useful in your case. 
My guess is you cannot initialize the credential store successfully due to this keystore type mismatch. 

Please feel free to follow up if you have any other questions. 

-- Prarthona
 

[Nate]

I am setting the properties on the command line as you can see above:  -u "keyStoreType=BCFKS;keyAlias=init;external=true;externalPath=~/initstore"

Also, I looked at the elytron code and JCEKS is hardcoded in there if external=true. Is this not a bug? Code: https://github.com/wildfly-security/wildfly-elytron/blob/b9d3be1853e66209d1ed15057b5d067d4116e9f9/credential/store/src/main/java/org/wildfly/security/credential/store/impl/KeyStoreCredentialStore.java#L989

[Prarthona Paul]

Hello Nate, 

You are right, the external credential stores are defaulted to JCEKS. This is why you were getting the JCEKS related errors because the tool was looking for a JCEKS keystore. 

However, there is a way to get around it. You can use -Djava.security.properties parameter to specify the path to file in which they specify the SunJCE in the fips.providers as opposed to using the --properties or -u option. 
It would look like this: 

./bin/elytron-tool.sh credential-store ... -Djava.security.properties PATH/TO/FILE containing fips.providers

I hope that helps. Feel free to let me know if you have any questions.

-- Prarthona

[Nate]

That just gives me  "Unrecognized option"

[root@local-rhel8 wildfly-26.1.3.Final]# bin/elytron-tool.sh credential-store -Djava.security.properties /usr/lib/jvm/java-11-openjdk/conf/security/java.security -c -a init -x Password1! -l ~/bcfips.keystore -p Password1! -u "keyStoreType=BCFKS;keyAlias=init;external=true;externalPath=~/initstore" --debug

Exception encountered executing the command:

Unrecognized option: -Djava.security.properties

[Diana Krepinska]

Hi Nate,

problem is that the default fips.provider list is missing SunJCE provider which the credential-store needs. You can either add this provider to your fips.provider list in /usr/lib/jvm/java-11-openjdk/conf/security/java.security file, or a more advisable solution is to pass it only to elytron-tool via a custom java.properties.security file that would have a content like this:

fips.provider.1=org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider
fips.provider.2=org.bouncycastle.jsse.provider.BouncyCastleJsseProvider fips:BCFIPS
fips.provider.3=SunJCE
fips.provider.4=SUN
fips.provider.5=SunEC
fips.provider.6=com.sun.net.ssl.internal.ssl.Provider

To pass this file to elytron-tool successfuly you can modify the JAVA_OPTS properties:

JAVA_OPTS="$JAVA_OPTS -Djava.security.properties=/home/example/my.java.security.properties"

and then when you use the same command:

./bin/elytron-tool.sh credential-store -c -a init -x Password1! -l ~/bcfips.keystore -p Password1! -u "keyStoreType=BCFKS;keyAlias=init;external=true;externalPath=~/initstore" --debug

it should work for you.