Re: Bad Record Mac
Nelson Bolyard
> I'm trying to create a FIPS 140-2 compliant SSL connection using the Sun
> JSSE (SunPKCS11) and NSS.
I suppose you mean a FIPS compliant TLS connection. TLS is SSL version 3.1
(or newer). FIPS 140 compliance requires the use of TLS.
Crypto questions are best asked in mozilla.dev.tech.crypto rather than
in mozilla.dev.security. I have redirected followups to m.d.t.crypto.
> In my code, I'm able to create a keystore that can pull my certificate (for
> 127.0.0.1) from the NSS db and display it's information. However, when I try
> to connect a client socket and a server socket, I receive this exception:
>
> javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
> at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
> at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:136)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1657)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:932)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1096)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1123)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1107)
> ...
>
> javax.net.ssl.SSLHandshakeException: bad handshake record MAC
> at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1591)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1549)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:847)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1096)
> at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:744)
> at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
> at java.io.InputStreamReader.read(InputStreamReader.java:167)
> at java.io.BufferedReader.fill(BufferedReader.java:136)
> at java.io.BufferedReader.readLine(BufferedReader.java:299)
> at java.io.LineNumberReader.readLine(LineNumberReader.java:182)
> ...
>
> I'm not sure what "Bad Record MAC" means, or how to go about fixing it.
It usually means that, at the end of the handshake, the client and the
server ended up with different keys, and the attempt to decrypt an encrypted
record received from the remote peer party produced a decrypted result that
is unintelligible.
> I thought that perhaps it was a hint that my certificate was corrupt,
No, there are different errors that occur when a cert is bad.
It might mean that your server is using the wrong private key,
not the one that corresponds to the public key in the cert.
> Does anyone have any suggestions on where to start looking? Is this a
> sign that I have NSS or certificates configured incorrectly, or should I
> be talking with the Sun gurus about my implementation?
Your message may get help from others in the mozilla.dev.tech.crypto
newsgroup (which is also the dev-tech-crypto AT lists.mozilla.org mailing
list). Sun Java gurus who best know the internals of the class
com.sun.net.ssl.internal.ssl.SSLSocketImpl may be able to provide more help.
Are you sure that your client and server are actually using NSS's PKCS#11
module? The exception stacks shown above provide no evidence that proves
or disproves that they do. Still, a common problem is that the software
is not using the provider that the developer imagines it is using.