Trying to implement a simple SCEP server using jSCEP - failing miserably

[Florent]

Hello fellow developer(s).
I am currently implementing an SCEP server using jscep as a basis.
I am using the latest code available on the SVN (r936). I use the
latest version of Eclipse for all my development tasks and I use Jetty
(embedded) to serve the HTTP requests (a bit overkill for what I'm
doing but well...).
I started from the example ScepServlet class available on the SVN and
implemented all the missing parts for my system.

First I am trying to get the 3 main operations (used by almost all
clients) to work : GetCACert, GetCACaps and PKIOperation-
>messageType=PKCSReq.
I got all of them to work but the last and most essential one: I
managed to use jscep to get the PKCS#10 certificate request from the
CertReq request. I also managed to create a certificate signed by my
CA using OpenSSL. However, the response I generate seems to be
invalid.
I tried several SCEP clients with my server. A lot of them give a
generic error message like "Bad SCEP response". However, sscep gives
me a rather meaningful output when trying to enroll: it tells me the
response has a valid PKCS#7 structure, but that the signature is
invalid. The exact error output is:

./sscep: server returned status code 200
./sscep: MIME header: x-pki-message
./sscep: valid response from server
./sscep: reading outer PKCS#7
./sscep: PKCS#7 payload size: 3233 bytes
./sscep: printing PEM fomatted PKCS#7
-----BEGIN PKCS7-----
-- Some Base64-encoded data --
-----END PKCS7-----
./sscep: PKCS#7 contains 1343 bytes of enveloped data
./sscep: verifying signature
./sscep: error verifying signature
10896:error:21071065:PKCS7 routines:PKCS7_signatureVerify:digest
failure:pk7_doit.c:948:

Below you will find the part of my code that deals with building the
response. I'm sorry I had to cut some parts to make it shorter so
maybe a few variable definitions are missing.

res.setHeader("Content-Type", "application/x-pki-message");
InputStream msgInputStream;
String msgString;
if (reqMethod == GET && (msgString = req.getParameter(MSG_PARAM))!=
null) {
msgInputStream = new ByteArrayInputStream(new
Base64().decode(msgString));
} else {
msgInputStream = req.getInputStream();
}
ContentInfo msgContentInfo = getContentInfo(msgInputStream);
PkiMessage msg = msgParser.parse(msgContentInfo);
MessageType msgType = msg.getMessageType();
ASN1Encodable msgData = msg.getPkcsPkiEnvelope().getMessageData();

SignedData msgSignedData =
SignedData.getInstance(msgContentInfo.getContent());
Certificate reqSignatureCert =
SignedDataUtil.extractCertStore(msgSignedData).getCertificates(new
X509CertSelector()).iterator().next();
final PKCS10CertificationRequest certReq = new
PKCS10CertificationRequest((DERSequence)msgData);
final List<X509Certificate> certs = doEnroll(certReq);
final X509Certificate caCert = doGetCaCertificate("").get(0);
final PrivateKey caKey = ScepCAPrivateKey.getInstance(); //
Custom class to get the CA key

PkiMessageGenerator pkiMessGenerator = new PkiMessageGenerator();
pkiMessGenerator.setTransactionId(msg.getTransactionId());
pkiMessGenerator.setMessageType(MessageType.CertRep);
pkiMessGenerator.setSenderNonce(Nonce.nextNonce());
pkiMessGenerator.setRecipientNonce(msg.getSenderNonce());
pkiMessGenerator.setCipherAlgorithm("DES");
pkiMessGenerator.setMessageDigest("SHA-1");
pkiMessGenerator.setPrivateKey(caKey);
pkiMessGenerator.setSigner(caCert);
if (!certs.isEmpty()) {
SignedDataGenerator dsdGenerator = new SignedDataGenerator();
for (X509Certificate cert : certs) {
dsdGenerator.addCertificate(cert);
}
SignedData dsd = dsdGenerator.generate();
pkiMessGenerator.setMessageData(dsd);
pkiMessGenerator.setPkiStatus(PkiStatus.SUCCESS);
pkiMessGenerator.setRecipient((X509Certificate)
reqSignatureCert);
} else {
pkiMessGenerator.setPkiStatus(PkiStatus.FAILURE);
pkiMessGenerator.setFailInfo(FailInfo.badRequest);
}
PkiMessage responseMess = pkiMessGenerator.generate();

//Dump output for analysis
FileOutputStream dumpOut = new FileOutputStream("/tmp/
scepDump.ber");
dumpOut.write(responseMess.getEncoded("BER"));
dumpOut.close();
res.getOutputStream().write(responseMess.getDEREncoded());

The output I dump at the end before sending the data to the client is
valid PKCS#7 according to openssl (it can't do much with it though).

In the long term I intend to build a more object-oriented structure
(CertRep and CertRepGenerator classes could be useful) but that's it
for now. I studied in detail the current SCEP draft and a little of
the PKCS#7 standard.
According to the info that I have, it seems the "message digest" isn't
calculated correctly. It could also be the encryption of this digest
message (I use the public key from the request message's signature as
asked by the SCEP draft). This hash is created in the
org.jscep.pkcs7.PkiMessageGenerator class (getSignerInfo() method) but
to me it seems everything is done correctly there.
Could someone who has worked with jscep's code (the main developer or
someone else) give me some insight on how I could solve this issue? Is
it because the library hasn't been fully developed with SCEP servers
in mind but rather clients? (that's how it looks like to me)

Thanks in advance and sorry for the long message.

[David Grant]

Hi Florent,

I'm sorry to hear that you've been having problems.

Throughout development, the primary focus of jSCEP has been a client-
side implementation. However, the long-term vision is that it should
be equally usable for server- and client-side implementations.

Can you provide a formatted dump of the PKCS#7 ASN.1 structure?

Dave

[Hiro2k]

I noticed that it says it couldn't verify the signature. Maybe that
means your client hasn't added the signer certificate to its
truststore.

[Florent]

Hi. Thanks for the reply!

I managed to solve this problem by using classes in the
org.bouncycastle.cms package to generate my server response. These
classes handle encryption and signatures for a SignedData object
(actually CMSSignedData) automatically so I don't need to use the
org.jscep.pkcs7.PkiMessageGenerator class that doesn't work for me. So
to me it looks like in jscep you "reinvented the wheel" a little bit
by writing code to do that.
I might post another message in a short time but I am quite busy right
now so goobye for now.

Florent

[David Grant]

Hi Florent,

jSCEP uses the bcprov library for constructing PKCS#7 objects instead
of the CMS support that comes with the bcmail objects. When you get
time, could you create an issue in the jSCEP bug tracker so this
problem can be managed and has visibility for others?

Dave

[Santiago]

Hi All,

I'm running into a similar problem to the one described by Florent in
this thread. Using a PkiMessageGenerator to create a CertRep response
results in a client error. In my case the client (an iPhone) error
message is "Cannot retrieve SCEP identity". My code looks similar to
the one posted earlier in this thread.

Have tried a number of different ways to generate the CertRep without
any luck. I'm fairly certain the problem is not in the certificate
being returned (in pkcsPKIEnvelope) but in the pkiMessage returned
itself.

Has anyone found a solution to this problem using
PkiMessageGenerator? Florent: Could you post your solution using BC
instead?

Thanks.

-- Santiago

> ...
>
> read more »

[Florent]

I can't post this code here because my company won't allow it. I work
in a very competitive business that is not very keen on sharing
code... I believe the MIT licence allows me to create private
derivatives from jSCEP's code though. Tell me if I'm wrong. :-p
I can still explain a little more how I developed a working SCEP
server based on jSCEP (I work with iPhones too).
I had to completely rewrite the message generation algorithm to use
other classes from Bouncy Castle. The main classes I used are in the
org.bouncycastle.cms package : CMSEnvelopedData,
CMSEnvelopedDataGenerator, CMSException, CMSProcessable,
CMSProcessableByteArray, CMSSignedData, CMSSignedDataGenerator. And in
the org.bouncycastle.asn1 package : DERObjectIdentifier,
DEROctetString, DERPrintableString, DERSet.
In order to build the message, you will have to read this RFC :
http://www.faqs.org/rfcs/rfc2315.html and of course the SCEP draft
( http://tools.ietf.org/id/draft-nourse-scep-20.txt ).
You could also have a look at EJBCA's SCEP code.
The Bouncy Castle classes above will automatically handle things like
the signature and the encryption of the contents once you provide them
with all the needed information.

I hope this will be of some help.

Florent

PS: I'm still not sure if the problem I encountered before that with
jSCEP was caused by a mistake in jSCEP's implementation of the
protocol if it was because of my use of it. All I know is that the
signature that was created was considered invalid by several clients.

[David Grant]

Hi Florent,

On 3 August 2010 20:50, Florent <fvig...@gmail.com> wrote:
> I can't post this code here because my company won't allow it. I work
> in a very competitive business that is not very keen on sharing
> code... I believe the MIT licence allows me to create private
> derivatives from jSCEP's code though. Tell me if I'm wrong. :-p

No, you're absolutely correct in this. I chose MIT because it is the
most permissive license offered by Google Code.

> I can still explain a little more how I developed a working SCEP
> server based on jSCEP (I work with iPhones too).
> I had to completely rewrite the message generation algorithm to use
> other classes from Bouncy Castle. The main classes I used are in the
> org.bouncycastle.cms package : CMSEnvelopedData,
> CMSEnvelopedDataGenerator, CMSException, CMSProcessable,
> CMSProcessableByteArray, CMSSignedData, CMSSignedDataGenerator. And in
> the org.bouncycastle.asn1 package : DERObjectIdentifier,
> DEROctetString, DERPrintableString, DERSet.
> In order to build the message, you will have to read this RFC :
> http://www.faqs.org/rfcs/rfc2315.html and of course the SCEP draft
> ( http://tools.ietf.org/id/draft-nourse-scep-20.txt ).
> You could also have a look at EJBCA's SCEP code.
> The Bouncy Castle classes above will automatically handle things like
> the signature and the encryption of the contents once you provide them
> with all the needed information.

I have been deliberately avoiding the bcmail package to try to reduce
the footprint of jSCEP, but if the implementation in jSCEP is faulty,
it might be time to reassess that decision.

> PS: I'm still not sure if the problem I encountered before that with
> jSCEP was caused by a mistake in jSCEP's implementation of the
> protocol if it was because of my use of it. All I know is that the
> signature that was created was considered invalid by several clients.

It's very likely that there are problems in the jSCEP implementation,
simply because there hasn't been much testing in this area. If you
could find a way to contribute your changes back, I'd be grateful.
Contact me off list if this is an avenue that can be explored.

Dave

[Joe]

Hi All,

I'm running into a the same (?) problem as the one described by
Florent in this thread using a iPhone as a Client. I tried the latest download
(1.1.3) and I still have the issue.

Not very familiar with the BC crypto code so am unsure of how to
address.

Has this been addressed and am I looking at a new problem? Does anyone
have any specific changes I can try?

Thanks,
Joe