> This is my first mail to this Group. Consider me a novice in Openssl
> since I am just getting used to the API's and understanding the behavior.
> I am facing one problem. We have an MFP/Printer which acts as a client
> and is trying to connect to a server through a Secure device. When the
> client is connected to a Win XP machine(server), then it works fine. But
> the same client when connected to a Windows 2k8 Server, is not able to
> complete the SSL handshake.
What do you mean 'through a Secure device' -- what is actually connecting?
You mention APIs; is this a program you wrote or otherwise have source to?
> The flow is as follows in case of Windows XP :
<snip normal handshake, appdata both ways, alert both ways, FIN, repeat>
Those 'Encrypted Alert's are most likely 'close' which is completely
normal. Although we can't be certain because they're encrypted.
> The flow is as follows in case of Windows 2k8 :
<snip normal handshake>
> Client sends Application Data, Application Data.(TLSv1)
> [Different from Win XP flow]
> Server sends Encrypted Alert.(TLSv1)
> Client sends Encrypted Alert.(TLSv1)
> Client sends FIN,ACK.
> Server sends ACK
> Client sends RST,ACK and the connection closes.
The SSL handshake completed just fine; the 'Change Cipher Spec'
and 'Encrypted Handshake Message' (which is actually Finished)
in both directions, not followed immediately by an abort due to
misverify on the Finished, is the end of the handshake.
There is something different about the *application* data,
and since it's encrypted we can't look at that difference.
> I am not able to figure out why(in case of Win 2k8) the Client
> is sending Application Data twice and the server is not sending back
> any data but only encryption alert.
> And why is the connection not starting again.
The alert might be an actual SSL-level problem, or just a close,
but this time a server-initiated close, which might well be
abnormal at the application level.
What software is receiving this connection on the Windows end?
A Microsoft driver, an OEM driver, something else? Does it log
any useful info anywhere? Always try the Windows event logs
to start, and files in %windir% and %temp%. Some software
logs other places like root, under appdata, or even in
programfiles although Vista/2008+ discourages that.
You say the client is an MFP/Printer; devices in that class
typically don't display much detail on their (limited) UIs,
and I'm assuming if yours did you would have provided it.
Does it have any kind of 'remote' or 'network' admin feature
you can look at? Some do; it varies all across the board.
If you can control the cipher configuration, you might get
SSL to negotiate an 'eNULL' suite (with no data encryption)
and then your trace is readable. However, this is usually
prohibited by default, precisely because it is not secure,
so you would need to configure both ends, and this level
of control is very rare on devices, and not common on
'mass market' software not focussed on security.
If the server (Windows) is using or can use a 'kaRSA' suite
and you have or can get a copy of its private key (perhaps a
temporary one) and you trace with Wireshark, it reportedly
can use that private key to decrypt the data (and alert).
Using kaRSA is probably easy; it's often the default, and
almost always supported. Getting the key might be harder.
If the program uses the normal Windows keystore you might
or might not be able to export the key depending on how it
was created, but if not you can probably replace it with
a key you generated (easy to do with OpenSSL), if you have
control of the PC (e.g. it's not someone else's server).
If the program uses its own keystore, who knows.
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org