Re: Error thrown by s3_pkt.c when connecting via flash sockets with socket.io over SSL

[Dr. Stephen Henson]

On Fri, Sep 28, 2012, Justin Meltzer wrote:

> Hello everyone,
>
> My company is running into a problem which has been causing us a lot of
> strife. We're using socket.io to connect a cross-domain client to our
> node.js server over flash sockets using SSL encryption. Unfortunately, one
> of the OpenSSL files seems to be throwing an error preventing the
> connection from being established. The crux of the problem is explained
> here:
> http://stackoverflow.com/questions/11571517/https-error-data-length-too-long-in-s3-pkt-c-from-socket-io
>
> I'd be very grateful if anyone could point me in the right direction.
>

Have you tried setting SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER?

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openss...@openssl.org
Automated List Manager majo...@openssl.org

[Jaaron Anderson]

Try SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER

I think its included in SSL_OP_ALL, which you can specify by supplyin
"-bugs" to s_client

[Justin Meltzer]

So I noticed that SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER is set in ssl.h and that the file that was throwing the error (s3_pkt.c) was including a file which then includes ssl.h. It seems that there is an IF statement in s3_pkt.c that checks for SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER and s->options, and if both are true, sets variable extra = SSL3_RT_MAX_EXTRA.

I wondered if maybe this if statement was not getting executed. So in the if statement that checks

if (rr->length > SSL3_RT_MAX_PLAIN_LENGTH+extra)

and if true, it throws the "data length to long" error, I changed it to:

if (rr->length > SSL3_RT_MAX_PLAIN_LENGTH+extra+0x00000080L)

and this seems to no longer throw the error.

However, it seems that packets that are very long tend to get broken up and truncated, and it's causing an error to surface up to my application code. It's proving very difficult to find the source of this error, but it's definitely truncating the data ("or at least only showing the first section of a large amount of data in my logs")

Where should I be looking next to debug this?