Potential Problem with Node.js Implementation of OpenSSL with large amounts of data

[Justin Meltzer]

I'm using node.js v. 0.8 and I'm running into a problem that has proven to be incredibly difficult to debug. Any ideas or pointers on how to debug this would be greatly appreciated.

For a bit of background, this involves a Flash socket connection over SSL in Internet Explorer 9. I'm using socket.io. Apparently, Internet Explorer is known to send unusually large packet sizes over SSL, which is the reason for the OpenSSL SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER option. Originally, OpenSSL was throwing a "data length too long" error in the s3_pkt.c file for me, and I received an answer on the openssl mailing list that I should make sure SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER is set: https://groups.google.com/forum/?fromgroups=#!topic/mailing.openssl.users/GmFAOKGa4q4

So I dug into the OpenSSL source code and it seemed that the option was already being set, so I went to the line that was throwing the error and changed the relevant if statement so that it would never throw the error.

And now everything seems fine, and flash sockets connect to my SSL server and can send small amounts of data back and forth. However, if I send a larger amount of data (for example, the entire html contents of a web page) over the wire, it causes this error to fire in the default.js socket.io source:

if (i === 0){

    if (chr != '\u0000')

        this.error('Bad framing. Expected null byte as first frame');

    else

        continue;

    }

}

It's sending over all of the data, but from what I can tell from the logs, node splits up the data into multiple parts so that when the second part reaches that conditional, it does not have a null byte as the first frame.

I tried digging into tls.js to see if something fishy were going on there, but I had little luck. All I could tell was that in the Cryptostream.prototype._push method, a "pool" buffer is sliced into chunks, decoded, and then emitted to the server socket. Maybe it's incorrectly handling large amounts of data here, or perhaps the bug really is in the socket.io source code in how it handles this corner case?

Any ideas would be great! Thanks!

[Ben Noordhuis]

Neither node.js or the bundled openssl currently sets
SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER (or SSL_OP_ALL for that matter) so
that could well explain it.

Does this patch[1] fix it? If not, what happens when you revert commit
f210530f[2]?

[1] https://gist.github.com/fead5033fe2eed452252
[2] https://github.com/joyent/node/commit/f210530f

[Justin Meltzer]

Thanks Ben!

Patch[1] gets rid of the "data length too long" error, and is better than the hack I did. However, it does not fix the breaking up of data.

Reverting commit f210530f unfortunately does not prevent the data from being broken up either.

Any other ideas?

[Justin Meltzer]

Looks like the _pusher method (either on the CleartextStream or EncryptedStream prototype, but I'm guessing it's the CleartextStream prototype at this point) is responsible for determining how much of the original buffer to read at one time. Maybe there's a bug originating from there?

On Saturday, October 6, 2012 8:00:59 PM UTC-4, Ben Noordhuis wrote:

[Ben Noordhuis]

On Sun, Oct 7, 2012 at 2:50 AM, Justin Meltzer <jus...@airtimehq.com> wrote:
> Thanks Ben!
>
> Patch[1] gets rid of the "data length too long" error, and is better than
> the hack I did. However, it does not fix the breaking up of data.
>
> Reverting commit f210530f unfortunately does not prevent the data from being
> broken up either.
>
> Any other ideas?

I'm afraid not.

I can land a patch that turns on SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER but
I can't tell if the socket.io error is a bug in socket.io or node.

Try contacting the socket.io authors - if it turns out it's a bug in
node, they can forward the bug report to us.

[Justin Meltzer]

Aww shucks :/

Out of curiosity, how does the _push method on the Cryptostream prototype know where one message ends and the other begins?

It seems to me that the while/do loop continues to call _pusher (which calls SSL_read under the hood) until either the buffer is all used up or the SSL_read function can't read anything (it returns -1). Then it'll emit a data event only if it has actually read a non-negative number of bytes.

When I was debugging, it occurred to me that maybe the data was coming in slowly in chunks so that after it had read the first one or two chunks from SSL_read, the next tick of the while loop fired so quickly that it returned -1 before the next chunk of data had reached the C buffer. Then it emitted this first part of the data as if it were its own isolated grouping. Is this possible?

[Ben Noordhuis]

On Sun, Oct 7, 2012 at 11:34 PM, Justin Meltzer <jus...@airtimehq.com> wrote:
> Aww shucks :/
>
> Out of curiosity, how does the _push method on the Cryptostream prototype
> know where one message ends and the other begins?

It doesn't, it feeds openssl encrypted data until decrypted data pops
out at the other side (or vice versa, of course).

> It seems to me that the while/do loop continues to call _pusher (which calls
> SSL_read under the hood) until either the buffer is all used up or the
> SSL_read function can't read anything (it returns -1). Then it'll emit a
> data event only if it has actually read a non-negative number of bytes.
>
> When I was debugging, it occurred to me that maybe the data was coming in
> slowly in chunks so that after it had read the first one or two chunks from
> SSL_read, the next tick of the while loop fired so quickly that it returned
> -1 before the next chunk of data had reached the C buffer. Then it emitted
> this first part of the data as if it were its own isolated grouping. Is this
> possible?

I'm not sure if I'm following you. Encryption and decryption in the
current implementation are synchronous. That while loop in lib/tls.js
doesn't get split over multiple ticks of the event loop.

For the record, SSL and TLS work like this:

1) they are frame based
2) a frame contains one or more messages
3) a message is either a protocol message or data

Frames and messages can span multiple TCP packets. If you feed openssl
input that contains only partial messages, or protocol messages and no
data, nothing comes out at the other side.