Received: by 10.236.181.234 with SMTP id l70mr9471322yhm.39.1349574817445; Sat, 06 Oct 2012 18:53:37 -0700 (PDT) X-BeenThere: nodejs@googlegroups.com Received: by 10.236.113.178 with SMTP id a38ls7667189yhh.9.gmail; Sat, 06 Oct 2012 18:53:26 -0700 (PDT) Received: by 10.236.74.161 with SMTP id x21mr1479842yhd.13.1349574806872; Sat, 06 Oct 2012 18:53:26 -0700 (PDT) Date: Sat, 6 Oct 2012 18:53:25 -0700 (PDT) From: Justin Meltzer To: nodejs@googlegroups.com Message-Id: <277b0d7b-3888-4e20-8c3a-9562acf466e6@googlegroups.com> In-Reply-To: References: <390e3708-7d1a-44c7-860a-ee2b92c041a5@googlegroups.com> Subject: Re: [nodejs] Potential Problem with Node.js Implementation of OpenSSL with large amounts of data MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1570_17085814.1349574805787" ------=_Part_1570_17085814.1349574805787 Content-Type: multipart/alternative; boundary="----=_Part_1571_2454328.1349574805787" ------=_Part_1571_2454328.1349574805787 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Looks like the _pusher method (either on the CleartextStream or EncryptedStream prototype, but I'm guessing it's the CleartextStream prototype at this point) is responsible for determining how much of the original buffer to read at one time. Maybe there's a bug originating from there? On Saturday, October 6, 2012 8:00:59 PM UTC-4, Ben Noordhuis wrote: > > On Sun, Oct 7, 2012 at 1:25 AM, Justin Meltzer > > wrote: > > I'm using node.js v. 0.8 and I'm running into a problem that has proven > to > > be incredibly difficult to debug. Any ideas or pointers on how to debug > this > > would be greatly appreciated. > > > > For a bit of background, this involves a Flash socket connection over > SSL in > > Internet Explorer 9. I'm using socket.io. Apparently, Internet Explorer > is > > known to send unusually large packet sizes over SSL, which is the reason > for > > the OpenSSL SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER option. Originally, > OpenSSL > > was throwing a "data length too long" error in the s3_pkt.c file for me, > and > > I received an answer on the openssl mailing list that I should make sure > > SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER is set: > > > https://groups.google.com/forum/?fromgroups=#!topic/mailing.openssl.users/GmFAOKGa4q4 > > > > So I dug into the OpenSSL source code and it seemed that the option was > > already being set, so I went to the line that was throwing the error and > > changed the relevant if statement so that it would never throw the > error. > > > > And now everything seems fine, and flash sockets connect to my SSL > server > > and can send small amounts of data back and forth. However, if I send a > > larger amount of data (for example, the entire html contents of a web > page) > > over the wire, it causes this error to fire in the default.js socket.io > > source: > > > > if (i === 0){ > > if (chr != '\u0000') > > this.error('Bad framing. Expected null byte as first frame'); > > else > > continue; > > } > > } > > > > It's sending over all of the data, but from what I can tell from the > logs, > > node splits up the data into multiple parts so that when the second part > > reaches that conditional, it does not have a null byte as the first > frame. > > > > I tried digging into tls.js to see if something fishy were going on > there, > > but I had little luck. All I could tell was that in the > > Cryptostream.prototype._push method, a "pool" buffer is sliced into > chunks, > > decoded, and then emitted to the server socket. Maybe it's incorrectly > > handling large amounts of data here, or perhaps the bug really is in the > > socket.io source code in how it handles this corner case? > > > > Any ideas would be great! Thanks! > > Neither node.js or the bundled openssl currently sets > SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER (or SSL_OP_ALL for that matter) so > that could well explain it. > > Does this patch[1] fix it? If not, what happens when you revert commit > f210530f[2]? > > [1] https://gist.github.com/fead5033fe2eed452252 > [2] https://github.com/joyent/node/commit/f210530f > ------=_Part_1571_2454328.1349574805787 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit Looks like the _pusher method (either on the CleartextStream or EncryptedStream prototype, but I'm guessing it's the CleartextStream prototype at this point) is responsible for determining how much of the original buffer to read at one time. Maybe there's a bug originating from there?

On Saturday, October 6, 2012 8:00:59 PM UTC-4, Ben Noordhuis wrote:
On Sun, Oct 7, 2012 at 1:25 AM, Justin Meltzer <jus...@airtimehq.com> wrote:
> I'm using node.js v. 0.8 and I'm running into a problem that has proven to
> be incredibly difficult to debug. Any ideas or pointers on how to debug this
> would be greatly appreciated.
>
> For a bit of background, this involves a Flash socket connection over SSL in
> Internet Explorer 9. I'm using socket.io. Apparently, Internet Explorer is
> known to send unusually large packet sizes over SSL, which is the reason for
> the OpenSSL SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER option. Originally, OpenSSL
> was throwing a "data length too long" error in the s3_pkt.c file for me, and
> I received an answer on the openssl mailing list that I should make sure
> SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER is set:
> https://groups.google.com/forum/?fromgroups=#!topic/mailing.openssl.users/GmFAOKGa4q4
>
> So I dug into the OpenSSL source code and it seemed that the option was
> already being set, so I went to the line that was throwing the error and
> changed the relevant if statement so that it would never throw the error.
>
> And now everything seems fine, and flash sockets connect to my SSL server
> and can send small amounts of data back and forth. However, if I send a
> larger amount of data (for example, the entire html contents of a web page)
> over the wire, it causes this error to fire in the default.js socket.io
> source:
>
> if (i === 0){
>     if (chr != '\u0000')
>         this.error('Bad framing. Expected null byte as first frame');
>     else
>         continue;
>     }
> }
>
> It's sending over all of the data, but from what I can tell from the logs,
> node splits up the data into multiple parts so that when the second part
> reaches that conditional, it does not have a null byte as the first frame.
>
> I tried digging into tls.js to see if something fishy were going on there,
> but I had little luck. All I could tell was that in the
> Cryptostream.prototype._push method, a "pool" buffer is sliced into chunks,
> decoded, and then emitted to the server socket. Maybe it's incorrectly
> handling large amounts of data here, or perhaps the bug really is in the
> socket.io source code in how it handles this corner case?
>
> Any ideas would be great! Thanks!

Neither node.js or the bundled openssl currently sets
SSL_OP_MICROSOFT_BIG_SSLV3_BUFFER (or SSL_OP_ALL for that matter) so
that could well explain it.

Does this patch[1] fix it? If not, what happens when you revert commit
f210530f[2]?

[1] https://gist.github.com/fead5033fe2eed452252
[2] https://github.com/joyent/node/commit/f210530f
------=_Part_1571_2454328.1349574805787-- ------=_Part_1570_17085814.1349574805787--