Sending BSON (binary json alternative) data with Socket.io ?

[Tom]

Hi,

Is it possible to use a BSON implementation like MessagePack (http://
msgpack.org/ and https://github.com/pgriess/node-msgpack) as a
transport protocol rather than JSON with Socket.io?

Has anything been done in this field?

Any feedback is appreciated.

Regards,
Tom

[Brian McKelvey]

Websocket at least sends data as utf-8 so it would have to be encoded somehow anyway, likely as base64 or some such.

Brian

Sent from my iPhone

[Clifton King]

BSON looks great for encoding typically sent from server-to-server as
JSON (memcached session data, publish/subscribe messages, value of a
redis key, etc). However, I doubt anything but a native browser
implementation would benefit users receiving BSON in a browser via
websocket or otherwise.

Clifton

[Tom]

@Clifton, the benefit is much higher efficiency thus allowing more users to connect (as less resources are used at any point in time on average).

- Tom

2011/4/21 Clifton King <clif...@gmail.com>

[Clifton King]

I don't know of any browser side implementations of BSON, though. You
can't send them BSON if they can't read it (and furthermore if
WebSocket enforces UTF-8 it won't work at all).

[Tom]

@Clifton,

Alright -- that's the answer to my question. A bit of a disappointment, I wonder why these things are never implemented fully.

Are there any JSON alternatives that also generate strings but provide more efficiency?

[Clifton King]

Well, that's a good question. How would you implement JSON, but more
efficiently, outputting to a string?

{} beginning and ending of a hash
[] beginning and end of an array
: separates keys and values
, separates items in an array

All tokens are 1 character. I don't think you can do any better than
that unless you go to a binary data format.

A better question is: Can V8's JSON parsing and stringifying functions
be further optimized?

[Tom]

"Can V8's JSON parsing and stringifying functions
be further optimized?"

This would seem likely. I tried JSONing about 3MB of binary data, and it took the same amount in seconds roughly. This sounds like a lot of time.

Thanks again.

Regards,

Tom

2011/4/21 Clifton King <clif...@gmail.com>

Well, that's a good question. How would you implement JSON, but more

efficiently, outputting to a string?

{} beginning and ending of a hash
[] beginning and end of an array
: separates keys and values
, separates items in an array

All tokens are 1 character. I don't think you can do any better than

that unless you go to abinary data format.

[Clifton King]

Can you not send the binary data as a buffer?

You could message that client that they *need* to get binary data
through JSON, then the client can make a normal GET or XHR request
directly to that data. That should yield much better performance so
long as you are leveraging nodejs's piping abilities.

Just a suggestion. I'm likely going to implement MessagePack soon in
our backend application for storing client session data (rather than
Marshal), so that our nodejs workers can use the same means of
authentication.

Clifton

[Tom]

@Clifton,

Yes, that is actually what I am doing now as a workaround.

Glad we've come up with the same solution. :)

Thanks again.