Best practices in implementing custom binary protocol

[yakov...@gmail.com]

Hello vert.x community!

I'm new to this framework, but I have a very concrete task for it. The task is to create a proxy between a custom binary protocol and a web socket. While everything is more or less clear with web sockets, I'm a bit lost for the other protocol. It's a TCP-based protocol that introduces a 'message' notion. The message is prefixed with a constant (so that it can be easily checked if the message start is valid), and then it contains length-prefixed string of arbitrary length. Just like this:

[4 bytes header containing length][... contents of length as in header...]

So the thing is, in vert.x I only have a 'dataHandler' callback, and I, if I understand correctly, I'll receive all the streamed TCP data in it, chunk by chunk, as if I'd read it from the TCP socket directly. I'm looking for a best practice in how to progressively read the incoming data, finding the message and payload boundaries correctly.

I hope my explanation makes sense, but please feel free to ask for more details if it's necessary to answer the question.

Thanks,
Roman.

[Jez P]

You might want to look at the code for RecordParser in the vert.x source. While it doesn't solve your problem, it may show you the best approach to plumbing in your code. You'd still have to code up your custom boundary detection, however.

[yakov...@gmail.com]

Hey, thanks for the quick reply. It's strange to me that vert.x doesn't have anything better, because Netty has. I mean the Netty's pipeline, where the data flows between the handlers, every of which can parse incoming chunk and then either decide to wait for more data or pass the resulting POJO further. I guess, it's just not the vert.x's main focus to support such low level things.

 But thanks for the RecordParser anyway, I'll use it a basis to implement my own handler, and then see if it makes sense to continue with vert.x or better use Netty directly.

воскресенье, 20 июля 2014 г., 20:53:34 UTC+2 пользователь Jez P написал:

[Tim Fox]

On 20/07/14 20:05, yakov...@gmail.com wrote:

Hey, thanks for the quick reply. It's strange to me that vert.x doesn't have anything better,

I'm curious what don't you like about RecordParser? Imho it's simpler to use than Netty pipelines as you just tell it your delimeter or frame size and it spits out the records for you...

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Fox]

BTW here is an example of how RecordParser is used in the event bus to parse a binary protocol from a TCP socket - each frame is prefixed by an int, which gives the length of the packet - basically exactly what you want to do:

https://github.com/eclipse/vert.x/blob/master/vertx-core/src/main/java/io/vertx/core/eventbus/impl/EventBusImpl.java#L252

...

[Jez P]

Hi Tim,

I don't think RecordParser handles Roman's scenario: "length-prefixed string of arbitrary length" - ie the length of the payload can vary from message to message. AIUI he effectively needs to roll his own RecordParser-style implementation to spit out the records.

...

[Tim Fox]

On 20/07/14 20:22, Jez P wrote:

Hi Tim,

I don't think RecordParser handles Roman's scenario: "length-prefixed string of arbitrary length"

Really? This is exactly what we do in the event bus. Unless I don't understand the requirement...

[Jez P]

Cool, that's neat, I hadn't thought of changing the "fixed size" size on the fly like that.

[yakov...@gmail.com]

Hey, thanks Tim! I also didn't understand initially that you might switch between parser modes on the fly. Will try it out. Thanks for help!

воскресенье, 20 июля 2014 г., 21:19:51 UTC+2 пользователь Tim Fox написал: