Odd protocol. Frame/Message decoding ideas requested.

Skip to first unread message

Gerald Runion

Feb 20, 2021, 2:09:39 AM2/20/21
to Netty discussions
I have odd protocol where responses to commands are of the form

case 1: <STX><ASCII DATA><ETX><Checksum>
case 2: <0x06>
case 3: <0x15>

which is proving to be a bit challenging to do with Netty without doing a bunch of byte by byte copying. I have my naive ByteToMessageDecoder read one byte and upon inspection of the byte decide how to progress. My 'frame' that I send up to the next layer in the pipeline is a nio buffer which either contains a single byte (cases 2 or 3) or the entirety of the message (<STX>....<Checksum>) for case 1.

This works fine and since I a dealing with a serial transport with a single connection that kind of highly iterative copying is probably of no real concern. However, this doesn't feel like it is idiomatic Netty and I was looking for other interesting solutions.

I suppose one solution would be dynamically add a handler to the pipeline in the event that the first byte in a message was an <STX> and then removing that handler once it had consumed <checksum>.

I could also see, if such a thing is possible, a pair of sibling (same order in the pipeline) decoders: a LineBasedFrameDecoder that terminates with <ACK> or <NACK>, and a subclass of LineBasedFrameDecoder that terminates on <ETX> but also consumes and extra byte.

So I am curious what approaches folks with more experience with Netty than I would come up with.


Feb 20, 2021, 2:05:47 PM2/20/21
to Netty discussions
A common approach to protocols like this (or others where you may have different versions of the protocol that are parsed differently), is to have a simple handler that detects which version of the protocol is in use, then replaces itself with the appropriate protocol specific handler.

Take a look at the socksx package for an example of a handler that detects the version of the protocol in use, and adds an appropriate ChannelHandler:

Hope this helps.


Gerald Runion

Mar 19, 2021, 3:20:42 PM3/19/21
to Netty discussions
This worked. Since the context of what type of message I expected is tightly coupled the message I am sending I load the pipeline with the appropriate decode before sending my request.

public byte[] sendTextResponseRequest(String command, String payload, long timeout, TimeUnit timeUnit) throws InterruptedException, TimeoutException, ExecutionException {
try {
TextPayloadResponseDecoder decoder = new TextPayloadResponseDecoder();
channel.pipeline().addBefore(PiePipelineManager.class.getName(), TextPayloadResponseDecoder.class.getName(), decoder);

sendRequest(command, payload);
return decoder.getFuture().get(timeout, timeUnit);
} finally {

Reply all
Reply to author
0 new messages