Re: Netty + Jackson/Smile, any byte I could use as a delimiter?

[Sunny Gleason]

I think 0xFF is available for this purpose? It seems to

be the case based on the WRITE_END_MARKER option

in Smile...

https://github.com/FasterXML/jackson-dataformats-binary/blob/master/smile/src/main/java/com/fasterxml/jackson/dataformat/smile/SmileConstants.java

Newlines could also work, except that they are

in the 4-byte smile header, IIRC, which is ":)\nX",

where X is the version + flags byte. I think setting the

option to disable the header (and making sure 7-bit

binary coding is enabled so newlines don't enter that

way) would make "\n" work as well?

(I'll definitely defer to Tatu's guidance though, he is the

encoding master!)

All the best,

-Sunny

On Fri, Aug 19, 2016 at 8:58 AM, Pierre Larger <pierre...@gmail.com> wrote:

Hi,

TLDR: Any byte I could use to split the network input so Jackson always has whole objects to work on?

I would like to use Smile in one of my experimental projects involving Netty and Jackson. Prior to Smile usage, I felt necessary to split the input to deserialize so Jackson always had the whole object to work on and I used the Netty's Delimiters.lineDelimiters ('\r' and '\n'). Could be better but it worked.
I encountered issues while trying to use Smile and removed the delimiters-based frame splitting. Surprisingly, it worked pretty well, except for some frames that seem to be cut off from the beginning so I was thinking about restoring the delimiter-based frame splitting with a different delimiter. Any idea of which one to use?

Not a native English speaker, feel free to fix my writing skills :)

--
You received this message because you are subscribed to the Google Groups "smile-format-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smile-format-discussion+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tatu Saloranta]

Yes, 0xFF is the "official" split marker to use; reading of one results in `null` being returned as token as presumable document end. There are couple of other bytes that similarly are not used (reserved); I'd have to re-read spec, but those are something like 0xFE, 0xFD and 0xFC (or some of them).

But I specifically did want to reserved these bytes, and especially 0xFF, for framing. So this sounds like a good use case.

The only minor caveat is that if binary data is included, you have to use "7-bit encoding" to ensure these bytes can not be embedded as literal data. If so, encoder guarantees avoidance of said bytes.

It is also possible to prevent writing of Smile header with one of `SmileGenerator.Feature`s, if linefeed there was problematic. However, linefeeds can be legitimately encoded for uses other than embedded linefeed in String.

-+ Tatu +-

To unsubscribe from this group and stop receiving emails from it, send an email to smile-format-discussion+unsubscr...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.