Fastest way to implement zero copy Strings...

[Kevin Burton]

So lets say you have a network protocol where you're getting in JSON or other data structures and string data.

Assuming you don't have to unpack the data, whats the fastest way to get the bytes[] into a String.

I don't think there's a zero copy way to do this...

Java implements Strings as char[] internally...

It seems a shame to have to convert them.

One solution could be to not use String but use your own string-like objects.

[Reynold Xin]

In Spark, we implemented a simple UTF8String using unsafe: https://github.com/apache/spark/blob/master/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jonathan Yu]

You could also ensure that the JSON is sent over the wire in UTF-16 [0], which has a number of advantages: confusing your colleagues, justifying increased network capacity, and enabling zero-copy into normal Strings.

On a more serious note, there's work on the JDK to address this, though it's not usable yet today: http://openjdk.java.net/jeps/254 (perhaps thank your friendly neighbourhood OpenJDK contributor and buy them a beer)

[0] Although JSON's de-facto standard encoding is UTF-8, UTF-16 and UTF-32 are permissible as well: https://tools.ietf.org/html/rfc7159#section-8.1

--

Jonathan Yu / @jawnsy on LinkedIn, Twitter, GitHub, Facebook

“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.” — Samuel Beckett, Worstward Ho (1983) 

“In an adaptive environment, winning comes from adapting to change by continuously experimenting and identifying new options more quickly and economically than others. The classical strategist's mantra of sustainable competitive advantage becomes one of serial temporary advantage.” — Navigating the Dozens of Different Strategy Options (HBR)

[Nitsan Wakart]

Note that cooking your own String means giving up on ALLOT of great optimizations that are baked as intrinsics into the String class (i.e copying the code will get you nowhere).

So unless you ONLY mean to use these handrolled Strings for moving bytes around you may well end up losing on this trade off.

[Kevin Burton]

Yeah. I'm aware of the intrinsics... This stuff is hard because there are a ton of variables to get right.