Serializing binary types to a JSON string

[Eric Dingle]

I'm working on a CL that uses base::JSONWriter to serialize some values to a JSON string. These values can be of any type of base::Value and can be/have binary types nested in them. The problem I'm having is this:

http://codesearch.google.com/#OAMlx_jo-ck/src/base/json/json_writer.cc&exact_package=chromium&l=186

Binary value types can't be serialized to JSON in our current implementation. I was thinking of computing the 64 byte digest of the binary data and using that as the serialized value. Obviously, you wouldn't be able to deserialize the value back to the original binary data.

Any thoughts or "No! Don't do that"s?

Thanks,

Eric

[Evan Martin]

On Wed, Aug 3, 2011 at 10:36 AM, Eric Dingle <ericd...@chromium.org> wrote:
> I'm working on a CL that uses base::JSONWriter to serialize some values to a
> JSON string. These values can be of any type of base::Value and can be/have
> binary types nested in them.

I think you left out the important info of "why" right here. :)

JSON is not well-suited to transporting or storing binary data. We
have protocol buffers in the tree and they might be a better fit for
your task.

> The problem I'm having is this:
> http://codesearch.google.com/#OAMlx_jo-ck/src/base/json/json_writer.cc&exact_package=chromium&l=186
> Binary value types can't be serialized to JSON in our current
> implementation. I was thinking of computing the 64 byte digest of the binary
> data and using that as the serialized value. Obviously, you wouldn't be able
> to deserialize the value back to the original binary data.
> Any thoughts or "No! Don't do that"s?
> Thanks,
> Eric
>

> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev
>

[Jói Sigurðsson]

If you actually have to use JSON and you need to be able to also
deserialize, you could e.g. Base64-encode the values. But I agree
with Evan, it's important to know why you want this.

Cheers,
Jói

[Eric Dingle]

I'm working on a feature for extension power users that would allow them to enable extension activity logging. For each extension that this is enabled on, extension API calls and their arguments will be logged to a web UI to give the user more information about what an extension is doing. I want to serialize the arguments to JSON as a simple way of displaying them to the user. In this case, I don't need to deserialize the values.

Eric

[Evan Martin]

Ah, so what you really have are extension API calls, which are
composed of JavaScript objects?

I wonder what the developer console does in this circumstance?

[Antony Sargent]

> Binary value types can't be serialized to JSON in our current implementation.

From my brief reading of json.org it doesn't look like JSON supports binary values. 

value -> string | number | object | array | true | false | null

object -> list of string -> value mappings

array -> ordered collection of values

That NOTREACHED() in json_writer.cc should probably be a CHECK(false) instead, or maybe we should add an argument to JSONWriter that controls whether to simply ignore binary values or consider their presence in the Value being serialized a fatal error. 

There is "BSON" which supports binary values: http://bsonspec.org/, but again if you're going to use a different format, you should probably just look at protocol buffers which we already have support for. 

Also, I'm guessing you already know this and it's why you're asking the question in the first place, but some of the arguments to extension API's are not just simple javascript data types that are JSON-serializable, but handles to C++ objects in the DOM.

[Eric Dingle]

I guess it just seemed strange to me that we have a JSONStringValueSerializer class (which uses JSONWriter internally) that can't serialize all Values. As for providing a parameter to JSONWriter to ignore binary values for now, that seems like a plausible hack that would unblock me.

Protocol buffers do seem like a cleaner way to do it, but my guess is that we don't have a protocol buffer definition for Values. Would my only option here be to create this definition myself?

Eric

[Evan Martin]

On Wed, Aug 3, 2011 at 2:18 PM, Eric Dingle <ericd...@chromium.org> wrote:
> I guess it just seemed strange to me that we have a
> JSONStringValueSerializer class (which uses JSONWriter internally) that
> can't serialize all Values.

Here's one way to look at it: you can convert arbitrary JSON into a
Value and back, but you can't convert an arbitrary Value into JSON and
back. Values are a superset of JSON objects.

[Eric Dingle]

Would adding a boolean parameter to the JSONWriter constructor to allow binary values (false by default) and encoding binary values to base64 if that parameter is true be a crazy idea?

[Jói Sigurðsson]

If you don't need to decode the binary values, and you don't need to
identify them either (e.g. as a hash over the binary blob) then I'd
say having a flag that simply leaves unknown value types out of the
JSON would make more sense. The default behavior should still be to
fail on unknown value types.

Cheers,
Jói

[Antony Sargent]

I agree with Joi. 

I'd suggest you do a preprocessing step on your input Value where you convert any BinaryValue nodes into a base64-encoded StringValue (perhaps adding on some special "magic" prefix to the string if you want to be able to distinguish these from regular strings wherever you consume the json), and then pass that to JSONWriter.