Proposal for JSON field mapping
David Symonds
with types you may not own.
And sometimes, in such a situation, you wish to use different key
names in JSON than the struct.
Since you don't own the type, you can't add a field tag; further,
someone might already be using that field tag in some other way.
Here's a proposal to introduce more flexibility:
// in package json
type NameMapper interface {
// ToJSON returns the JSON key for a struct field name, map key or ...
ToJSON(fieldName string) (jsonKey string)
FromJSON(jsonKey string) (fieldName string)
}
When encoding or decoding JSON, if the data type implements
NameMapper, the json package will use those methods in preference to
any field tags.
An alternative interface would be
type NameMapper interface {
JSONNameMap() map[string]string // fieldName => jsonKey
}
with very similar semantics, though you lose some efficiency unless
you then returned two maps (or two methods with one map each) for the
inverse mapping.
A further option is to have an alternate json.Marshal/json.Unmarshal
function pair that takes a map. That kinda sucks because you've got to
have a map on hand, and it doesn't work naturally for recursive types.
Thoughts? Any other options?
Dave.
Andrew Gerrand
type FieldMatcher interface {
MatchField(dst, src string) bool
}
Where it takes an arbitrary source and destination name and returns
whether they match. This would allow a more "fuzzy" matching of names.
David Symonds
> Another idea. What about this:
>
> type FieldMatcher interface {
> MatchField(dst, src string) bool
> }
>
> Where it takes an arbitrary source and destination name and returns
> whether they match. This would allow a more "fuzzy" matching of names.
That's not going to help. It's a concrete mapping function that's
needed; on the encode side you simply don't have the dst field, and on
the decode side this becomes O(N^2).
Dave.
Andrew Gerrand
> That's not going to help. It's a concrete mapping function that's
> needed; on the encode side you simply don't have the dst field, and on
> the decode side this becomes O(N^2).
Yeah. To be honest, I wasn't thinking about it too hard about it when
I suggested it. There's a lesson there.
Andrew
Russ Cox
> It comes up, from time to time, that you wish to use the json package
> with types you may not own.
This isn't enough justification. Everything comes up from time to time,
but we can't put in custom code for everything.
What if I want to encode a protocol buffer using different field numbers?
That probably comes up from time to time too.
Marshal and Unmarshal already have well-defined, generic hooks.
I would rather see those continue to be used when customization
is necessary. If that means copying the data structure fields into
a separate struct to pass to marshal, that doesn't seem so bad for
an occasional need.
Russ
David Symonds
> What if I want to encode a protocol buffer using different field numbers?
> That probably comes up from time to time too.
I haven't seen that come up myself, and I suspect it's a very niche
situation. However, it's quite the norm for JSON libraries to support
alternate key names, and for people to use them. The Go JSON package
supports that; it's just not supported in a convenient form for some
uses.
> Marshal and Unmarshal already have well-defined, generic hooks.
> I would rather see those continue to be used when customization
> is necessary. If that means copying the data structure fields into
> a separate struct to pass to marshal, that doesn't seem so bad for
> an occasional need.
It seems quite inefficient, too. And, if you want to do it generically
(e.g. for protocol buffers) then you must resort to either a large
chunk of generated code, or copy the data into a map with all the
pursuant tedium of type conversion that the JSON package already does
quite nicely. And you have to do that for every single
marshal/unmarshal operation.
This applies to any type that isn't under one's control. MarshalJSON
and UnmarshalJSON are great for adding to one's own small types, but
it's a lot of overhead if you want to convert a type in another
package into JSON, and you just want to rename the fields.
Convenience is not a trump card, but it does weigh on scales, and the
only question is whether its contribution outweighs its drawbacks. The
drawbacks here are pretty minor on the API side (it would be almost
invisible), and pretty light on the efficiency side (an extra
interface conversion in a few places). As for the benefits, it would
make generic JSON conversion of protocol buffers straightforward, and
I suspect other uses would be found. I'm hoping someone else will
chime in for the latter.
Dave.
Russ Cox
> only question is whether its contribution outweighs its drawbacks. The
> drawbacks here are pretty minor on the API side (it would be almost
> invisible)
Almost invisible is much more expensive than not there.
You are heavily discounting the cost of reading about them
in the documentation (which you typically have to do to
decide you can ignore them) and the cost of maintaining
them forever, which constrains future implementations.
The current JSON semantics and API allow an implementation
that could do a precompilation phase the same way gob does,
if speed ever became an issue. The proposed NameMapper
is so general that a struct could choose different key names
every time you encode it, which eliminates any possibility of
precompilation.
Russ
Ian Lance Taylor
> // in package json
>
> type NameMapper interface {
> // ToJSON returns the JSON key for a struct field name, map key or ...
> ToJSON(fieldName string) (jsonKey string)
> FromJSON(jsonKey string) (fieldName string)
> }
>
> When encoding or decoding JSON, if the data type implements
> NameMapper, the json package will use those methods in preference to
> any field tags.
Rather than expressing this programatically, what if you express it in
the tags? Presumably the struct you are encoding is itself a field in
some enclosing struct. Define a tag format for a struct field which
provides the names not only of the current field, but also of the fields
of the struct value.
This does still carry a documentation weight. I don't know how often
this issue actually arises in practice.
Ian
Kyle Lemons
type NameMapper interface {
// ToJSON returns the JSON key for a struct field name, map key or ...
ToJSON(fieldName string) (jsonKey string)
FromJSON(jsonKey string) (fieldName string)
}