Json deep merge array processing strategy

57 views
Skip to first unread message

Dzmitry Sankouski

unread,
Oct 26, 2018, 6:32:56 PM10/26/18
to jackson-user
I have two Jsons I want to merge. Arrays in json should be also merged, not concatenated.

Example:
Json 1:
{
  "level1": {
    "value": "a",
    "anotherValue": "z",
    "array1": [
      {
        "value": "b",
        "anotherValue": "z"
      }
    ]
  }
}


Json 2:
{
  "level1": {
    "value": "c",
    "array1": [
      {
        "value": "d"
      }
    ]
  }
}

After merge, I want to get:
{
  "level1": {
    "value": "c",
    "anotherValue": "z",
    "array1": [
      {
        "value": "d",
        "anotherValue": "z"
      }
    ]
  }
}


Using code from this suggestion about json merge, I got array1 not merged, but concatenated:
{
  "level1": {
    "value": "c",
    "anotherValue": "z",
    "array1": [
      {
        "value": "b",
        "anotherValue": "z"
      },
      {
        "value": "d",
        "anotherValue": "z"
      }
    ]
  }
}


How do I achieve arrays in json being merged, not concatenated?

Tatu Saloranta

unread,
Oct 28, 2018, 9:00:00 PM10/28/18
to jackso...@googlegroups.com
Merging for arrays does mean concatenation (append for Collections),
and nothing else. This because while there are many possible ways
users might
want to handle merging of two sequence values, it is surprisingly
difficult to figure out declarative way that covers enough without
being very complicated.

So: instead of trying to figure out a complex solution that might not
cover all cases, I went with simple, predictable solution, which does
not cover all cases.

In future it could be possible to extend `@JsonMerge` to perhaps allow
custom handler of some sort that would allow custom logic, but this
does not yet exist.

-+ Tatu +-
> --
> You received this message because you are subscribed to the Google Groups "jackson-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
> To post to this group, send email to jackso...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Saurin Joshi

unread,
Apr 6, 2021, 7:28:41 PM4/6/21
to jackson-user
Apologies for opening a zombie thread. I am also interested in deep merge of 2 arrays based on index.

If I have 2 JSON strings JSON string1 and JSON string2 as below, then I would like the output merged property like Merged JSON

JSON string1:
{
    "property1": [
        {
            "property2": "value1",
        },
        {
            "property2": "value2",
        }
    ]
}

JSON string2:
{
    "property1": [
        {
            "property3": "value3"
        },
        {
            "property3": "value4"
        }
    ]
}

Merged JSON:
{
    "property1": [
        {
              "property2": "value1",
              "property3": "value3"
        },
        {
              "property2": "value2",
              "property3": "value4"
        }
    ]
}

As one of the solutions, I tried registering custom deserializer (which can look at array index position for merging) by extending CollectionDeserializer like below. This code gets executed but the custom deserializer doesn't get called while deserializing my class. Jackson still calls CollectionDeserializer.

    private static void registerDeserializer(ObjectMapper mapper) {

        SimpleModule module = new SimpleModule();

        module.setDeserializerModifier(new BeanDeserializerModifier() {

        @Override 

        public JsonDeserializer<?> modifyCollectionDeserializer(DeserializationConfig config, CollectionType type, BeanDescription beanDesc, JsonDeserializer<?> deserializer) {

            if (List.class.isAssignableFrom(beanDesc.getBeanClass()) && deserializer instanceof CollectionDeserializer) {

                CollectionDeserializer collectionDeserializer = (CollectionDeserializer)deserializer;

                if (LIST_CLASSES.stream().anyMatch(clazz -> clazz.isAssignableFrom(type.getContentType().getRawClass()))) {

                    return new CustomCollectionDeserializer(type, nullnull, collectionDeserializer.getValueInstantiator());

                } 

            }

            return deserializer;

        }});

        mapper.registerModule(module);

    }


 public static <T> T mergeObjects(T obj1, String obj2) throws JsonMappingException, JsonProcessingException {

        ObjectReader objectReader = mapper.readerForUpdating(obj1);

        return objectReader.readValue(obj2);

    }


Can you help me with:
- how to register custom CollectionDeserializer, so that it gets invoked? or
- suggest if there are better ways to achieve index based deep merge of 2 arrays

Regards,
Saurin

Tatu Saloranta

unread,
Apr 6, 2021, 7:33:14 PM4/6/21
to jackson-user
I would recommend against trying to implement or extend
CollectionDeserializer. Getting that right requires a lot of work.
There is no way to change definition of merging for Arrays currently
and it seems unlikely there will be much support
given that there seem to be multiple things users may want to and no
obvious simple configuration options to cover options
(or at least I have not been able to think of such).

I think most users would read content as JsonNode and implement
merging on those instances with relatively
simple traversal: this allows you to use exact rules you want.
You can then convert (mapper.treeToValue()) resulting tree into actual
value type you want (and vice versa).

-+ Tatu +-

Saurin Joshi

unread,
Apr 6, 2021, 11:48:06 PM4/6/21
to jackson-user
Thanks Tatu for prompt response and suggestion.

I could solve the 1st issue by overriding withResolved() method and got CustomCollectionDeserializer working as described below. However, I see the complications it will bring in for maintaining this class. I will instead go with your suggested approach to do manipulation directly on JSON node and deserialize the merged object.

public class CustomCollectionDeserializer extends CollectionDeserializer {

    public CustomCollectionDeserializer(JavaType collectionType, JsonDeserializer<Object> valueDeser, TypeDeserializer valueTypeDeser, ValueInstantiator valueInstantiator) {

        super(collectionType, valueDeser, valueTypeDeser, valueInstantiator);

    }

    protected CustomCollectionDeserializer(JavaType collectionType, JsonDeserializer<Object> valueDeser, TypeDeserializer valueTypeDeser, ValueInstantiator valueInstantiator, JsonDeserializer<Object> delegateDeser, NullValueProvider nuller, Boolean unwrapSingle) {

        super(collectionType, valueDeser, valueTypeDeser, valueInstantiator, delegateDeser, nuller, unwrapSingle);

    }

    @Override

    protected CollectionDeserializer withResolved(JsonDeserializer<?> dd, JsonDeserializer<?> vd, TypeDeserializer vtd,

            NullValueProvider nuller, Boolean unwrapSingle) {

        return new CustomCollectionDeserializer(_containerType, (JsonDeserializer<Object>) vd, vtd, _valueInstantiator, (JsonDeserializer<Object>) dd, nuller, unwrapSingle);

    }

    @Override

    public Collection<Object> deserialize(JsonParser p, DeserializationContext ctxt, Collection<Object> intoValue)

            throws IOException {

        return super.deserialize(p, ctxt, intoValue);

    }

}

Regards,
Saurin

Tatu Saloranta

unread,
Apr 7, 2021, 12:38:53 PM4/7/21
to jackson-user
On Tue, Apr 6, 2021 at 8:48 PM Saurin Joshi <joshi....@gmail.com> wrote:
>
> Thanks Tatu for prompt response and suggestion.
>
> I could solve the 1st issue by overriding withResolved() method and got CustomCollectionDeserializer working as described below. However, I see the complications it will bring in for maintaining this class. I will instead go with your suggested approach to do manipulation directly on JSON node and deserialize the merged object.

Right: it is technically perfectly doable, it's just that sub-classing
(and inheritance more generally) is a bit fragile,
and over time CollectionDeserializer constructors have and will be
changing when new configuration settings need
to be passed. Old versions are kept as per SemVer, where possible, but
there will be less testing to try to ensure that
things actually fully work as expected with new default/placeholder
values -- tests typically do not cover @Deprecated
methods, fields or constructors

This may be fine when you control the Jackson version in use and can
work around possible complications wrt upgrades.
But I like to mention it as a general suggestion.

Good luck!

-+ Tatu +-
> --
> You received this message because you are subscribed to the Google Groups "jackson-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/0422e123-991b-4101-b615-ba489a0a27f2n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages