Polymorphic serialisation of Map values doesn't work?

[Marc Dzaebel]

Hi,

I defined a mapper that serializes Exception's polymorphically, but the following simple map serializes without @class attribute in its values:

Map<Integer, Exception> map = new HashMap<>();
map.put(1, new Exception("test"));

mapper.writeValueAsString(map) --> {"1":{"stackTrace":[...], ...}}  // missing @class

while e.g. Exception is serialized correctly:

mapper.writeValueAsString(new Exception("test")) --> {"@class":"java.lang.Exception", "stackTrace":[...], ...}

Is this intended or is there a setting, that enables transitive polymorphic serialisation in Map values? May be I have to write serializers/deserializers?

Here is a minimal example:

import java.util.HashMap;
import java.util.Map;

import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTypeResolverBuilder;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;

public class MapValues {
    public static void main(String[] args) throws JsonProcessingException {
        final ObjectMapper mapper = new ObjectMapper();
        DefaultTypeResolverBuilder typeResolver = new DefaultTypeResolverBuilder(DefaultTyping.NON_FINAL) {
            @Override public boolean useForType(JavaType t) {
                return Throwable.class.isAssignableFrom(t.getRawClass());
        }};
        typeResolver.init(JsonTypeInfo.Id.CLASS, null);
        typeResolver.inclusion(JsonTypeInfo.As.PROPERTY);
        mapper.setDefaultTyping(typeResolver);
        System.out.println("Excpetion:\n" + mapper.writeValueAsString(new Exception()));

        Map<Integer, Exception> map = new HashMap<>();
        map.put(Integer.valueOf(1), new Exception());
        System.out.println("\nMap:\n" + mapper.writeValueAsString(map));
    }
}

[Tatu Saloranta]

On Thu, Sep 12, 2019 at 11:27 AM Marc Dzaebel <mdza...@gmail.com> wrote:
>
> Hi,
>
> I defined a mapper that serializes Exception's polymorphically, but the following simple map serializes without @class attribute in its values:
>
> Map<Integer, Exception> map = new HashMap<>();
> map.put(1, new Exception("test"));
> mapper.writeValueAsString(map) --> {"1":{"stackTrace":[...], ...}} // missing @class
>
> while e.g. Exception is serialized correctly:
>
> mapper.writeValueAsString(new Exception("test")) --> {"@class":"java.lang.Exception", "stackTrace":[...], ...}
>
> Is this intended or is there a setting, that enables transitive polymorphic serialisation in Map values? May be I have to write serializers/deserializers?

This is bit of a FAQ, but the problem is Java Type Erasure: attempts
to serialize values of generic types as root values is problematic as
there is no generic runtime type available: all Jackson sees is a
`Map<?, ?>` (unlike with properties, where generic type is retained in
class file).

Because of this, I personally recommend not using root values that are
of generic type, and instead always use a Bean class as root value,
even if as simple wrapper around Map or List:

public class Response {
public Map<Integer, Exception> exceptions;
}

since `Response` is not generic, and it all will work fine.

There are 2 workarounds:

1. Use a non-generic helper type:

public class IntExceptionMap extends HashMap<Integer, Exception> { }

in this case full generic type info IS available (due to sub-classing)

2. Force type on serialization

String json = mapper.writerFor(new TypeReference<Map<Integer,
Exception>>() { })
.writeValueAsString(map)

I hope this helps,

-+ Tatu +-

[Marc Dzaebel]

Thanks Tatu,

good to know about both alternatives. I'll use you recommendation, the wrapper. I also found it on SO here, where they used the old/deprecaded mapper.writerWithType()

[Tatu Saloranta]

On Mon, Sep 16, 2019 at 1:27 AM Marc Dzaebel <mdza...@gmail.com> wrote:

Thanks Tatu,

good to know about both alternatives. I'll use you recommendation, the wrapper. I also found it on SO here, where they used the old/deprecaded mapper.writerWithType()

Right, `writerWithType()` was renamed as `writerFor()`.

-+ Tatu +-

 

--
You received this message because you are subscribed to the Google Groups "jackson-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/b401bc11-8ead-48d2-9684-335ac02d60fb%40googlegroups.com.

[Marc Dzaebel]

What's about deserializing polymorhic Json into an Object? E.g.:

mapper.readValue("{\"@class\":\"java.lang.Exception\",\"message\":\"Test\"}", Object.class).getClass()

will be a LinkedHashMap, rather than an Exception. Is there a way to achieve this? I know, that the resolver could always return true, but than I'll get "Missing type id" for those objects, that are not polymorphic.

Thanks a lot in advance

Marc

[Tatu Saloranta]

I am not quite sure what the question here is -- use of `Object` as
nominal type for readValue() works the same as any other type: if
polymorphic type information is expected (via default typing, or
resolver indicating it should), type id is searched for, polymorphic
handling enabled.
But If not, "natural" mapping is used, resulting in
Map/List/Number/String/Boolean.

There is one sort of related fix going in 2.10.0:

https://github.com/FasterXML/jackson-databind/issues/1093

for writer side, however. That resolves the case of

objectWriter.writerFor(Object.class).writeValue(....)

so that type information is included for all types if intended base
type is `Object` (similar to how `public Object value;` field would
work, instead of using actual runtime type for determination).

-+ Tatu +-

[Marc Dzaebel]

Thank's Tatu,

As you said, Jackson only deserializes polymorphically if the target type indicates it via resolver or default typing. However, if you e.g. serialize an object with Java writeObject and readObject again, the result has correct type and properties, which is because Java serialization always add's the type (which is possible for Jackson too, but ubiquitous, so much more complex than needed).

So a class, that holds heterogenous types in an Object property is not transferable, unless you globally configure, that all Objects have to be serialized with a type attribute. It's certainly possible to define Serializers/Deserializers, that achieve the needed polymorphic deserialization, but it could be much simpler, if there'd be an option at least for deserialization, that says, if there is a type attribute at the beginning, create this type, rather than a natural one. Of course, this could only be used in secured environments, but would ensure readable and compact JSON without the need to think about deserialization at all.

Marc

[Tatu Saloranta]

This is what "Default Typing" is about, and yes, has caused endless
grief with CVEs... :-D

I am not sure if you are familiar with it, but if not, you may want to
have a look -- in 2.9 and before, it's
"ObjectMapper.enableDefaultTyping()", and with 2.10 it will be
replaced with safer "activeDefaultTyping()" (which now requires
`PolymorphicTypeValidator` to validate that type is accepted).
Implementation-wise it is similar to adding `@JsonTypeInfo` as mix-in
to a wide category of types.

-+ Tatu +-

[Marc Dzaebel]

Hi Tatu,

1. Using only ObjectMapper#enableDefaultTyping() does not serialize into a polymorphic Json-String
2. Using only ObjectMapper#enableDefaultTyping() does not deserialize a polymorphic Json-String --> error
   Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Object

So the question remains. How to serialize/deserialize an Object polymorphically.

Here is the code, that demonstrates the problem:

import java.io.IOException;
import com.fasterxml.jackson.annotation.JsonTypeInfo;

import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTypeResolverBuilder;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;

public class MapValues2 {
    public static void main(String[] args) throws IOException, ClassNotFoundException {
        final ObjectMapper serializer = new ObjectMapper(), deserializer = new ObjectMapper();

        DefaultTypeResolverBuilder typeResolver = new DefaultTypeResolverBuilder(DefaultTyping.NON_FINAL) {
            @Override public boolean useForType(JavaType t) {
                return Throwable.class.isAssignableFrom(t.getRawClass());
        }};
        typeResolver.init(JsonTypeInfo.Id.CLASS, null);
        typeResolver.inclusion(JsonTypeInfo.As.PROPERTY);

        serializer.setDefaultTyping(typeResolver);
        deserializer.enableDefaultTyping();
        String polyJson = serializer.writeValueAsString(new Exception());
        System.out.println("Excpetion:\n" + polyJson);
        System.out.println("1: Read poly:" + deserializer.readValue(polyJson, Object.class).getClass()); // error
    }
}

Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Object
 at [Source: (String)"{"@class":"java.lang.Exception","cause":null,"stackTrace":["[Ljava.lang.StackTraceElement;",[{"@class":"java.lang.StackTraceElement","methodName":"main","fileName":"MapValues2.java","lineNumber":25,"className": ...

[Tatu Saloranta]

On Wed, Sep 25, 2019 at 12:35 AM Marc Dzaebel <mdza...@gmail.com> wrote:
>
> Hi Tatu,
>
> Using only ObjectMapper#enableDefaultTyping() does not serialize into a polymorphic Json-String
> Using only ObjectMapper#enableDefaultTyping() does not deserialize a polymorphic Json-String --> error
> Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Object
>
> So the question remains. How to serialize/deserialize an Object polymorphically.
>
> Here is the code, that demonstrates the problem:
>
> import java.io.IOException;
> import com.fasterxml.jackson.annotation.JsonTypeInfo;
> import com.fasterxml.jackson.databind.JavaType;
> import com.fasterxml.jackson.databind.ObjectMapper;
> import com.fasterxml.jackson.databind.ObjectMapper.DefaultTypeResolverBuilder;
> import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;
>
> public class MapValues2 {
> public static void main(String[] args) throws IOException, ClassNotFoundException {
> final ObjectMapper serializer = new ObjectMapper(), deserializer = new ObjectMapper();
> DefaultTypeResolverBuilder typeResolver = new DefaultTypeResolverBuilder(DefaultTyping.NON_FINAL) {
> @Override public boolean useForType(JavaType t) {
> return Throwable.class.isAssignableFrom(t.getRawClass());
> }};
> typeResolver.init(JsonTypeInfo.Id.CLASS, null);
> typeResolver.inclusion(JsonTypeInfo.As.PROPERTY);

^^^ why?

What is the purpose of custom TypeResolverBuilder here?
Especially one that ONLY adds type information for something declared
as `Throwable`.

No, this should not work the way code expects here.

Just use existing criteria.

> serializer.setDefaultTyping(typeResolver);
> deserializer.enableDefaultTyping();
> String polyJson = serializer.writeValueAsString(new Exception());
> System.out.println("Excpetion:\n" + polyJson);
> System.out.println("1: Read poly:" + deserializer.readValue(polyJson, Object.class).getClass()); // error
> }
> }
>
> Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Object
> at [Source: (String)"{"@class":"java.lang.Exception","cause":null,"stackTrace":["[Ljava.lang.StackTraceElement;",[{"@class":"java.lang.StackTraceElement","methodName":"main","fileName":"MapValues2.java","lineNumber":25,"className": ...

Now: while I am not sure I fully understand the question here, there
IS one important fix, included in 2.10.0.pr3:

https://github.com/FasterXML/jackson-databind/issues/1093

which should now support use of `Object.class` as the target type WITH
polymorphic deserialization.

So you may want to try your original code with 2.10.0.pr3. Same fix
will be in 2.10.0.
It would have prevented earlier usage.

-+ Tatu +-

[Marc Dzaebel]

Hi Tatu,

I checked out the latest version 2.10.0.pr3 and have the following results:

1. mapper.enableDefaultTyping() does not serialize (e.g. an Exception) polymorphically via following code:
   
• mapper.enableDefaultTyping(); // now deprecated
• System.out.println(mapper.writeValueAsString(new Exception("Test"))); // no type info in Json string
  
2. So in order to get type info in JSON I obviously need a TypeResolver (defined above) but then
1. mapper.readValue(json, Object.class) results in a LinkedHashMap instance 

So I tried to use the new activateDefaultTyping(..) and now the resulting object is of type Exception, as expected. So, up to 2.9.9 I did not find a way to serialize/deserialize e.g. an Exception into an Object polymorphically, but with the new version, it's possible.

Thanks, Marc

[Marc Dzaebel]

2.10.0.pr3:

ObjectMapper mapper = new ObjectMapper();

mapper.setDefaultTyping(new DefaultTypeResolverBuilder(DefaultTyping.NON_FINAL, new AllowTypeValidator()) {
    @Override public boolean useForType(JavaType t) { return true; }}});

mapper.writeValueAsString(new Exception("Test")); // error

java.lang.IllegalStateException: Cannot build, 'init()' not yet called
    at com.fasterxml.jackson.databind.jsontype.impl.StdTypeResolverBuilder.idResolver(StdTypeResolverBuilder.java:264)

So polymorphic serialisation can't be controlled, but by the 5 DefaultTyping strategies.

Thanks, Marc

[Marc Dzaebel]

Same problem without AllowTypeValidator() parameter in the above code.

[Tatu Saloranta]

On Thu, Sep 26, 2019 at 6:35 AM Marc Dzaebel <mdza...@gmail.com> wrote: 

Same problem without AllowTypeValidator() parameter in the above code.

--

Have you tried calling `init()` method as suggested by exception message?

That's what unit tests (see DefaultTypeResolver2472Test)

do, for example.

-+ Tatu +-

 

You received this message because you are subscribed to the Google Groups "jackson-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/b202e4f1-4ecd-4598-a4d8-6e2120769454%40googlegroups.com.

[Marc Dzaebel]

You're right, forgot init(), however, the following code prints "LinkedHashMap". If I uncomment setDefaultTyping(..) it prints "Exception" correctly. So still, polymorphic serialisation can't be controlled for an Object target type.

import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.annotation.JsonTypeInfo.As;

import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTypeResolverBuilder;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;

import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.jsontype.impl.LaissezFaireSubTypeValidator;

public class Main4 {
    public static void main(String[] args) throws java.io.IOException, ClassNotFoundException {
        ObjectMapper mapper = JsonMapper.builder().activateDefaultTyping(
                LaissezFaireSubTypeValidator.instance, DefaultTyping.NON_FINAL, As.PROPERTY).build();

        DefaultTypeResolverBuilder typeResolver =
                new DefaultTypeResolverBuilder(DefaultTyping.NON_FINAL) {

            @Override public boolean useForType(JavaType t) { return Throwable.class.isAssignableFrom(t.getRawClass()); } };
        typeResolver.init(JsonTypeInfo.Id.CLASS, null);
        typeResolver.inclusion(JsonTypeInfo.As.PROPERTY);
        mapper.setDefaultTyping(typeResolver); // comment this
        String json=mapper.writeValueAsString(new Exception("Test"));
        System.out.println(mapper.readValue(json, Object.class).getClass());  // prints LinkedHashMap

    }
}

By the way, you might replace NoCheckSubTypeValidator with LaissezFaireSubTypeValidator and delete NoCheckSubTypeValidator, as it it completely equivalent.

Thanks, Marc

[Tatu Saloranta]

On Thu, Sep 26, 2019 at 11:02 PM Marc Dzaebel <mdza...@gmail.com> wrote:
>
> You're right, forgot init(), however, the following code prints "LinkedHashMap". If I uncomment setDefaultTyping(..) it prints "Exception" correctly. So still, polymorphic serialisation can't be controlled for an Object target type.

No. There is still a problem with type matching, the way you wrote it:

> import com.fasterxml.jackson.annotation.JsonTypeInfo;
> import com.fasterxml.jackson.annotation.JsonTypeInfo.As;
> import com.fasterxml.jackson.databind.JavaType;
> import com.fasterxml.jackson.databind.ObjectMapper;
> import com.fasterxml.jackson.databind.ObjectMapper.DefaultTypeResolverBuilder;
> import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;
> import com.fasterxml.jackson.databind.json.JsonMapper;
> import com.fasterxml.jackson.databind.jsontype.impl.LaissezFaireSubTypeValidator;
>
> public class Main4 {
> public static void main(String[] args) throws java.io.IOException, ClassNotFoundException {
> ObjectMapper mapper = JsonMapper.builder().activateDefaultTyping(
> LaissezFaireSubTypeValidator.instance, DefaultTyping.NON_FINAL, As.PROPERTY).build();
> DefaultTypeResolverBuilder typeResolver =
> new DefaultTypeResolverBuilder(DefaultTyping.NON_FINAL) {
> @Override public boolean useForType(JavaType t) { return Throwable.class.isAssignableFrom(t.getRawClass()); } };

This ONLY enables polymorphic type handling for declared type of
`Throwable` but...

> typeResolver.init(JsonTypeInfo.Id.CLASS, null);
> typeResolver.inclusion(JsonTypeInfo.As.PROPERTY);
> mapper.setDefaultTyping(typeResolver); // comment this
> String json=mapper.writeValueAsString(new Exception("Test"));

type here is runtime type, `Exception` (you could force that with
`writerFor(type)`), and

> System.out.println(mapper.readValue(json, Object.class).getClass()); // prints LinkedHashMap

here you declare base type to be `Object`.
Neither is `Throwable`. This is why Type Id is not used, nor expected.

And I still do not understand quite why not simply enable default
typing for `DefaultTyping.NON_FINAL`: that should match both types
used here.
There is no reason to sub-class `DefaultTypeResolverBuilder`.

-+ Tatu +-

[Marc Dzaebel]

Tatu, thanks for your persistent help! If my program is not correct, could you post a sample (like in your new blog post) that simultaneously configures an ObjectMapper to:

• use JsonTypeInfo (without annotations) for certain types and their subclasses to disable polymorphic serialisation (if I use default typing)
  
• only use certain types and their subclasses to be serialized and deserialized polymorphically (with class attribute) even to Object target/generic types
• don't serialize Collections/Maps polymophically (only certain types)
  
• All instances of the resulting graph should be of same class as in the original graph (according to config)
  

Thanks, Marc

Following is an other approach, where Test0 should not be serialized/deserialized polymorphically, but it serializes Test0 which leads to error (if @JsonTypeInfo is not set):

import com.fasterxml.jackson.annotation.JsonTypeInfo.As;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.jsontype.BasicPolymorphicTypeValidator;
import com.fasterxml.jackson.databind.jsontype.PolymorphicTypeValidator;

public class Main4 {
//    @JsonTypeInfo(use = Id.NONE)     // with this, it works
    static class Test0  { public Object test1; }
    static class Test1  { public Object exception; }

    public static void main(String[] args) throws java.io.IOException, ClassNotFoundException {

        Test0 test0 = new Test0();
        Test1 test = new Test1();
        test.exception=new Exception("Test");
        test0.test1=test;
        PolymorphicTypeValidator ptv = BasicPolymorphicTypeValidator.builder().allowIfSubType(Test1.class).allowIfSubType(Exception.class).build();
        ObjectMapper mapper = JsonMapper.builder().activateDefaultTyping(ptv,
                DefaultTyping.NON_FINAL, As.PROPERTY).disable(SerializationFeature.FAIL_ON_EMPTY_BEANS).build();
        String json=mapper.writeValueAsString(test0);
        System.out.println(json);
        Test0 readValue = mapper.readValue(json, Test0.class);
        System.out.println(readValue.test1.getClass());
        System.out.println(readValue.getClass());
}}

[Marc Dzaebel]

@JsonTypeInfo could be introduced by Mixin or programmatically (which is needed) with JacksonAnnotationIntrospector. However, it's not clear how to implement this usecase.

[Tatu Saloranta]

On Mon, Sep 30, 2019 at 1:11 AM Marc Dzaebel <mdza...@gmail.com> wrote:

Tatu, thanks for your persistent help! If my program is not correct, could you post a sample (like in your new blog post) that simultaneously configures an ObjectMapper to:

• use JsonTypeInfo (without annotations) for certain types and their subclasses to disable polymorphic serialisation (if I use default typing)
  
• only use certain types and their subclasses to be serialized and deserialized polymorphically (with class attribute) even to Object target/generic types
• don't serialize Collections/Maps polymophically (only certain types)
  
• All instances of the resulting graph should be of same class as in the original graph (according to config)
  

Thanks, Marc

If I had unlimited time, I'd definitely do that, but to be honest I am not sure this is very common use case... there are so many things to document.

But I can help with specific issue here.

 

Following is an other approach, where Test0 should not be serialized/deserialized polymorphically, but it serializes Test0 which leads to error (if @JsonTypeInfo is not set):

import com.fasterxml.jackson.annotation.JsonTypeInfo.As;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.jsontype.BasicPolymorphicTypeValidator;
import com.fasterxml.jackson.databind.jsontype.PolymorphicTypeValidator;

public class Main4 {
//    @JsonTypeInfo(use = Id.NONE)     // with this, it works

Yes, `@JsonTypeInfo` override works: Default Typing is more general rule, annotation can override it.

 

    static class Test0  { public Object test1; }
    static class Test1  { public Object exception; }
    public static void main(String[] args) throws java.io.IOException, ClassNotFoundException {
        Test0 test0 = new Test0();
        Test1 test = new Test1();
        test.exception=new Exception("Test");
        test0.test1=test;
        PolymorphicTypeValidator ptv = BasicPolymorphicTypeValidator.builder().allowIfSubType(Test1.class).allowIfSubType(Exception.class).build();

Note that PTV is concerned with security aspect: is given type ever allowed to be handled via polymorphic deserialization or not. It is not used or relevant for question of whether Default Typing is enabled. While from user perspective these are related, from Jackson perspective these are orthogonal concepts.

So inclusion (and thereby expectation) of Polymorphic handling is simply driven by this:

 

        ObjectMapper mapper = JsonMapper.builder().activateDefaultTyping(ptv,
                DefaultTyping.NON_FINAL, As.PROPERTY).disable(

  ^^^ DefaultTyping.NON_FINAL

and `Test0` not being `final`, type id is written and is expected. You could add `final` in declaration to prevent that, or `@JsonTypeInfo`. Or, possible, change to use `DefaultTyping.JAVA_LANG_OBJECT`.

Or perhaps change `test1` and `exception` fields to have `@JsonTypeInfo` instead of DefaultTyping.

-+ Tatu +-