Jackson polymorphic type (de)serialization with JsonIdentityInfo

[Alex T]

Hi everyone!

When serializing entity with many references to another same entity, if that entity marked @JsonIdentityInfo, first occurence of entity serialized as full entity, other occurrences are serialized as id. Its work.

I need to serialize entities with type information, so I use polymorphic type handler.

I created test project: https://github.com/alex-t0/deserialization-fail-example.git

Please see this classes:

1. User - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/main/java/deserialization/fail/example/User.java
   
2. UserPair - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/main/java/deserialization/fail/example/UserPair.java
3. MapperUtil - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/main/java/deserialization/fail/example/MapperUtil.java
4. UserSerializationTest - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/test/java/deserialization/fail/example/UserSerializationTest.java

Serialization works as I expect:

[
    "deserialization.fail.example.UserPair",
    {
        "user1": [
            "deserialization.fail.example.User",
            {
                "id": [
                    "java.lang.Long",
                    42
                ],
                "login": "aaa"
            }
        ],
        "user2": 42
    }
]

Second occurence is integer value 42. But when deserializing I get this exception:

com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (VALUE_NUMBER_INT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.lang.Long
 at [Source: (String)"["deserialization.fail.example.UserPair",{"user1":["deserialization.fail.example.User",{"id":["java.lang.Long",42],"login":"aaa"}],"user2":42}]"; line: 1, column: 140] (through reference chain: deserialization.fail.example.UserPair["user2"])

So, jackson not understand user2 value as integer.

Is is my improper use/misconfiguration? Or it is a bug?

Workaround?

[Tatu Saloranta]

On Mon, May 11, 2020 at 9:52 AM Alex T <nnm....@gmail.com> wrote:
>
> Hi everyone!
>
> When serializing entity with many references to another same entity, if that entity marked @JsonIdentityInfo, first occurence of entity serialized as full entity, other occurrences are serialized as id. Its work.
> I need to serialize entities with type information, so I use polymorphic type handler.
>
> I created test project: https://github.com/alex-t0/deserialization-fail-example.git
>
> Please see this classes:
>
> User - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/main/java/deserialization/fail/example/User.java
> UserPair - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/main/java/deserialization/fail/example/UserPair.java
> MapperUtil - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/main/java/deserialization/fail/example/MapperUtil.java
> UserSerializationTest - https://github.com/alex-t0/deserialization-fail-example/blob/master/src/test/java/deserialization/fail/example/UserSerializationTest.java
>
> Serialization works as I expect:
>
> [
> "deserialization.fail.example.UserPair",
> {
> "user1": [
> "deserialization.fail.example.User",
> {
> "id": [
> "java.lang.Long",
> 42
> ],

^^^^^^^^^

This is wrong. Id should be plain number (`42`) and suggests the most
likely problem.
It should never be wrapped in type information (in fact, not even
normal `long`/`Long` properties, being
one of small number of "natural" types).

Definition JsonIdentityInfo info is the problem in this case:

@JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "id",
scope = User.class
)

... because you are trying to use an actual physical field (`private
long Id`), but this definition would
try to generate new value and assumes there is no "real" field.
For using fields, you'd need to use
`ObjectIdGenerators.PropertyGenerator` (but then id must be populated
by your code).

Alternatively you should either just remove the field (if not needed),
or, if needed for Hibernate or such,
add `@JsonIgnore` on it.
But assuming this Id value comes from DB (sequence?), the first option
is probably what you want.

Combination of Hibernate module and JSON Identity might cause other
issues as well, since unfortunately Hibernate module
is not maintained and its support for Proxy types seems to cause
problems with some Jackson features.
Another possibility just for testing might be to try to not register
that module, just to isolate problematic component.

I hope this helps,

-+ Tatu +-

[Alex T]

Hi Tatu!

For now I solve issue changing from mapper.activateDefaultTyping(ptv, ObjectMapper.DefaultTyping.EVERYTHING); to
mapper.activateDefaultTyping(ptv, ObjectMapper.DefaultTyping.NON_FINAL);

Please see https://github.com/alex-t0/deserialization-fail-example/commit/5c1e9f73a21567c04823145993ffd09c314a276e#diff-590456d68d9f15ce57513829910aa370.

Now it does not wrap user id. And deserialization works fine.

But I'm not understand what you mean about @JsonIgnore'ing or removing my id field. It is a primary key and I need this field. If I add @JsonIgnore to id field I get this error:

Caused by: java.lang.IllegalArgumentException: Invalid Object Id definition for deserialization.fail.example.User: cannot find property with name 'id'
    at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.constructObjectIdHandler(BeanSerializerFactory.java:491)
    at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.constructBeanOrAddOnSerializer(BeanSerializerFactory.java:414)
    at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.findBeanOrAddOnSerializer(BeanSerializerFactory.java:286)
    at com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:231)
    at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:165)
    at com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1474)
    at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1422)
    ... 100 more

PS. Can you reemove my second duplicate thread https://groups.google.com/forum/#!topic/jackson-user/MK8RLm8Unu0?

[Tatu Saloranta]

On Thu, May 14, 2020 at 7:38 AM Alex T <nnm....@gmail.com> wrote:

Hi Tatu!

For now I solve issue changing from mapper.activateDefaultTyping(ptv, ObjectMapper.DefaultTyping.EVERYTHING); to
mapper.activateDefaultTyping(ptv, ObjectMapper.DefaultTyping.NON_FINAL);

Please see https://github.com/alex-t0/deserialization-fail-example/commit/5c1e9f73a21567c04823145993ffd09c314a276e#diff-590456d68d9f15ce57513829910aa370.

Now it does not wrap user id. And deserialization works fine.

That makes sense. Actually having Type Id for `Long` is fine in itself, but it would not have worked as type id most likely. And there can not be any polymorphism anyway in that location.

But I'm not understand what you mean about @JsonIgnore'ing or removing my id field. It is a primary key and I need this field. If I add @JsonIgnore to id field I get this error:

What I mean is that combination of settings you had was incompatible. You need to either:

1. Let Jackson generate Object Id by itself and NOT have id field in your POJO, OR

2. Use an existing field as Object Id and make sure Jackson uses that field and does NOT generate Object id.

For second, I already said that...

"For using fields, you'd need to use  `ObjectIdGenerators.PropertyGenerator`"

so change generator type to that, from what you had.

-+ Tatu +-

 

--
You received this message because you are subscribed to the Google Groups "jackson-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/52f9385a-3185-439d-a1f0-c1f881587bfc%40googlegroups.com.

[maxxyme]

Sorry to jump into the conversation, but wasn't it what he has been doing in his User class already?

(using ObjectIdGenerators.PropertyGenerator in his JsonIdentityInfo definition)

Thanks for any explanation.

maxxyme

To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAGrxA27a8GjJ6mgybdJTYcNqYpbJ38%3Daa7uYgOrgBVtWBD75MA%40mail.gmail.com.

[Tatu Saloranta]

On Sat, May 16, 2020 at 12:41 AM maxxyme <max...@gmail.com> wrote:
>
> Sorry to jump into the conversation, but wasn't it what he has been doing in his User class already?
> (using ObjectIdGenerators.PropertyGenerator in his JsonIdentityInfo definition)
>
> Thanks for any explanation.
> maxxyme

No problem at all jumping to help. Looking at code now, you are right:
PropertyGenerator is being used.
Earlier I thought I saw it use `IntSequenceGenerator`, which would not
have worked.

So that part of my suggestion was misleading.

Object Id being encapsulated with Type Id would have been problematic,
either way, although I am not
sure what the remaining problem then would be.

-+ Tatu +-

> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAEEu3mBTO7X0PomkYEfAWiXGsFxchuHu3kpX%3Dy8BjiCxSZJ71A%40mail.gmail.com.

[Alex T]

> Object Id being encapsulated with Type Id would have been problematic,
> either way, although I am not
> sure what the remaining problem then would be.

I understand correctly that ObjectId in json must be "raw" value? "Raw" I mean not wrapped.

So, seems using ObjectMapper.DefaultTyping.NON_FINAL did the trick.

But I have latest question: is ObjectMapper.DefaultTyping.EVERYTHING really useful? What is case of using it?

>>> To unsubscribe from this group and stop receiving emails from it, send an email to jackso...@googlegroups.com.

>>> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/52f9385a-3185-439d-a1f0-c1f881587bfc%40googlegroups.com.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "jackson-user" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to jackso...@googlegroups.com.

>> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/CAGrxA27a8GjJ6mgybdJTYcNqYpbJ38%3Daa7uYgOrgBVtWBD75MA%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups "jackson-user" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to jackso...@googlegroups.com.

[Tatu Saloranta]

On Sun, May 17, 2020 at 1:54 PM Alex T <nnm....@gmail.com> wrote:
>
> > Object Id being encapsulated with Type Id would have been problematic,
> > either way, although I am not
> > sure what the remaining problem then would be.
>
> I understand correctly that ObjectId in json must be "raw" value? "Raw" I mean not wrapped.

It has to be Scalar value -- most commonly integer or String -- with a
small exception that an Object
with a single scalar-valued property can also be used.

The problem here, I think, is the way Object Ids are handled
internally: they are serialized and deserialized
using regular value (de)serializers, with no knowledge of their special nature.
Ideally I think polymorphic handling would not be used at all,
regardless of Default Typing settings,
but right now no special handling or checks are made.

I think I will actually file an issue to see if this could be
improved, to avoid problems like this.

> So, seems using ObjectMapper.DefaultTyping.NON_FINAL did the trick.
> But I have latest question: is ObjectMapper.DefaultTyping.EVERYTHING really useful? What is case of using it?

I share your skepticism on general applicability of this option, but
it was requested by a user and was deemed useful
for a particular use case:

https://github.com/FasterXML/jackson-databind/issues/2349

Still, I would recommend not using it without understanding ramifications.

-+ Tatu +-

> To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jackson-user/a52486b5-4681-4677-91f6-b19e39071b28%40googlegroups.com.