Genson serialization for maps with complex keys

[Shailendrasingh Patil]

One of the feauture listed on Genson home page is "Serialization and Deserialization of maps with complex keys".

While I'm trying to serialize a map with Keys as complex java objects into json string, and then deserialize them back to java Map. The deserialized map keys are always strings. Can someone please help me on how to use genson for such complex key map serialization and deserialization?

Here is my code

    Genson genson = new GensonBuilder().useClassMetadata(true).useRuntimeType(true).create();
    VO vo = new VO();
    Key key = new Key(18314212, new Timestamp(System.currentTimeMillis()),new Timestamp(System.currentTimeMillis()));
    vo.setEndTime(new Timestamp(System.currentTimeMillis()));
    vo.setStartTime(new Timestamp(System.currentTimeMillis()));
    vo.setItemID(18314212);
    vo.setKey(key);
    Map<Object, Object> map = new HashMap<Object, Object>();
    map.put(key, vo);
    String json  = genson.serialize(map);
    System.out.println(json); //the json map key does not have @Class attribute 
    Map jsonMap =  genson.deserialize(json, Map.class);
    ((Map.Entry)jsonMap.entrySet().iterator().next()).getKey().getClass(); //it is always a string

[Eugen Cepoi]

Hey there,

I have answered to your question on SO http://stackoverflow.com/questions/30057860/genson-serialization-for-maps-with-complex-keys/30058670#30058670

In short don't use runtimeType and when serializing define the type:
String json = genson.serialize(m, new GenericType<Map<Key,Value>>() {});

Cheers,

Eugen

--
You received this message because you are subscribed to the Google Groups "Genson user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genson+un...@googlegroups.com.
To post to this group, send email to gen...@googlegroups.com.
Visit this group at http://groups.google.com/group/genson.
To view this discussion on the web visit https://groups.google.com/d/msgid/genson/e397fd05-aa76-4241-b21f-63b944a1ca7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shailendrasingh Patil]

Thanks Eugen for your help. Do you know of any workaround? I can not disable runtime type and do not know what type of objects the map will hold. 

[Eugen Cepoi]

Ok so it looks like your use case is a bit tricky :)

Is the type of the key that is dynamic (Key and subclasses of Key) or you always have instances of Key class as key?

To view this discussion on the web visit https://groups.google.com/d/msgid/genson/4c4b96bf-d1c8-447a-8db6-8314a6cdcdca%40googlegroups.com.

[Shailendrasingh Patil]

The type of the key is dynamic. It could be any Object.

[Eugen Cepoi]

Ah I see...I don't know what your use case is, but in general I think you should avoid having complex keys.

The quickest to implement solution would be to always use ComplexMapConverter when you have a map. The downside is that even maps with simple keys (like string) will not be serialized as a json object but as a array of pairs (like for complex maps). But maybe this is good for you?

As current version of ComplexMapConverter has a private constructor, you will have to duplicate it.

Then you can try this (use your copy of the converter):

import static com.owlike.genson.reflect.TypeUtil.*;

Genson genson = new GensonBuilder()
        .useClassMetadata(true)
        .useRuntimeType(true)
        .withConverterFactory(new Factory<Converter<Map<?, ?>>>() {
          @Override
          public Converter<Map<?, ?>> create(Type type, Genson genson) {
            Type expandedType = type;
            if (getRawClass(type).getTypeParameters().length == 0) {
              expandedType = expandType(lookupGenericType(Map.class, getRawClass(type)), type);
            }

            Type keyType = typeOf(0, expandedType);
            Type valueType = typeOf(1, expandedType);

            if (keyType == Object.class)
              return new ComplexMapConverter(genson.provideConverter(keyType), genson.provideConverter(valueType));
            else return null;
          }
        })
        .create();

Sorry for all the boilerplate code...does it work for you?

If yes, in the next release I'll provide an option in genson builder to always ser/de maps for which the key is Object as a complex map.

To view this discussion on the web visit https://groups.google.com/d/msgid/genson/9eb23b9e-0469-452f-9612-21e27c6920ad%40googlegroups.com.

[Eugen Cepoi]

You can find here the code for ComplexMapConverter https://github.com/owlike/genson/blob/master/genson/src/main/java/com/owlike/genson/convert/DefaultConverters.java#L883-L930

[Shailendrasingh Patil]

Thanks Eugen. It worked for most of the cases. But it did not work for deserialization of map which contains another object of map as a value.

[Eugen Cepoi]

Ah :(
Can you provide an example of the serialized object that gives this error when trying to deser. it back?

It would be easier for me to reproduce it and find a more general solution.

Thanks,

Eugen

To view this discussion on the web visit https://groups.google.com/d/msgid/genson/c5b20ea0-82d0-466b-9ef3-43f81b02672c%40googlegroups.com.

[Shailendrasingh Patil]

   Something like following code snippet

    VO vo = new VO();
    Key key = new Key(18314212, new Timestamp(System.currentTimeMillis()),new Timestamp(System.currentTimeMillis()));
    vo.setEndTime(new Timestamp(System.currentTimeMillis()));
    vo.setStartTime(new Timestamp(System.

currentTimeMillis()));
    vo.setItemID(18314212);
    vo.setKey(key);
    Map<Object, Object> map = new HashMap<Object, Object>();
    map.put(key, vo);
    Map map1 = new HashMap();
    map1.put("data", "12312321");
    map1.put("datamap", map);
    String json  = genson.serialize(map1);
    System.out.println(json); 
    Map jsonMap =  genson.deserialize(json, Map.class); // throws exception fails at this line

com.owlike.genson.stream.JsonStreamException: Illegal character at row 0 and column 62 expected [ but read '"' !
	at com.owlike.genson.stream.JsonReader.newWrongTokenException(JsonReader.java:951)
	at com.owlike.genson.stream.JsonReader.begin(JsonReader.java:427)
	at com.owlike.genson.stream.JsonReader.beginArray(JsonReader.java:151)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:153)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:1)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.Genson.deserialize(Genson.java:389)
	at com.owlike.genson.convert.DefaultConverters$UntypedConverterFactory$UntypedConverter.deserialize(DefaultConverters.java:1074)
	at com.owlike.genson.convert.ClassMetadataConverter.deserialize(ClassMetadataConverter.java:100)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.convert.DefaultConverters$CollectionConverter.deserialize(DefaultConverters.java:175)
	at com.owlike.genson.convert.DefaultConverters$CollectionConverter.deserialize(DefaultConverters.java:158)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.Genson.deserialize(Genson.java:389)
	at com.owlike.genson.convert.DefaultConverters$UntypedConverterFactory$UntypedConverter.deserialize(DefaultConverters.java:1074)
	at com.owlike.genson.convert.ClassMetadataConverter.deserialize(ClassMetadataConverter.java:100)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:168)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:1)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.Genson.deserialize(Genson.java:389)
	at com.owlike.genson.Genson.deserialize(Genson.java:306)
	at com.theocc.service.client.TestDataServiceClient.testJSONDeserialization(TestDataServiceClient.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:73)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:82)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:73)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:217)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:83)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:68)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:163)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
com.owlike.genson.JsonBindingException: Could not deserialize to type interface java.util.Map
	at com.owlike.genson.Genson.deserialize(Genson.java:391)
	at com.owlike.genson.Genson.deserialize(Genson.java:306)
	at com.theocc.service.client.TestDataServiceClient.testJSONDeserialization(TestDataServiceClient.java:145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:73)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:82)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:73)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:217)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:83)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:68)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:163)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: com.owlike.genson.JsonBindingException: Could not deserialize to type interface java.util.List
	at com.owlike.genson.Genson.deserialize(Genson.java:391)
	at com.owlike.genson.convert.DefaultConverters$UntypedConverterFactory$UntypedConverter.deserialize(DefaultConverters.java:1074)
	at com.owlike.genson.convert.ClassMetadataConverter.deserialize(ClassMetadataConverter.java:100)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:168)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:1)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.Genson.deserialize(Genson.java:389)
	... 31 more
Caused by: com.owlike.genson.JsonBindingException: Could not deserialize to type interface java.util.Map
	at com.owlike.genson.Genson.deserialize(Genson.java:391)
	at com.owlike.genson.convert.DefaultConverters$UntypedConverterFactory$UntypedConverter.deserialize(DefaultConverters.java:1074)
	at com.owlike.genson.convert.ClassMetadataConverter.deserialize(ClassMetadataConverter.java:100)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.convert.DefaultConverters$CollectionConverter.deserialize(DefaultConverters.java:175)
	at com.owlike.genson.convert.DefaultConverters$CollectionConverter.deserialize(DefaultConverters.java:158)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.Genson.deserialize(Genson.java:389)
	... 40 more
Caused by: com.owlike.genson.stream.JsonStreamException: Illegal character at row 0 and column 63 expected ] but read 'k' !
	at com.owlike.genson.stream.JsonReader.newWrongTokenException(JsonReader.java:951)
	at com.owlike.genson.stream.JsonReader.end(JsonReader.java:437)
	at com.owlike.genson.stream.JsonReader.endArray(JsonReader.java:174)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:174)
	at com.theocc.service.client.JSONMessageParser$ComplexMapConverter.deserialize(JSONMessageParser.java:1)
	at com.owlike.genson.convert.RuntimeTypeConverter.deserialize(RuntimeTypeConverter.java:47)
	at com.owlike.genson.convert.NullConverter$NullConverterWrapper.deserialize(NullConverter.java:56)
	at com.owlike.genson.Genson.deserialize(Genson.java:389)
	... 49 more

[Eugen Cepoi]

Ok I see. I am not sure to have a solution to this.

The problem is that during deserialization we don't have the type information for the array that represents the contained map, so Genson uses the classic collection deserializer. This is because Genson provides the metadata mechanism only for json objects.

When thinking about your structure, I am also wondering if you are going the right way. Storing maps inside maps all using complex keys sounds wrong and might be a warning that there is a design issue. Why don't you use pojos (even if polymorphic)?

Here are some workarounds I can think of:

  - Use pojos if you can, it's the cleanest

  - Use only maps with complex keys only where you can provide the type (for example inside a pojo that says that this property will be a map)

  - Don't use maps but instead only Lists of pairs (key, value), in this situation Genson would successfully ser/de the list and retain the generic info for the nested objects

  - A mix of the 3 above

  - Last idea but I don't think you should go this route is to provide your own UntypedConverter that would deser any json array it encounters as a map (that in the end would use the complexmapconverter). The idea would be inside the deserialize method to check if the value type is an array and in that case deserialize into a Map otherwise do the same thing it is doing now https://github.com/owlike/genson/blob/master/genson/src/main/java/com/owlike/genson/convert/DefaultConverters.java#L1061-L1092

I don't see any other solution yet. What do you think, does any work for you?

Please let me know how it goes and don't hesitate to ask if you need more explanation on how to do those things.

Good luck,

Eugen

To view this discussion on the web visit https://groups.google.com/d/msgid/genson/b603148b-4e43-40a9-9e40-4d8e2b24bcd0%40googlegroups.com.

[Shailendrasingh Patil]

Hi Eugen,

I am creating a generic rest interface for a cache. The aim is to allow store any java object in the cache, as long as the class of that object is in the classpath, At the same time we do not want client to adhere to any specific object format, so we did not create our wrapper object. So the rest service method accepts various arguments in the form of hashmap. 

Therefore I need a heterogenous map. I'm not sure whether I could explain myself clearly. But from your responses I got a temporary workaround. In my rest client, I always convert Map to the list of keys and list of values and then send it to the server which recreates the map after deserializing the two lists. 

Thanks a lot for your help. Genson is a great library and I'm glad to have a library which can serialize/deserialize almost any object using its runtime information. It is so simple to use and it is very fast too.

Thanks,

Shailendra

[Eugen Cepoi]

Hi Shailendra,

I am glad to know that you are happy with Genson :)

2015-05-07 3:36 GMT+02:00 Shailendrasingh Patil <shailendra...@gmail.com>:

Hi Eugen,

I am creating a generic rest interface for a cache. The aim is to allow store any java object in the cache, as long as the class of that object is in the classpath, At the same time we do not want client to adhere to any specific object format, so we did not create our wrapper object. So the rest service method accepts various arguments in the form of hashmap. 

You could also store the data as plain bytes in the cache instead of the deserialized objects, so the client would do on his side the ser/de of his data to/from a byte array. That's what things like memcached do.

 

Therefore I need a heterogenous map. I'm not sure whether I could explain myself clearly. But from your responses I got a temporary workaround. In my rest client, I always convert Map to the list of keys and list of values and then send it to the server which recreates the map after deserializing the two lists. 

Ha good! Though I am not sure it would work in all the situations.

I opened this new issue https://github.com/owlike/genson/issues/64, to provide a system that retains type info for any kind of json value not only objects.

The idea is to wrap every value in a json object containing metadata info (the class name) and a property "value" that points to the real value (that can be a json object/array/string/etc).

To wrap like that all the default ser/de mechanism one would have to implement a ChainedFactory (like this one https://github.com/owlike/genson/blob/master/genson/src/main/java/com/owlike/genson/convert/RuntimeTypeConverter.java) that creates an instance of a Converter doing the un/wrapping logic. Then register it using GensonBuilder.withConverterFactory instead of enabling class metadata.

It is not perfect as it would decrease performances and the produced json is kind of ugly but would have the benefit of working in any situation.

If it sounds good to you and want to implement it, let me know. Contributing it via a pull request is very welcome :)

Cheers,

Eugen

 

To view this discussion on the web visit https://groups.google.com/d/msgid/genson/f486e828-21a2-4f61-91d9-a4d0119f18d1%40googlegroups.com.