Storing json vs object vs map in a map

[aronnora]

Hi,

I am new to Hazelcast and would appreciate your thoughts on the below.

Use case : I have a database table CUSTOMER (id, firstname, lastname,age) , and would like to store it in a distributed map. There would be need to query(may be predicate) the collection and general get/put operations. There will be somewhere around a million records and I have 2 nodes at my disposal.

What would the best approach be keeping performance and memory in mind.

1. Store the records as Map of Maps ; IMap<String, IMap<String, String>> ; where the keys in the inner map are the column names

Or

2. Store the records as json;  IMap<String,String> Ex: [ "123" : { "id" : "123", "firstname" : "john", "lastname" : "Deer", "age" : "25" }]

Or

2. Create a Customer DTO and store it in IMap<String,Customer>

Thanks

Aron

[noctarius]

Hi Aron,

The last option using an Entity class is preferred, however if you expect the object often to be changed JSON might be preferable for schemaless (since you're talking about a database table I guess you don't need that).

Btw your first option is not working since IMap itself is not serializable. You could do IMap<String,  Map<String,  String>> but the problem with this approach is that the inner map needs to be completely deserialized on every get which will kill you're performance ;-)

Chris

--
You received this message because you are subscribed to the Google Groups "Hazelcast" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast+...@googlegroups.com.
To post to this group, send email to haze...@googlegroups.com.
Visit this group at http://groups.google.com/group/hazelcast.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast/b2d4719e-fe7e-4630-9888-64426a8f5931%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[aronnora]

Thank you for your quick response. 

[Noctarius]

Also answered on Stackoverflow to not let the question open there.

[Dmitry Andrianov]

>> but the problem with this approach is that the inner map needs to be completely deserialized on every get

How is JSON better than storing a Map?
The de-serialization is not going anywhere so you will still have to de-serialize the object entirely for each get. It is different kind of de-serialization but it does not change anything.
Also, to my understanding, the preferred way (using Entity class) is not different too - it will also be de-serialized on each get().

So if I had to answer this question myself I would say:
* JSON is a no-go because you object becomes monolithic - you will not be able for example change age of a user without touching other fields. With Map and Entity you can do it if you want - with EntryProcessors. Also with JSON you will not be able to use predicates to search for entries with a specific value in a field.
* Chosing between Map and Entity - the Entity is better unless list of columns is going to change a lot. In which case I would go with Map. Entity will also allow you using predicates.

Depending on what you do it may be best to have Entity object with all the important fields including the ones you use with predicates while keep a Map inside that Entity for other fields. It will also allow you adding them in the future.

Cheers