ConcurrentModificationException serialising a map

[Paolo Di Tommaso]

Dear all, 

I'm getting this exception in rare conditions while serialising a map objects. I've tried both 2.23 and 2.24 versions. 

That's very strange because the code serialising that object is supposed to be thread safe. 

Any clue about the cause?  

 

May-19 22:57:33.816 [Thread-4] WARN  nextflow.processor.DelegateMap - Cannot serialize context map. Cause: null -- Resume will not work on this process

java.util.ConcurrentModificationException: null

at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859) ~[na:1.7.0_45]

at java.util.ArrayList$Itr.next(ArrayList.java:831) ~[na:1.7.0_45]

at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:81) ~[kryo-2.24.0.jar:na]

at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:22) ~[kryo-2.24.0.jar:na]

at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:599) ~[kryo-2.24.0.jar:na]

at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:95) ~[kryo-2.24.0.jar:na]

at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21) ~[kryo-2.24.0.jar:na]

at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:599) ~[kryo-2.24.0.jar:na]

at com.esotericsoftware.kryo.Kryo$writeClassAndObject$5.call(Unknown Source) ~[na:na]

at nextflow.util.KryoHelper.serialize(SerializationHelper.groovy:141) ~[main.jar:na]

at nextflow.util.KryoHelper$serialize$0.call(Unknown Source) ~[na:na]

at nextflow.processor.DelegateMap.save(DelegateMap.groovy:162) ~[main.jar:na]

at nextflow.processor.DelegateMap$save$15.call(Unknown Source) ~[na:na]

at nextflow.processor.TaskProcessor.finalizeTask(TaskProcessor.groovy:1356) ~[main.jar:na]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_45]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_45]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_45]

at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_45]

Cheers,

Paolo

[mongonix]

Hi Paolo,

On Monday, May 19, 2014 11:21:05 PM UTC+2, Paolo Di Tommaso wrote:

Dear all, 

I'm getting this exception in rare conditions while serialising a map objects. I've tried both 2.23 and 2.24 versions. 

That's very strange because the code serialising that object is supposed to be thread safe. 

Any clue about the cause?  

Based on your log, it looks like your maps use ArrayLists either as keys or as values (or their sub-objects). And these ArrayLists are accessed concurrently by Kryo and your other application threads, which leads to problems. You should remember that ArrayLists are not thread-safe as opposed e.g. to Vectors.

-Leo

[Paolo Di Tommaso]

Hi, 

Sure, the point is that I'm using the ArrayList because the code is supposed to be thread safe. 

Also, I'm supposing that Kryo executes the serialization process in the same thread, and so it is thread safe as well.

One more thing, could be a problem that I'm creating Kryo instances using a ThreadLocal in a similar manner how explained in this post? 

http://blog.hazelcast.com/2013/10/16/kryo-serializer/

Is safe this approach?

Thanks,

Paolo

[mongonix]

Hi ,

On Tuesday, May 20, 2014 11:15:47 AM UTC+2, Paolo Di Tommaso wrote:

Hi, 

Sure, the point is that I'm using the ArrayList because the code is supposed to be thread safe. 

Also, I'm supposing that Kryo executes the serialization process in the same thread, and so it is thread safe as well.

Kryo never modifies any object concurrently, unless you try to use the same Kryo/Input/Output by multiple threads simultaneously, which you should never do. But Kryo works in the original object that you submitted for serialization, not on its copy. 

Therefore if during serialization of your object by Kryo you modify your object or objects reachable from it by another thread, i.e. concurrently, you may run into problems. And your stack trace seems to indicate that this is exactly what happens in your code. Are you sure that you do not modify your maps/collections or objects reachable from it by another thread during serialization?

 

One more thing, could be a problem that I'm creating Kryo instances using a ThreadLocal in a similar manner how explained in this post? 

http://blog.hazelcast.com/2013/10/16/kryo-serializer/

Is safe this approach?

ThreadLocal is fine as long as you follow the rules described above.

 

-Leo

[Nate]

Maybe you have references off and you serialize a graph that contains an arraylist which contains objects that reference the arraylist?

-Nate

--
You received this message because you are subscribed to the "kryo-users" group.
http://groups.google.com/group/kryo-users
---
You received this message because you are subscribed to the Google Groups "kryo-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kryo-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.