ArrayIndexOutOfBounds in BTreeMap.replace()

[Jens Bertram]

Hi, I'm unsure if i hit a bug or doing something wrong here.

I'm accessing a BTreeMap concurrently and trying to update some value, if it's already there.

The following code snipped causes MapDB (0.9.10) to throw a ArrayIndexOutOfBoundsException at org.mapdb.BTreeMap.replace(BTreeMap.java:1171):
(The idxTermsMap is shared between threads.)

final long ttf = termsEnum.totalTermFreq(); // get a new long value

// add value up for all fields, field is a string
// BytesWrap is a immutable object simply wrapping a bytes array

final Fun.Tuple2<String, BytesWrap> fieldTerm = Fun.t2(field, bw.clone());

Long oldValue = idxTermsMap.putIfAbsent(fieldTerm, ttf);
if (oldValue != null) {
  for (;;) {
    oldValue = idxTermsMap.get(fieldTerm);
    if (idxTermsMap.replace(fieldTerm, oldValue, oldValue + ttf)) { // exception cause
      break;
    }
  } 

} 

If I add some debugging code printing to the console (which slows down execution a bit) the error does not occur. Also the values when it throws an error are always different.

I'm thankful for any help :)

[Jan Kotek]

It is a bug. I opened new issue:

https://github.com/jankotek/MapDB/issues/304

 

Could you post full stack trace? I recently fixed bunch of similar concurrent issues. But non in BTreeMap itself.

 

Jan

--
You received this message because you are subscribed to the Google Groups "MapDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jens Bertram]

There's not much information:

[ERROR] 24-06 11:06:47,793 DirectIndexDataProvider:921 - Error: f=text t=00037 v=7
java.lang.ArrayIndexOutOfBoundsException: 30
        at org.mapdb.BTreeMap.replace(BTreeMap.java:1171)
        at de.unihildesheim.lucene.index.DirectIndexDataProvider$IndexTermsCollectorTarget.runProcess(DirectIndexDataProvider.java:914)
        at de.unihildesheim.util.concurrent.processing.Target.run(Target.java:132)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

BTW, the code is located here https://bitbucket.org/jesn/lucene-scoring if it's of any help.

[Jens Bertram]

FYI this also fails with the latest snapshot (mapdb-0.9.11-20140324.092301-5) available at sonatype.

Am Montag, 24. März 2014 10:25:23 UTC+1 schrieb Jens Bertram:

[Jan Kotek]

Hi,

 

I started with simple test case (attached), it does not reproduce.

So I will need some help to replicate this issue:

 

* You mentioned immutability, so I assume you know how MapDB handles that

 

* Are you positive that BytesWrap implements Comparable correctly? ArrayIndexOOBE could be caused by incorrect comparators.

Use `Fun.BYTE_ARRAY_COMPARATOR` if you are not sure.

 

* Are there other modifications performed on map. Most importantly removals?

 

* Try to disable cache.

 

* Try to enable async writes.

 

* Try on-heap mode, it does not use serialization:

DB db = new DB(new StoreHeap());

 

Thats all for now,

Jan

[Jens Bertram]

Am Montag, 24. März 2014 11:55:27 UTC+1 schrieb Jan Kotek:

Hi,

 

I started with simple test case (attached), it does not reproduce.

Thank you for looking into this.
 

So I will need some help to replicate this issue:

 

* You mentioned immutability, so I assume you know how MapDB handles that

I think i got it (looking at the examples given). BytesWrap keeps a private copy of a byte array and is only created ad-hoc for storing. There are no methods to modify it after being created. It also uses a custom serializer (simple packInt, write & unpackInt, readFully).
 

* Are you positive that BytesWrap implements Comparable correctly? ArrayIndexOOBE could be caused by incorrect comparators.

Use `Fun.BYTE_ARRAY_COMPARATOR` if you are not sure.

 

 Same result, if I use 'Fun.BYTE_ARRAY_COMPARATOR'.

 

* Are there other modifications performed on map. Most importantly removals?

The error happens very early. The map is created from a Lucene index upon application startup. There's only the filling of the map happening. No other modifications take place.

 

* Try to disable cache.

Using 'dbMkr.cacheDisable();' the error is still happening.
 

 

* Try to enable async writes.

Using 'dbMkr.asyncWriteEnable();' still no success (tried also with caches diabled).
 

 

* Try on-heap mode, it does not use serialization:

DB db = new DB(new StoreHeap());

Still getting the error. But there was one successful run - that's strange.
 
Running your test gives the following error (in 2 of 10 test runs):

Exception in thread "pool-1-thread-4" java.lang.ArrayIndexOutOfBoundsException: 19
        at org.mapdb.BTreeMap.replace(BTreeMap.java:1172)
        at Issue304Test$1.run(Issue304Test.java:64)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

Maybe this is caused by hardware issues?

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+unsubscribe@googlegroups.com.

[Jens Bertram]

Experimented a bit. If I increase the node size it triggers another error.

..with 'idxTermsMapMkr.nodeSize(120);'

[ERROR] 24-23 16:23:29,657 DirectIndexDataProvider:944 - Error: f=text t=4229162 v=1
java.lang.ArrayIndexOutOfBoundsException: 7693172
        at org.mapdb.Volume$ByteBufferVol.putData(Volume.java:311)
        at org.mapdb.StoreDirect.put2(StoreDirect.java:394)
        at org.mapdb.StoreDirect.update2(StoreDirect.java:540)
        at org.mapdb.StoreDirect.update(StoreDirect.java:491)
        at org.mapdb.Caches$HashTable.update(Caches.java:254)
        at org.mapdb.EngineWrapper.update(EngineWrapper.java:65)
        at org.mapdb.BTreeMap.put2(BTreeMap.java:753)
        at org.mapdb.BTreeMap.putIfAbsent(BTreeMap.java:1134)
        at de.unihildesheim.lucene.index.DirectIndexDataProvider$IndexTermsCollectorTarget.runProcess(DirectIndexDataProvider.java:931)

        at de.unihildesheim.util.concurrent.processing.Target.run(Target.java:132)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

..
[ERROR] 24-26 16:26:59,644 DirectIndexDataProvider:944 - Error: f=text t=errreur v=1
java.lang.ArrayIndexOutOfBoundsException: 8457674
        at org.mapdb.Volume$ByteBufferVol.getByte(Volume.java:336)
        at org.mapdb.Volume.getUnsignedShort(Volume.java:96)
        at org.mapdb.StoreDirect.longStackTake(StoreDirect.java:932)
        at org.mapdb.StoreDirect.freePhysTake(StoreDirect.java:1054)
        at org.mapdb.StoreDirect.physAllocate(StoreDirect.java:655)
        at org.mapdb.StoreDirect.update2(StoreDirect.java:535)
        at org.mapdb.StoreDirect.update(StoreDirect.java:491)
        at org.mapdb.Caches$HashTable.update(Caches.java:254)
        at org.mapdb.EngineWrapper.update(EngineWrapper.java:65)
        at org.mapdb.BTreeMap.put2(BTreeMap.java:747)
        at org.mapdb.BTreeMap.putIfAbsent(BTreeMap.java:1134)
        at de.unihildesheim.lucene.index.DirectIndexDataProvider$IndexTermsCollectorTarget.runProcess(DirectIndexDataProvider.java:931)

[Jan Kotek]

This could indicate that node size is somehow wrongly written. Or your serializer reads more/less bytes than it used to serialize key or value.

 

How does it work if you use BTreeKeySerializer.TUPLE2 as key serializer?

 

J

--
You received this message because you are subscribed to the Google Groups "MapDB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.

[Jens Bertram]

All errors except the last two occurred using the default Tuple2 serializer.
I forgot to re-enable the custom serializer before.
I could try a few more tests when I get back in a few hours.

[Jan Kotek]

I think I have enough tests for now.

 

Perhaps if you could send me more information about our OS and JVM

 

Jan

[Jens Bertram]

System

Linux 3.9.3-1-ck #1 SMP PREEMPT Sun May 19 22:16:27 EDT 2013 x86_64 GNU/Linux
OpenJDK Runtime Environment (IcedTea 2.4.5) (ArchLinux build 7.u51_2.4.5-1-x86_64)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)

Java Options

-Xms128m -Xmx2G -server
also tested with additional: -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled

The tests where executed using maven 3.2.1. The system has 4GB memory in total.

Let me know, if you need more information. (I'm also subscribed to the bug)

[Jan Kotek]

Hi,

 

This is now fixed in snapshot, it was caused by typo, similar bug was in `put` method as well.

 

More details:

https://github.com/jankotek/MapDB/issues/304

 

thanks for your help and bug report,

[Jens Bertram]

Hi,

thanks for working on that issue - it seems now fixed. Glad I could help.

However now I'm hitting the same error as reported in #308. I'm currently looking into this.