BTreeMap: Writer thread failed: Negative position

[Jens Bertram]

Hi, again I'm unsure if I hit a bug, because I'm not able to reproduce this error in a test case.

Using a database created like this:

DBMaker dbMkr = DBMaker.newFileDB(...);
dbMkr.transactionDisable()
 .asyncWriteEnable()
 .asyncWriteFlushDelay(100)
 .mmapFileEnableIfSupported()
 .closeOnJvmShutdown();
this.db = dbMkr.make();

I get the following exception:

java.lang.RuntimeException: Writer thread failed
        at org.mapdb.AsyncWriteEngine.checkState(AsyncWriteEngine.java:328)
        at org.mapdb.AsyncWriteEngine.update(AsyncWriteEngine.java:428)
        at org.mapdb.Caches$HashTable.update(Caches.java:254)
        at org.mapdb.EngineWrapper.update(EngineWrapper.java:65)
        at org.mapdb.BTreeMap.put2(BTreeMap.java:755)
        at org.mapdb.BTreeMap.put(BTreeMap.java:646)
        at de.unihildesheim.lucene.index.ExternalDocTermDataManager.setData(ExternalDocTermDataManager.java:180)
        at de.unihildesheim.lucene.index.DirectIndexDataProvider.setTermData(DirectIndexDataProvider.java:709)
        at de.unihildesheim.lucene.scoring.clarity.impl.DefaultClarityScore.calcDocumentModel(DefaultClarityScore.java:205)
        at de.unihildesheim.lucene.scoring.clarity.impl.DefaultClarityScorePrecalculator$DocumentModelCalculator.runProcess(DefaultClarityScorePrecalculator.java:121)
        at de.unihildesheim.util.concurrent.processing.Target.run(Target.java:132)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IllegalArgumentException: Negative position
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:822)
        at org.mapdb.Volume$MappedFileVol.makeNewBuffer(Volume.java:484)
        at org.mapdb.Volume$ByteBufferVol.tryAvailable(Volume.java:285)
        at org.mapdb.Volume.ensureAvailable(Volume.java:49)
        at org.mapdb.StoreDirect.freePhysTake(StoreDirect.java:1083)
        at org.mapdb.StoreDirect.physAllocate(StoreDirect.java:655)
        at org.mapdb.StoreDirect.update2(StoreDirect.java:535)
        at org.mapdb.StoreDirect.update(StoreDirect.java:491)
        at org.mapdb.EngineWrapper.update(EngineWrapper.java:65)
        at org.mapdb.AsyncWriteEngine.access$101(AsyncWriteEngine.java:74)
        at org.mapdb.AsyncWriteEngine.runWriter(AsyncWriteEngine.java:220)
        at org.mapdb.AsyncWriteEngine$WriterRunnable.run(AsyncWriteEngine.java:170)
        ... 1 more

ExternalDocTermDataManager.setData simply calls .put(Fun.t3(documentId, key, term.clone()), value) where documentId is an Integer, key a String, term a wrapped byte array that clones to a new Object and value is a Double. The method gets invoked concurrently.
If I remove  .asyncWriteEnable() & .asyncWriteFlushDelay(100) the error gets triggered much later while processing and fails only with the Negative position exception.

I'm running this on a 64bit linux machine.
java version "1.7.0_51"
OpenJDK Runtime Environment (IcedTea 2.4.6) (ArchLinux build 7.u51_2.4.6-1-x86_64)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)
--DEFAULT FLAGS/GC:
-XX:+CMSClassUnloadingEnabled -XX:InitialHeapSize=134217728 -XX:MaxHeapSize=2147483648 -XX:MaxNewSize=174485504 -XX:MaxTenuringThreshold=6 -XX:OldPLABSize=16 -XX:+PrintCommandLineFlags -XX:+PrintGCDetails
 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC

Thanks for any pointers :)

[Jens Bertram]

Forgot to mention: I'm using the latest 1.0 snapshot from the sonartype repos.

[Jens Bertram]

Disabling mmap support gets rid of the error but slows down the processing quite a bit.

[Jens Bertram]

Ok, sorry for the noise. I was mislead by the errors reported. Looks like it was caused by a custom Serializer reporting the wrong fixedSize()value. Now the wrong position error makes sense.

[Jens Bertram]

I'm still facing this isse. Changing quite a few bits in my source I'm not entirely sure this isn't a bug. But still I'm not able to reproduce this in a test-case. So if anybody knows what may cause this error this would be of great help.

[Jan Kotek]

Hi,

 

could you try latest 1.0.0 snapshots? There were two bugs in 0.9.11 which are now fixed.

--
You received this message because you are subscribed to the Google Groups "MapDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jan Kotek]

Also please try to enable assertion with -ea JVM command line switch

[Jan Kotek]

Hi,

 

I believe this is duplication of this bug and is fixed now in current 1.0.0 snapshot:

https://github.com/jankotek/MapDB/issues/312

 

Jan

[Jens Bertram]

Hmm, I just built mapdb from git and #312 is passing fine. However I still get the error.
This is the stacktrace with -ea turned on:

java.lang.RuntimeException: Writer thread failed
        at org.mapdb.AsyncWriteEngine.checkState(AsyncWriteEngine.java:328)
        at org.mapdb.AsyncWriteEngine.update(AsyncWriteEngine.java:428)
        at org.mapdb.Caches$HashTable.update(Caches.java:254)
        at org.mapdb.EngineWrapper.update(EngineWrapper.java:65)

        at org.mapdb.BTreeMap.put2(BTreeMap.java:793)
        at org.mapdb.BTreeMap.put(BTreeMap.java:644)
        at de.unihildesheim.lucene.index.ExternalDocTermDataManager.setData(ExternalDocTermDataManager.java:166)
        at de.unihildesheim.lucene.index.DirectIndexDataProvider.setTermData(DirectIndexDataProvider.java:752)

        at de.unihildesheim.lucene.scoring.clarity.impl.DefaultClarityScore.calcDocumentModel(DefaultClarityScore.java:205)
        at de.unihildesheim.lucene.scoring.clarity.impl.DefaultClarityScorePrecalculator$DocumentModelCalculator.runProcess(DefaultClarityScorePrecalculator.java:121)
        at de.unihildesheim.util.concurrent.processing.Target.run(Target.java:132)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

Caused by: java.lang.AssertionError

        at org.mapdb.Volume$MappedFileVol.makeNewBuffer(Volume.java:484)
        at org.mapdb.Volume$ByteBufferVol.tryAvailable(Volume.java:285)
        at org.mapdb.Volume.ensureAvailable(Volume.java:49)

        at org.mapdb.StoreDirect.freePhysTake(StoreDirect.java:1086)

        at org.mapdb.StoreDirect.physAllocate(StoreDirect.java:655)
        at org.mapdb.StoreDirect.update2(StoreDirect.java:535)
        at org.mapdb.StoreDirect.update(StoreDirect.java:491)
        at org.mapdb.EngineWrapper.update(EngineWrapper.java:65)
        at org.mapdb.AsyncWriteEngine.access$101(AsyncWriteEngine.java:74)
        at org.mapdb.AsyncWriteEngine.runWriter(AsyncWriteEngine.java:220)
        at org.mapdb.AsyncWriteEngine$WriterRunnable.run(AsyncWriteEngine.java:170)
        ... 1 more

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+unsubscribe@googlegroups.com.

[Jens Bertram]

Ok, filed a bug (#313) after a lot of trial and error :)

[Jan Kotek]

Hi,

 

this should be fixed in latest snapshot.

 

J.

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "MapDB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mapdb+un...@googlegroups.com.