IndexOutOfBoundsException

[Johannes Lerch]

Error in current snapshot (retrieved from svn 1 hour ago):

Exception in thread "main" com.orientechnologies.orient.core.exception.ODatabaseException: Error on saving record in cluster #5
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:512)
    at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:58)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:202)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.save(ODatabaseRecordTx.java:40)
    at com.orientechnologies.orient.core.db.ODatabaseRecordWrapperAbstract.save(ODatabaseRecordWrapperAbstract.java:194)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:159)
    at com.orientechnologies.orient.core.db.object.ODatabaseObjectTx.save(ODatabaseObjectTx.java:212)
    at com.orientechnologies.orient.core.db.object.ODatabaseObjectTx.save(ODatabaseObjectTx.java:186)
    at orientdb_test.Main.store(Main.java:133)
    at orientdb_test.Main.loadFromCouchDb(Main.java:114)
    at orientdb_test.Main.<init>(Main.java:60)
    at orientdb_test.Main.main(Main.java:50)
Caused by: java.lang.IndexOutOfBoundsException: Index: 7, Size: 7
    at java.util.ArrayList.RangeCheck(Unknown Source)
    at java.util.ArrayList.get(Unknown Source)
    at com.orientechnologies.orient.core.type.tree.OMVRBTreePersistent.getBestEntryPoint(OMVRBTreePersistent.java:697)
    at com.orientechnologies.common.collection.OMVRBTree.getEntry(OMVRBTree.java:330)
    at com.orientechnologies.common.collection.OMVRBTree.getEntry(OMVRBTree.java:307)
    at com.orientechnologies.common.collection.OMVRBTree.get(OMVRBTree.java:244)
    at com.orientechnologies.orient.core.index.OIndexMVRBTreeAbstract.get(OIndexMVRBTreeAbstract.java:188)
    at com.orientechnologies.orient.core.index.OIndexUnique.checkEntry(OIndexUnique.java:65)
    at com.orientechnologies.orient.core.index.OPropertyIndex.checkEntry(OPropertyIndex.java:63)
    at com.orientechnologies.orient.core.index.OPropertyIndexManager.checkIndexedProperties(OPropertyIndexManager.java:158)
    at com.orientechnologies.orient.core.index.OPropertyIndexManager.onRecordBeforeCreate(OPropertyIndexManager.java:40)
    at com.orientechnologies.orient.core.hook.ODocumentHookAbstract.onTrigger(ODocumentHookAbstract.java:68)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.callbackHooks(ODatabaseRecordAbstract.java:621)
    at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeSaveRecord(ODatabaseRecordAbstract.java:461)
    ... 11 more

[Luca Garulli]

Hi,

it's quite hard to understand what's happened. 

What is your HW/SW config? What snapshot? from SVN or from Maven? What are you saving?

Lvc@

[Johannes Lerch]

Ok, let's see. I have checked out revision 3053 from svn. I build it on my own with maven and copied core and commons jars to my test project. The test project loads CompilationUnit objects from a CouchDB and puts it right into a local OrientDB using ODatabaseObjectTx. Source of that CompilationUnit class can be found here: http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/org.eclipse.recommenders.commons.analysis/src/org/eclipse/recommenders/internal/commons/analysis/codeelements/CompilationUnit.java

I have a schema definition for property id to be able to create an index of type unique. I run Windows 7 64Bit and the program gets executed by
java version "1.6.0_23"
Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)

The exception is not thrown on the same insertion each run so i guess it's hard to reproduce.

I found another exception logged as warning (program keeps running after that one):
java.lang.IndexOutOfBoundsException: Requested index 0 when the range is 0-0
    at com.orientechnologies.common.collection.OMVRBTreeEntry.getKey(OMVRBTreeEntry.java:165)
    at com.orientechnologies.common.collection.OMVRBTreeEntry.getFirstKey(OMVRBTreeEntry.java:393)
    at com.orientechnologies.orient.core.type.tree.OMVRBTreePersistent.optimize(OMVRBTreePersistent.java:287)
    at com.orientechnologies.orient.core.type.tree.OMVRBTreeDatabaseLazySave.optimize(OMVRBTreeDatabaseLazySave.java:71)
    at com.orientechnologies.orient.core.index.OIndexMVRBTreeAbstract$1.optimize(OIndexMVRBTreeAbstract.java:121)
    at com.orientechnologies.orient.core.index.OIndexMVRBTreeAbstract$1.memoryUsageLow(OIndexMVRBTreeAbstract.java:87)
    at com.orientechnologies.orient.core.OMemoryWatchDog$1.handleNotification(OMemoryWatchDog.java:89)
    at sun.management.NotificationEmitterSupport.sendNotification(Unknown Source)
    at sun.management.MemoryImpl.createNotification(Unknown Source)
    at sun.management.MemoryPoolImpl$PoolSensor.triggerAction(Unknown Source)
    at sun.management.Sensor.trigger(Unknown Source)

[Luca Garulli]

Hi,

by a quick look to the stack trace seems a known bug yet open: the optimization of memory resources thrown by OMemoryWatchDog class:

 ...

 at com.orientechnologies.orient.core.index.OIndexMVRBTreeAbstract$1.memoryUsageLow(OIndexMVRBTreeAbstract.java:87)
 at com.orientechnologies.orient.core.OMemoryWatchDog$1.handleNotification(OMemoryWatchDog.java:89)

 ...

Since it's a memory problem you can avoid this "optimization" by changing some config settings. How many entries are you inserting? How much RAM have you got? Probably the best thing is to use transactions and to commit every X entries.

tx.begin();

for( int i = 0; i < SIZE; ++i ){

  // YOUR INSERTION

  if( i % 10000 == 0 )

  {

    tx.commit();

    tx.begin();

  }

}

tx.commit();

Lvc@

[Tomas Espeleta]

Hello,
I think this is related with my post "New 1.0rc2-snapshot and acknowledgments"... but I'm not using any transaction.

regards, Tomas

[Luca Garulli]

Exactly the bug seems the same but it's no related to TX, where instead TX can help to reduce the memory footprint and avoid memory problems until it's fixed.

I suggest you to use the Massive Insertion tricks:

http://code.google.com/p/orient/wiki/PerformanceTuning#Massive_Insertion

Lvc@

[Tomas Espeleta]

ah.. ok. Understood.
Tomorrow I'll test it. Thank you!

[Johannes Lerch]

I'm already using the OIntentMassiveInsert. My machine has 4GB RAM and i'm executing the test tool with -Xmx256m.
Error has occured after inserting 6000 entries at one time and at another time after 10000. I'll try that transaction stuff.