Fwd: Orient Exception and the death of my system

[Michael Widmann]

Hi everybody!

Got this error while iterating over the keys (CompositeKeys) of an index with approximately 30 million entries.

I would love to give you some detail information,
but my system just hang up after this exception and all I can do for now is writing a mail.

I have no idea why he needs so much memory.
During the test I was monitoring the Java Visual VM but all I saw were a lot of byte[].

Any help would be great!


Exception in thread "Thread-5" java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.lang.StringBuilder.toString(StringBuilder.java:430)
    at com.orientechnologies.orient.core.serialization.serializer.OStringSerializerHelper.split(OStringSerializerHelper.java:322)
    at com.orientechnologies.orient.core.serialization.serializer.OStringSerializerHelper.split(OStringSerializerHelper.java:310)
    at com.orientechnologies.orient.core.serialization.serializer.OStringSerializerHelper.split(OStringSerializerHelper.java:300)
    at com.orientechnologies.orient.core.id.ORecordId.fromString(ORecordId.java:252)
    at com.orientechnologies.orient.core.id.ORecordId.<init>(ORecordId.java:62)
    at com.orientechnologies.orient.core.serialization.serializer.record.string.ORecordSerializerStringAbstract.getTypeValue(ORecordSerializerStringAbstract.java:416)
    at com.orientechnologies.orient.core.serialization.serializer.stream.OStreamSerializerLiteral.fromStream(OStreamSerializerLiteral.java:34)
    at com.orientechnologies.orient.core.serialization.serializer.stream.OCompositeKeySerializer.fromStream(OCompositeKeySerializer.java:35)
    at com.orientechnologies.orient.core.type.tree.provider.OMVRBTreeMapEntryProvider.keyFromStream(OMVRBTreeMapEntryProvider.java:313)
    at com.orientechnologies.orient.core.type.tree.provider.OMVRBTreeMapEntryProvider.getKeyAt(OMVRBTreeMapEntryProvider.java:57)
    at com.orientechnologies.orient.core.type.tree.OMVRBTreeEntryPersistent.getKeyAt(OMVRBTreeEntryPersistent.java:544)
    at com.orientechnologies.common.collection.OMVRBTreeEntry.getKey(OMVRBTreeEntry.java:120)
    at com.orientechnologies.common.collection.AbstractEntryIterator.nextKey(AbstractEntryIterator.java:54)
    at com.orientechnologies.common.collection.OMVRBTree$KeyIterator.next(OMVRBTree.java:1332)
    at com.bayoda.ods.store.orientdb.OrientDBStore.initReferenceBloomFilter(OrientDBStore.java:775)
    at com.bayoda.ods.store.orientdb.OrientDBStore.access$7(OrientDBStore.java:760)
    at com.bayoda.ods.store.orientdb.OrientDBStore$1.run(OrientDBStore.java:717)
    at java.lang.Thread.run(Thread.java:662)


Orient Revision: 4614

System environment variables (as I can remember because my system is starting to die slowly but steady)

- mmap 0
- 2.5 gb RAM for Java
- first and second level cache disabled
- node page size 256 ?


I will execute the test again. Any settings I should try?

Regards,
Michael


PS: going for a hard reboot, I do like Java ;-)

--
bayoda.com - Professional Online Backup Solutions for Small and Medium Sized Companies

[Luca Garulli]

Hi Micheal,

the "GC overhead limit exceeded" is thrown because too much time has been spent by GC to free resources. This could happen because there are too many small objects to free. You could also disable this exception with:

-XX:-UseGCOverheadLimit

But I prefer to fix it rather than suppress the message/crash. What we need is at least a oprofiler dump and full params.

Lvc@