Exception in thread "OrientDB MemoryWatchDog" java.lang.OutOfMemoryError: GC overhead limit exceeded

[Tom Howe]

Hi,
We are are using an orientdb database as a buffer cache in our application. The database has a single 'table' and stores 500,000 items.
When the application starts, it connects to the database. The application recieves a steady stream of xml files containing items that need to be added or updated in the 'table'. Thus we have added an index to enable fast lookups so we can delete existing entries and replace them.

Every now and then, we receive a mass update xml that means we have to replace the entire contents of the table with a new set. To do this, we drop all 500,000 items in the table, then add 500,000 new ones. With the index on the table, we are getting the exception in the subject. Without the index, we don't seem to get the exception, but the updates are horrendously slow.

I'm not sure if and what code is useful to include so please let me know.

 Any advice appreciated.

We are using version : 1.0rc9

Thanks, Tom

[Tom Howe]

Here is our code to create and setup the schema and index..

  private ODatabaseObjectTx connect() {
        File file = new File(getFileName());
        if (!file.exists()) {
            LOGGER.info("Creating new OrientDb at {}", getFileName());
            new ODatabaseObjectTx("local:" + getFileName()).create();
        }
        ensureSchemaIsCreated();
        return getDatabase();
    }

    private void ensureSchemaIsCreated() {
        ODatabaseObjectTx db = getDatabase();
        try {
            OSchema schema = db.getMetadata().getSchema();

            OClass bufferItemClass = schema.getClass(SiItem.class);
            if (bufferItemClass == null) {
                bufferItemClass = schema.createClass(SiItem.class);
                bufferItemClass.createProperty("key", OType.STRING).setMandatory(true);
                bufferItemClass.createProperty("type", OType.STRING).setMandatory(true);
                bufferItemClass.createProperty("bytes", OType.BINARY).setMandatory(true);
                bufferItemClass.createIndex("keyIdx", OClass.INDEX_TYPE.UNIQUE, "key");

            }
            schema.save();
            db.getEntityManager().registerEntityClasses("com.epg.model");

        } finally {
            db.close();
        }
    }
   

[Luca Garulli]

Hi Tom,

we fixed so many bugs on indexes and memory usage that I strongly suggest you to make a try with 1.0-SNAPSHOT.

Lvc@

[Tom Howe]

oh, just how buggy is it? :)

I tried to   move to 1.0-SNAPSHOT and got an exception that it could not find a symbol.

[ERROR] symbol  : class ODatabaseObjectPool
[ERROR] location: package com.orientechnologies.orient.core.db.object

Suggestions?

Thanks, Tom

[Gabriel Vince]

Hi ,

orient db 1.0 use OObjectDatabase instead , it is completly refactored, there is a separate module for the object database, so you have to update your dependencies / libraries

Gabriel

On Thursday, May 10, 2012 12:21:11 PM UTC+2, tommytastic wrote:

oh, just how buggy is it? :)

I tried to   move to 1.0-SNAPSHOT and got an exception that it could not find a symbol.

[ERROR] symbol  : class ODatabaseObjectPool
[ERROR] location: package com.orientechnologies.orient.core.db.object

Suggestions?

Thanks, Tom

On Thu, May 10, 2012 at 10:23 AM, Luca Garulli wrote:

Hi Tom,

we fixed so many bugs on indexes and memory usage that I strongly suggest you to make a try with 1.0-SNAPSHOT.

Lvc@

On 10 May 2012 09:44, Tom Howe wrote:

[Tom Howe]

Hi Gabriel,
Could you point me to some documentation that will guide me through this refactor/example on a wiki maybe?

Which is the maven module for the object database?

Thanks, Tom

[Gabriel Vince]

Hi Tom,

seems wiki is update, so http://code.google.com/p/orient/wiki/ObjectDatabase

good luck
      Gabriel

Op donderdag 10 mei 2012 14:13:42 UTC+2 schreef tommytastic het volgende:

Hi Gabriel,
Could you point me to some documentation that will guide me through this refactor/example on a wiki maybe?

Which is the maven module for the object database?

Thanks, Tom

[Tom Howe]

I'm now stuck at :

java.lang.NoClassDefFoundError: javassist/util/proxy/Proxy
    at com.orientechnologies.orient.object.enhancement.OObjectEntitySerializer.serializeObject(OObjectEntitySerializer.java:94)
    at com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:294)
    at com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:241)

I tried adding javassist to my pom.xml ..

        <dependency>
            <groupId>javassist</groupId>
            <artifactId>javassist</artifactId>
            <version>3.12.1.GA</version>
        </dependency>

Cheers, Tom

[Gabriel Vince]

Hi Tom,

what environment do you use? A normal java applicatiom, J2EE, OSGi? I have a similar issue using Apache Karaf, I will resolve it when having a little of time. When running a standalone app or J2EE there should be no problem (well, in J2EE beware of classloaders, it there are no multiple javassist libs)

Gabriel

On Thursday, May 10, 2012 6:02:58 PM UTC+2, tommytastic wrote:

I'm now stuck at :

java.lang.NoClassDefFoundError: javassist/util/proxy/Proxy
    at com.orientechnologies.orient.object.enhancement.OObjectEntitySerializer.serializeObject(OObjectEntitySerializer.java:94)
    at com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:294)
    at com.orientechnologies.orient.object.db.OObjectDatabaseTx.save(OObjectDatabaseTx.java:241)

I tried adding javassist to my pom.xml ..

        <dependency>
            <groupId>javassist</groupId>
            <artifactId>javassist</artifactId>
            <version>3.12.1.GA</version>
        </dependency>

Cheers, Tom

[Tom Howe]

Turned out I needed the very latest javassist - 3.16.1-GA.

[Tom Howe]

We need to have quite a high heap space for our application so that a part of our application can use it for caching. so we use -Xms4096M -Xmx4096M.

How do I configure orientdb to only use a relatively small amount of heap space, say 256Mb for caching?

Thanks, Tom

[Luca Garulli]

Hi,

there are several caching structure inside of OrientDB. By default the behavior is to use all the available memory and release it upon GC free. So if you've set -Xmx4096M OrientDB will cache as much as it can until GC reclaims for memory. 

The structures that "eats" memory and can be configured are:

• 1st level cache:  http://code.google.com/p/orient/wiki/Caching#Level_1
• 2nd level cache: http://code.google.com/p/orient/wiki/Caching#Level_2
• MVRB-Trees (indexes). Look at the parameters in the section  http://code.google.com/p/orient/wiki/PerformanceTuning#Parameters. In particular  mvrbtree.lazyUpdates tells the number of buffered changes before to flush them to disk. Reduce them in case you've millions of entries in the index or low memory

Lvc@

[Tom Howe]

Great thanks, will try them

Also - is there a way to completely drop and recreate my database programatically?

Thanks,
Tom

[Luca Garulli]

Hi

course you can. If you're in local:

db.drop();

db.create();

Using remote:

OServerAdmin serverAdmin = new OServerAdmin("remote:localhost/testgraph").connect("root", "9CFB1B6F393EA5D9C2D534DC3434A6C92A2006F160D1016FB2A698BB63D34");

serverAdmin.dropDatabase();

serverAdmin.createDatabase("document", "local");

Lvc@