MVStore in-memory and versions

[IanP]

Hi,

I'm experimenting with using MVStore as an in-memory store to hold a tree structure of objects. 

I'd like to use it to to help implement undo/redo functionality in my app, so I was wondering about practical limits for holding versions in-memory. From the docs it looks like in-memory by default holds the last 5 versions, but the value is settable through setVersionsToKeep(). I'm looking for hints and tips about how best to approach this, and want to check my understanding and some things I've found.

1. Is there a way to set versionsToKeep to no-limit? If not ( and from looking at the code I suspect not) I guess I could just set it to some number. e.g. 10,000, which the undo stack is unlikely to ever reach

2. Bug? It seemed that using the off-heap storage would be a good idea with potentially so many versions, but when I use that it seems to only keep 2 versions. (See test class)

3. Bug? There seems to be a (rather trivial) bug on versions to keep. It seems to keep 2 more than the value set by calling setVersionsToKeep(). (See test class)

4. Is in-memory the best approach here, or should I be using a file-based store and deleting the file when the app is closed? I have no reason to keep a file - my long term storage is using H2 database to store the tree as a json-serialised string

Cheers,

Ian.

Test output when running with normal store and versionsToKeep == 15

------------------------------------------------------------------------------------------------------

Writing mem versions...

Reading mem versions...

Mem read: string=Version 19, version=19

Mem read: string=Version 18, version=18

...[Some lines snipped]...

Mem read: string=Version 4, version=4

Mem read: string=Version 3, version=3

Ran out of versions at: 2

Test output when running with off-heap store and versionsToKeep == 15

------------------------------------------------------------------------------------------------------

Writing mem versions...

Reading mem versions...

Mem read: string=Version 19, version=19

Mem read: string=Version 18, version=18

Ran out of versions at: 17

Test Class

----------------

import java.io.IOException;

import org.h2.mvstore.MVMap;

import org.h2.mvstore.MVStore;

import org.h2.mvstore.OffHeapStore;

public class MVStoreVersionPlay {

public static void main(String[] args) throws IOException {

MVStoreVersionPlay mvp = new MVStoreVersionPlay();

mvp.versionPlay();

}

public void versionPlay() {

// Use off-heap, seems only to keeps 2 regardless of setVersionsToKeep()

// MVStore vMemStore = new MVStore.Builder().autoCommitDisabled().fileStore(new OffHeapStore()).open();

MVStore vMemStore = new MVStore.Builder().autoCommitDisabled().open();

// Unset, keeps 7 versions

// Set to 9 keeps 11 versions

// vMemStore.setVersionsToKeep(9);

// Set to 15 keeps 17 versions

vMemStore.setVersionsToKeep(15);

MVMap<Long, String> memMap = vMemStore.openMap("versions");

String versionMe;

System.out.println("Writing mem versions...");

for ( int i=0; i<20; i++ ) {

versionMe = "Version " + i;

memMap.put(Long.valueOf(1), versionMe);

// System.out.printf("Mem write: string=%s, currentVersion=%d\n", versionMe, vMemStore.getCurrentVersion());

vMemStore.commit();

}

System.out.println("Reading mem versions...");

for ( long i=19; i>=0; i--) {

MVMap<Long, String> oldMap = null;

try {

oldMap = memMap.openVersion(i);

}

catch ( IllegalArgumentException e ) {

System.out.println("Ran out of versions at: " + i);

break;

}

System.out.printf("Mem read: string=%s, version=%d\n", oldMap.get(Long.valueOf(1)), oldMap.getVersion());

}

vMemStore.close();

}

}

[Thomas Mueller]

Hi,

> Is there a way to set versionsToKeep to no-limit?

I would use a high number (Integer.MAX_VALUE for example).

2. Bug? It seemed that using the off-heap storage would be a good idea with potentially so many versions, but when I use that it seems to only keep 2 versions.

 

Any kind of store (including an off-heap store) is considered a "persistence", while an "in-memory store" means objects are not persisted and fully kept in the JVM heap. I understand the term "in-memory" is a but fuzzy... I will document it.

You would need to use the MVStore.setRetentionTime(Integer.MAX_VALUE). For regular files this is 45 seconds by default, for the OffHeapStore this is 0 by default.

3. Bug? There seems to be a (rather trivial) bug on versions to keep. It seems to keep 2 more than the value set by calling setVersionsToKeep(). (See test class)

Yes, this is a bug, I will fix it. Actually I think it is just one off, as the number should mean how many _old_ versions to keep. That way, 0 means no old versions.

 

4. Is in-memory the best approach here, or should I be using a file-based store and deleting the file when the app is closed? I have no reason to keep a file - my long term storage is using H2 database to store the tree as a json-serialised string

I would probably use in-memory if you have enough (heap-) memory.

Regards,

Thomas

[IanP]

That's all very clear. Thanks Thomas. And thanks for a great product too.