MVStore and interrupts?

[Dan]

I've been going down this road: https://github.com/h2database/h2database/issues/227 trying to figure out how to work around interrupts with MVStore.

Everyone just says use 'retry' or don't interrupt... not interrupting is easier said than done with other libraries.

I can't figure out the syntax for "retry".

I've tried something like this:

this.store = new MVStore.Builder().fileName("retry:" + new File(mvFolder, MV_STORE + ".mv").getAbsolutePath()).open();

But that doesn't work:

 Directory does not exist: nio:retry:/mnt.....

I've been poking around in the source code, trying to figure out how to pass it in, and I'm quite confused.

In the the maven packaged jar for 1.4.197, there is no class named FilePathRetryOnInterrupt which is what is supposed to provide the scheme, as far as I can tell.

It looks like that class is only included in the H2 jar file?

If MV Store is what gets broken, why isn't the fix / workaround included in the MV Store jar?

This issue / workaround might be a good thing to add to the FAQ, as well.

Thanks, 

Dan

[Thomas Mueller Graf]

Hi,

Try this:  "retry:nio:" + ... See also TestConcurrent.testInterruptReopen() (It would be good to improve documentation for this I guess).

Regards,

Thomas

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[Noel Grandin]

On 2018/06/28 9:59 AM, Dan wrote:
> I've been going down this road: https://github.com/h2database/h2database/issues/227 trying to figure out how to work
> around interrupts with MVStore.
>
> Everyone just says use 'retry' or don't interrupt... not interrupting is easier said than done with other libraries.

The easiest approach is just to run H2 as a TCP server. Even if it's just inside the same process, the TCP stack will
act as a cut-out.

[Thomas Mueller Graf]

Hi,

> not interrupting is easier said than done with other libraries.

Yes... One case, that was biting us, is using java.util.concurrent.ExecutorService.shutdownNow(). This was calling Thread.interrupt() in our case. It was relatively easy to resolve: now we use shutdown() instead. See https://issues.apache.org/jira/browse/SLING-5416

Regards,
Thomas

[Dan]

Yea, a quick search of my classpath shows about 5 different libraries doing an interrupt at one point or another, and I'm not even sure yet which one was biting me.

I'm not trying to use H2, I'm just trying to use the MVStore directly.  The MVStore jar package doesn't contain the classes necessary to use the Retry workaround... I had to add the H2 jar files to my classpath to get it to work - at least, I think I have it working right now.  I need to test more.

FYI, I've been doing a compare of Xodus vs MV Store for my use pattern, and so far, MVStore is way faster than Xodus :)

The one thing I don't have a good handle on with MVStore, however, is how the versions work if I don't want any versions beyond current.  The documentation for .setVersionsToKeep(0); only references inMemory stores.... for disk stores, I'm a little lost.

Does it work to set it to 0?  

Or, do I also need to call store.compactMoveChunks() at some interval?

Thanks, 

Dan

[Andrei Tokar]

Hi Dan, with respect to version management in MVStore, there are significant differences between 1.4.197 release and trunk, that’s why I would recommend to check out latest code.
Number of versions kept in a file-based store is determined by usage. In absense of registerVersionUsage() / deregisterVersionUsage() calls, only latest version will be kept. This is only safe to do in a single-threaded scenario without auto-commit. Once you have more than one thread (including background auto-commit one), “current” version may be not what you think it is, and your operation, i.e. map iteration via cursor, may fail due to the fact that version it was iterating on, has been removed. That is why interest in specific version (and coincidentally all subsequent ones) should be expressed via registerVersionUsage().
In addition, time-based policy may be used with setRetentionTime(), but it never be 100% reliable, and mainly exists for usage simplicity and backward compatibility.
You may call compactMoveChunks() if you want, but that not strictly necessary, because space occupied by unused versions will be eventually re-used.

[Dan]

Hi Andrei, 

For my project, I need released maven dependencies to point at, so I can't move up to trunk yet.

If I'm understanding things correctly, the default behavior of in the trunk code, is that if I only want the latest version, then, I just do nothing special, and it only keeps the latest version.

Is that also true in the 1.4.197 release?  I've been calling the setVersionsToKeep(0), as despite the documentation, it seems to be used for some things in the file store.... but that's just from a cursory look at the code.

That's really all I want, is the disk-backed ConcurrentHashMap behavior, not the version store.

I tried not calling compactMoveChunks(), but in my use pattern, it leaves the DB way too big.  

It seems to be ok, if I only call it when there aren't any other threads hitting the data store.  If I call it when other things are happening, it breaks the MVStore pretty reliably, per the other thread I started.  So there are obviously some multi-threading issues in the 1.4.197 line.

I could do some testing at some point with your trunk code, if you want me to see if the multithread issues are fixed in trunk.

Thanks, 

Dan