Chunk metadata too long

[Clyde Stubbs]

I had this exception:

Caused by: java.lang.IllegalStateException: Chunk metadata too long [1.4.199/3]
 at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:893)
 at org.h2.mvstore.Chunk.writeChunkHeader(Chunk.java:165)
 at org.h2.mvstore.MVStore.storeNow(MVStore.java:1370)
 at org.h2.mvstore.MVStore.store(MVStore.java:1233)
 at org.h2.mvstore.MVStore.tryCommit(MVStore.java:1178)
 at org.h2.mvstore.MVStore.writeInBackground(MVStore.java:2821)
 at org.h2.mvstore.MVStore$BackgroundWriterThread.run(MVStore.java:3191)

Unlike the case posted by someone else, there was plenty of disk space available at the time. The operation that apparently triggered it was the deletion of a large number (over 100K) of map entries in succession. I am not sure how far it got.

The deletions were being done on one thread, there were likely to have been insertions into the same map happening concurrently on another thread, at a low rate (once per second.) Since MVStore is supposed to support concurrent reads and writes I assume this should not be a problem.

Any thoughts on what might have caused this or more importantly what could be done to avoid it? And a further question - after this exception any further attempts to access the database resulted in "This store is closed" errors - is there a recommended way to deal with this, i.e. have the store automatically reopened?

[Andrei Tokar]

First of all - it has nothing to do with disk space available. It is a bug in MVStore. When chunk is saved and insufficient space for chunk header is reserved at the beginning, but later when all chunk's data is written and it's time to write actual values into the header (and this is JSON-like formatted string with chunk attributes) it turn's out that reserved space is insufficient. Your massive entries removal indded may trigger that.  Latest version (1.4.200) does more conservative estimate, so chance of this happening is reduced, but not eliminated. We just need to handle that condition in a more robust way (it is totally recoverable), instead of throwing the towel.

As far as automatic reopening goes - it not always a good idea, because root cause for premature closure could be anything. If you think that you need that, it can be easily implemented by application.

[Clyde Stubbs]

Thanks for the feedback. I have updated the app to use 1.4.200 and will see how that goes. I had to patch the code in MVStore.java in 1.4.200 to enable it to run on older Android JVMs. I have submitted a PR with the patch. There are other issues in more recent commits (1.4.201) to the code base that introduce other Java 8 classes which are not supported on Android.

On 14 Nov 2019, at 13:49, Andrei Tokar wrote:

First of all - it has nothing to do with disk space available. It is a bug in MVStore. When chunk is saved and insufficient space for chunk header is reserved at the beginning, but later when all chunk's data is written and it's time to write actual values into the header (and this is JSON-like formatted string with chunk attributes) it turn's out that reserved space is insufficient. Your massive entries removal indded may trigger that.  Latest version (1.4.200) does more conservative estimate, so chance of this happening is reduced, but not eliminated. We just need to handle that condition in a more robust way (it is totally recoverable), instead of throwing the towel.

As far as automatic reopening goes - it not always a good idea, because root cause for premature closure could be anything. If you think that you need that, it can be easily implemented by application.

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/h2-database/2a231bda-7be1-484d-8e30-edf43c20cee9%40googlegroups.com.