File corruption with nioMemLZF

Anatolii K

unread,

Jan 2, 2017, 7:02:10 AM1/2/17

to H2 Database

Continuation of https://groups.google.com/forum/#!topic/h2-database/oHpaE_LMLz4

I'm getting following error with nioMemLZF (url= jdbc:h2:nioMemLZF:db):

org.h2.jdbc.JdbcSQLException: General error: "java.lang.IllegalStateException: File corrupted in chunk 28, expected page length 4..32, got -1621686845 [1.4.193/6]"; SQL statement:

update account set balance = ? where id = ? [50000-193]

at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)

at org.h2.message.DbException.get(DbException.java:168)

at org.h2.message.DbException.convert(DbException.java:295)

at org.h2.command.Command.executeUpdate(Command.java:266)

at org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:201)

at Bug_H2_4$Arte.call(Bug_H2_4.java:99)

at Bug_H2_4$Arte.call(Bug_H2_4.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.IllegalStateException: File corrupted in chunk 28, expected page length 4..32, got -1621686845 [1.4.193/6]

at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:766)

at org.h2.mvstore.Page.read(Page.java:649)

at org.h2.mvstore.Page.read(Page.java:195)

at org.h2.mvstore.MVStore.readPage(MVStore.java:1952)

at org.h2.mvstore.MVMap.readPage(MVMap.java:738)

at org.h2.mvstore.Page.getChildPage(Page.java:217)

at org.h2.mvstore.MVMap.binarySearch(MVMap.java:470)

at org.h2.mvstore.MVMap.binarySearch(MVMap.java:471)

at org.h2.mvstore.MVMap.get(MVMap.java:452)

at org.h2.mvstore.db.TransactionStore$TransactionMap.getValue(TransactionStore.java:1203)

at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:1180)

at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:1148)

at org.h2.mvstore.db.MVPrimaryIndex.getRow(MVPrimaryIndex.java:215)

at org.h2.mvstore.db.MVTable.getRow(MVTable.java:463)

at org.h2.mvstore.db.MVSecondaryIndex$MVStoreCursor.get(MVSecondaryIndex.java:489)

at org.h2.index.IndexCursor.get(IndexCursor.java:288)

at org.h2.table.TableFilter.get(TableFilter.java:594)

at org.h2.command.dml.Update.update(Update.java:109)

at org.h2.command.CommandContainer.update(CommandContainer.java:98)

Please try attached test. It fails after 10-60 seconds run

Bug_H2_4.java

Anatolii K

unread,

Jan 19, 2017, 6:50:03 AM1/19/17

to H2 Database

Sorry for bothering you, but could you please look at it

Noel Grandin

unread,

Jan 19, 2017, 8:32:49 AM1/19/17

to h2-da...@googlegroups.com

I have pushed a fix for this, it now survives your test.

I had to simplify the current code and just synchronize the methods, which means it may lose some performance. If this
is a problem for you, I'm sorry, but you'll have to come up with some patches yourself.

Anatolii K

unread,

Jan 19, 2017, 11:36:17 AM1/19/17

to H2 Database

Sorry to say so, but performance dropped dramatically.

Previously compressed storage was just a bit slower than uncompressed.

But now it become almost 100 times slower!

Noel Grandin

unread,

Jan 19, 2017, 12:51:36 PM1/19/17

to H2 Database

Can you post your performance test code?

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

Anatolii K

unread,

Jan 20, 2017, 7:23:41 AM1/20/17

to H2 Database

Please see attached.

At my notebook the test takes 0.9s without compression and 50s with compression

Bug_H2_5.java

Noel Grandin

unread,

Jan 20, 2017, 9:26:20 AM1/20/17

to h2-da...@googlegroups.com

I have pushed a fix for this.

Anatolii K

unread,

Jan 20, 2017, 10:04:07 AM1/20/17

to H2 Database

Have you aready commited at github? I don't see it

Noel Grandin

unread,

Jan 20, 2017, 12:41:21 PM1/20/17

to h2-da...@googlegroups.com

Ah that was silly, forgot to hit the sync button.Done now.

Anatolii K

unread,

Jan 21, 2017, 3:57:28 PM1/21/17

to H2 Database

I found one more scenario where compressed database is 30 times slower - see attachment

But now it's not related to synchronization because the test is single threaded

Bug_H2_6.java

Noel Grandin

unread,

Jan 22, 2017, 8:44:33 AM1/22/17

to h2-da...@googlegroups.com

Firstly note that this test generates fairly widely varying results, so you need to run it a couple of times and average the output.

That said, this performance here is completely dependant on the CACHE_SIZE constant in FileNioMemData.

A value of anything > approx 80 produces decent performance.

However, the point is that now we are approaching trade-off zone. If we want more performance, we need to let enough data remain uncompressed to fit the working set of your queries. However, that will be different from program to program, and even different depending on what the user is doing with the program.

But a larger cache means less compressed data, which means the DB will use more RAM.

So right now I am leaning towards sizing the cache as a % of the size of the DB, and adding a setting to allow users to override that %.

Anatolii K

unread,

Jan 23, 2017, 2:42:19 AM1/23/17

to H2 Database

Thank you Noel for the explanation

I think it would be very useful to be able to control the cach size. And I would like to be able to do it dynamically, without database restart

Noel Grandin

unread,

Jan 23, 2017, 2:45:07 AM1/23/17

to h2-da...@googlegroups.com

On 2017/01/23 9:42 AM, Anatolii K wrote:
> Thank you Noel for the explanation
>
> I think it would be very useful to be able to control the cach size. And I would like to be able to do it dynamically,
> without database restart
>
>

Yeah, sorry, the "dynamic without restart" part is not going to happen, not unless you supply a patch yourself. My time
budget for this particular problem is pretty much exhausted.
And even then I don't know how we'd do that, we don't have dynamic control for anything else down in that part of the code.

Noel Grandin

unread,

Jan 23, 2017, 4:55:38 AM1/23/17

to h2-da...@googlegroups.com

I have committed a cache-sizing feature for this filesystem

Anatolii K

unread,

Jan 23, 2017, 6:16:32 AM1/23/17

to H2 Database

One more problem :( :

It happens rarely and unpredictable

Caused by: java.lang.ClassCastException: org.h2.value.ValueNull cannot be cast to org.h2.value.ValueArray

at org.h2.mvstore.db.MVPrimaryIndex$MVStoreCursor.get(MVPrimaryIndex.java:402)

at org.h2.mvstore.db.MVPrimaryIndex$MVStoreCursor.getSearchRow(MVPrimaryIndex.java:412)

at org.h2.index.IndexCursor.getSearchRow(IndexCursor.java:293)

at org.h2.table.TableFilter.next(TableFilter.java:499)

at org.h2.command.dml.Select.queryGroup(Select.java:336)

at org.h2.command.dml.Select.queryWithoutCache(Select.java:650)

at org.h2.command.dml.Query.query(Query.java:341)

at org.h2.command.dml.Query.query(Query.java:309)

at org.h2.command.dml.Query.query(Query.java:36)

at org.h2.command.ddl.Analyze.analyzeTable(Analyze.java:115)

at org.h2.mvstore.db.MVTable.analyzeIfRequired(MVTable.java:738)

at org.h2.mvstore.db.MVTable.addRow(MVTable.java:725)

at org.h2.table.Table.updateRows(Table.java:483)

at org.h2.command.dml.Update.update(Update.java:145)

at org.h2.command.CommandContainer.update(CommandContainer.java:98)

at org.h2.command.Command.executeUpdate(Command.java:258)

Andrey Belyaev

unread,

Apr 4, 2017, 3:14:03 AM4/4/17

to H2 Database

I faced the same issue today.

Reply all

Reply to author

Forward