General error: "java.lang.IllegalStateException: Chunk 2408 not found [1.4.197/9]"

[Ivaylo Dobrikov]

Hello,

I recently updated our H2 to version 1.4.197. After the update we observed that sometimes out application cannot execute some of the SQL statement, terminating with the following error:

2018-08-19 16:26:25,969 INFO  - running processor 388

2018-08-19 16:26:54,713 ERROR - General error: "java.lang.IllegalStateException: Chunk 2408 not found [1.4.197/9]"; SQL statement:

org.h2.jdbc.JdbcSQLException: General error: "java.lang.IllegalStateException: Chunk 2408 not found [1.4.197/9]"; SQL statement:

UPDATE TABLE_NAME

       SET columnName = ....

            at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)

            at org.h2.message.DbException.get(DbException.java:168)

            at org.h2.message.DbException.convert(DbException.java:307)

            at org.h2.command.Command.executeUpdate(Command.java:274)

            at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:233)

            at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:205)

            ... 4 more

Caused by: java.lang.IllegalStateException: Chunk 2408 not found [1.4.197/9]

            at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:875)

            at org.h2.mvstore.MVStore.getChunk(MVStore.java:993)

            at org.h2.mvstore.MVStore.readPage(MVStore.java:2034)

            at org.h2.mvstore.MVMap.readPage(MVMap.java:547)

            at org.h2.mvstore.Page$NonLeaf.getChildPage(Page.java:943)

            at org.h2.mvstore.Cursor.hasNext(Cursor.java:52)

            at org.h2.mvstore.tx.TransactionMap$1.fetchNext(TransactionMap.java:567)

            at org.h2.mvstore.tx.TransactionMap$1.next(TransactionMap.java:609)

            at org.h2.mvstore.db.MVSecondaryIndex$MVStoreCursor.next(MVSecondaryIndex.java:511)

            at org.h2.index.IndexCursor.next(IndexCursor.java:315)

            at org.h2.table.TableFilter.next(TableFilter.java:502)

            at org.h2.command.dml.Update.update(Update.java:112)

            at org.h2.command.CommandContainer.update(CommandContainer.java:102)

            at org.h2.command.Command.executeUpdate(Command.java:261)

            ... 6 more

Previously, we used a very old version of H2 (1.2) and we haven't experienced such problems. Can anyone help me and maybe explain in which circumstances does this error occur?

Thanks a lot in advance!

Greetings

Ivaylo Dobrikov

[Andrei Tokar]

Hi Ivaylo,

Circumstances under which this bug will appear may vary greatly (because it is a fairly low-level problem), but I believe it is fixed after last release. If you can build H2 from the latest code, I would be very interested in your feedback about whether or not you problem is resolved, and if you still have a reproducible test case, I'll gladly take it from there.

Thanks,

Andrei

[Ivaylo Dobrikov]

Hi Andrei,

thank you for the answer. The problem occurs that I report above occurs on H2 1.4.197/9. H2 1.4.197 is already the latest version of H2 that can be downloaded from the website.

I will try these days to check out the latest source code and build and then test again. Do you know which commit on GitHub fixes the issue? Currently, we have a workaround for the problem. We just disabled the new MVStore storage by setting MV_STORE to FALSE.

However, I have another question. After upgrading to H2 1.4 (previously we used H2 1.2) I observed a performance drop by executing a huge number of select statements. After setting MV_STORE to FALSE the performance issues disappeared. Is this MVStore performance problem well-known and if so, do you know if the optimisation of MVStore is in the TODO list for the next release?

Thanks and greetings,

Ivaylo.

[Andrei Tokar]

It depends what do you mean by "executing huge number of select statements". MVStore is somewhat slower than older PageStore (MV_STORE=FALSE) in a single-threaded scenario, but with latest changes (post 1.4.97) that gap is down to about 20-40%. But multi-threaded scenario is really where MVStore will shine (again in post 1.4.97). Try to build from source and see for yourself. Optimization of MVStore is not just on the TODO list for next release, many are out there already.

[Ivaylo Dobrikov]

Hi Andrei,

I built today from the source (latest merge on the master branch 2018-08-21 15:01:11) and we have tested whether the IllegalStateException above occurs when the newest H2 jar is used. The error does not appear any more.

However, the performance issues are still existing. I compared the SQL Query execution times of some SQL statements between MVStore and PageStore and observed the following differences:

 - Inserts perform equally good for both

 - The execution times for Select-queries on a 7 column table having more than 305 000 rows are 2-3 times larger for MVStore as for PageStore. Even the execution times for queries such as 'SELECT * FROM tableName' are much larger for MVStore as for PageStore. In my case testing this query in Squirl I observed execution times on MVStore nearly equal to 4 sec and on PageStore nearly equal to 1.5 sec.

We have, for example, one test case where we insert a huge amount of data to H2 and then read that data and write it to an Excel file. The execution times for inserting the data were similar, whereas for selecting the data 4 times larger for MVStore. The gap is obviously much more than 20-40%. 

Greentings

Ivaylo

[Ivaylo Dobrikov]

In another discussion I have already mentioned the problem with the huge execution times: https://groups.google.com/forum/#!topic/h2-database/Fcj2EKTyb-E

[Andrei Tokar]

Hi Ivaylo,
I am glad to hear that “Chunk not found” bug is no longer appear with latest build.
That other discussion, you have mentioned, is related to a different performance problem, and according to Noel Grandin has been addressed in latest build, as well.
Going back to your claim of slow “SELECT * FROM” - my experiments with 300000 rows, 7 column table shows that execution time with PageStore is prety much consistent across 1.3.175, 1.4.197 and latest snapshot. MVStore execution time in 1.4.197 is 1.5 times that, while in latest snapshot it’s 2.5 times faster than PgStore.
So, apparently your case is different from mine in some important aspects (indexes, constraints, column tipes). I am afraid, that until you can provide simple, standalone, reproducible test case, not much we can do here.

[Ivaylo Dobrikov]

Thank you for the response! I will try to create a standalone tests case next week.

[Ivaylo Dobrikov]

Hi Andrei,

I created a standalone test case. I wrote a simple Java application that can be executed from the command line. 

You can download the application from here: https://www.dropbox.com/sh/3mgnl310ltum389/AAA8Km-ggSQpXXiktyP6lfbHa?dl=0. In the zip file you will find the file h2Standalone.jar that you need to execute in order to generate a h2 data base as a file and insert a big amount of data in a table. In the zip file you will find also the h2 jar that I built from the sources last week. Remove the suffixes '.removeme' before executing the jar below. 

Extract both jars in some folder and execute from there the following commands:

1. java -jar h2Standalone.jar -dir "dir1"

2. java -jar h2Standalone.jar -dir "dir2" -mvstore

The first command creates a h2 data base with PgStore, wehreas the second one a h2 data base with MVStore. dir1 and dir2 are the absolute paths to existing directories and they must be different. Note that dir1 and dir2 must be existing directories, otherwise the application will terminate with an error. Both commands create a table named "TableB" and insert in this table nearly 306 000 items. The command 'java -jar h2Standalone.jar -help' gives also hints how the application should be called. Note that -mvstore must be at the end of the command.

I used Squirl to test the execution times of "SELECT * FROM TableB" on both generated data bases. For the current h2 driver the execution times for selecting all items with PgStore are 2 times faster than with MVStore.

In case you want to execute the test case with a newer version of h2 you can just change the h2.jar file in the directory where h2standalone.jar is placed.

If you have questions about how to execute the test case, then don't hesitate to write.

I really hope that this test case can help to optimize the executions of select queries in H2. Getting execution times that are 2.5 times faster will improve many tasks in our main application.

Thank you in advance!

Greetings,

Ivaylo

[Andrei Tokar]

Hi Ivaylo,

It would be MUCH easier if you just post java code here instead.

BTW, table population with auto-commit on, no batches, and embedded data instead of prepared statement is not the fastest way of doing this, so when you say "the execution times for inserting the data were similar", it realy means "similarly terrible".

Going back to SELECT, my results for "SELECT * FROM TableB" execution time:

v. 1.4.197:

PgStore: 2.3 sec.

MVStore: 5.1 sec.

v. 1.4.198 SNAPSHOT, built from commit ac05e08c8aafe6d8680ec09b9b2ee371355ffb4c, which you mentioned (With latest snapshot results are similar):

PgStore: 2.4 sec.

MVStore: 1.2 sec.

It seems like there is absolutely nothing special about your test case, and results are consistent with what I posted above for some 7 column 300000 rows table of my own making. Most likely, when you was measuring time with some SQL client (Squirl ?), it does not pick up driver, you expect. Why not just run your test as standalone java program? And please build latest clean jar, by following instructions, because one you posted is a bit weird, even if you just look at it's directory listing.

The bottom line is:

in 1.4.197 this select ~ 2 times slower with MVStore vs. PgStore, whereas in latest version it's 2 times faster.

Best of luck!

Andrei

[Ivaylo Dobrikov]

Hi Andrei,

today I checked out the latest version of the master branch (0ff4499b032104e0f83d2d7c5978e5273547a480) and it seems that the latest build of H2 has resolved the performance issues for MVStore. I am pretty sure that I built from the latest source code last week and tested the H2 driver on our application with this build.

Our tests have shown some improvements in the execution times with the latest H2 version. I couldn't observe a significant improvement in the execution times (, e.g. 2-3 times faster than prevously), but the improvements are satisfactory.

Thank you for the great support!

Greetings,

Ivaylo