Exception when using MULTI_THREADED=1

[Uli]

Hi,

when using MULTI_THREADED=1 sometimes I get an exception when running a query:

org.h2.jdbc.JdbcSQLException: General error: "java.lang.RuntimeException: page[50387075] data leaf table:124233 TEMP_RESULT_SET_124233 entries:0 parent:0 keys:null offsets:null parent 0 expected 50392430"; SQL statement:
SELECT 1376648402056  "time", ifnull(v1.value, v.c1), ifnull(v2.value, v.c1), ifnull(v3.value, v.c1) , ifnull(v4.value, v.c1)
FROM values('null') v
LEFT OUTER JOIN (
    SELECT vi1.value
        FROM rvalues_110501 vi1
        WHERE vi1.time = (SELECT min(time) FROM rvalues_110501 WHERE timeend > 1376648402056) AND vi1.time < 1376648402056
) v1 ON 1=1
LEFT OUTER JOIN (
    SELECT vi2.value
        FROM rvalues_110398 vi2
        WHERE vi2.time = (SELECT min(time) FROM rvalues_110398 WHERE timeend > 1376648402056) AND vi2.time < 1376648402056
) v2 ON 1=1
LEFT OUTER JOIN (
    SELECT vi3.value
        FROM rvalues_110689 vi3
        WHERE vi3.time = (SELECT min(time) FROM rvalues_110689 WHERE timeend > 1376648402056) AND vi3.time < 1376648402056
) v3 ON 1=1
LEFT OUTER JOIN (
    SELECT vi4.value
        FROM rvalues_110475 vi4
        WHERE vi4.time = (SELECT min(time) FROM rvalues_110475 WHERE timeend > 1376648402056) AND vi4.time < 1376648402056
) v4 ON 1=1
UNION
SELECT times.time, v1.value, v2.value, v3.value, v4.value
FROM
  (
   SELECT DISTINCT v.time
   FROM rvalues_110501 v
   WHERE v.time BETWEEN 1376648402056 and 1376907602056
   UNION
   SELECT DISTINCT v.time
   FROM rvalues_110398 v
   WHERE v.time BETWEEN 1376648402056 and 1376907602056
   UNION
   SELECT DISTINCT v.time
   FROM rvalues_110689 v
   WHERE v.time BETWEEN 1376648402056 and 1376907602056
   UNION
   SELECT DISTINCT v.time
   FROM rvalues_110475 v
   WHERE v.time BETWEEN 1376648402056 and 1376907602056
  ) AS times
LEFT OUTER JOIN
    rvalues_110501 v1
    ON times.time=v1.time AND v1.time BETWEEN 1376648402056 AND 1376907602056
LEFT OUTER JOIN
    rvalues_110398 v2
    ON times.time=v2.time AND v2.time BETWEEN 1376648402056 AND 1376907602056
LEFT OUTER JOIN
    rvalues_110689 v3
    ON times.time=v3.time AND v3.time BETWEEN 1376648402056 AND 1376907602056
LEFT OUTER JOIN
    rvalues_110475 v4
    ON times.time=v4.time AND v4.time BETWEEN 1376648402056 AND 1376907602056
ORDER BY "time"
LIMIT 500001
Caused by: java.lang.RuntimeException: page[50387075] data leaf table:124233 TEMP_RESULT_SET_124233 entries:0 parent:0 keys:null offsets:null parent 0 expected 50392430
        at org.h2.message.DbException.throwInternalError(DbException.java:228)
        at org.h2.index.PageDataIndex.getPage(PageDataIndex.java:244)
        at org.h2.index.PageDataNode.addRowTry(PageDataNode.java:129)
        at org.h2.index.PageDataIndex.addTry(PageDataIndex.java:172)
        at org.h2.index.PageDataIndex.add(PageDataIndex.java:135)
        at org.h2.table.RegularTable.addRow(RegularTable.java:124)
        ... 32 more

I am using H2 1.3.172.
The database URL is: jdbc:h2:data/properties;LOCK_TIMEOUT=86400000;MULTI_THREADED=1
My database is quite large (around 15000 tables with 104GB size).

Unfortunately I cannot provide a small simple test case. I can only give some information about the circumstances.

A small explanation for the query:
The query selects the time based data from four different properties (one table for one propery) and merges them as one table.

So having the following tables (with two columns time and value):
A:
2000 A_1
2500 A_2

B:
2100 B_1
2500 B_2

C:
2300 C_1
2400 C_2

D:
2300 D_1

would result in:
      A     B     C      D
2000  A_1  null  null  null
2100 null   B_1  null  null
2300 null  null   C_1   D_1
2400 null  null   C_2  null
2500  A_2   B_2  null  null


I guess the exception is thrown due to the fact that new data is added to the tables while the query is running?
The database seems to be ok after the exception is thrown (so no corruption).

Maybe the exception gives you a hint about a problematic source code section?

Uli

[Noel Grandin]

This looks a lot of a temp -result-set bug I fixed in 1.3.173.

I suggest updating to the newer version.

[Uli]

Great!
I'll try 1.3.173.

[Uli]

I've updated to 1.3.173. Now I sometimes get the following Exception:

09-19 12:15:38 jdbc[2]: exception
org.h2.jdbc.JdbcSQLException: File corrupted while reading record: "page[61779266] b-tree leaf table:212525 entries:0". Possible solution: use the recovery tool; SQL statement:
  <-- as described in previous posts -->
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:331)
    at org.h2.message.DbException.get(DbException.java:171)
    at org.h2.message.DbException.get(DbException.java:148)
    at org.h2.index.PageDataIndex.getPage(PageDataIndex.java:239)
    at org.h2.index.PageDataNode.addRowTry(PageDataNode.java:129)
    at org.h2.index.PageDataNode.addRowTry(PageDataNode.java:130)

    at org.h2.index.PageDataIndex.addTry(PageDataIndex.java:172)
    at org.h2.index.PageDataIndex.add(PageDataIndex.java:135)

    at org.h2.table.RegularTable.addRow(RegularTable.java:125)
    at org.h2.result.ResultTempTable.addRow(ResultTempTable.java:113)
    at org.h2.result.LocalResult.addRow(LocalResult.java:267)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:194)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.index.ViewIndex.find(ViewIndex.java:253)
    at org.h2.index.BaseIndex.find(BaseIndex.java:126)
    at org.h2.index.IndexCursor.find(IndexCursor.java:155)
    at org.h2.table.TableFilter.next(TableFilter.java:328)
    at org.h2.command.dml.Select.queryFlat(Select.java:518)
    at org.h2.command.dml.Select.queryWithoutCache(Select.java:625)
    at org.h2.command.dml.Query.query(Query.java:314)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:184)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.Query.query(Query.java:36)
    at org.h2.command.CommandContainer.query(CommandContainer.java:91)
    at org.h2.command.Command.executeQuery(Command.java:195)
    at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:177)
    at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:156)
    ...

09-19 12:15:38 jdbc[9]: exception
org.h2.jdbc.JdbcSQLException: File corrupted while reading record: "page[61779265] data leaf table:212524 TEMP_RESULT_SET_212524 entries:0 parent:0 keys:null offsets:null". Possible solution: use the recovery tool; SQL statement:
  <-- as described in previous posts -->
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:331)
    at org.h2.message.DbException.get(DbException.java:171)
    at org.h2.message.DbException.get(DbException.java:148)
    at org.h2.index.PageBtreeIndex.getPage(PageBtreeIndex.java:154)
    at org.h2.index.PageBtreeNode.find(PageBtreeNode.java:304)
    at org.h2.index.PageBtreeIndex.find(PageBtreeIndex.java:180)
    at org.h2.index.PageBtreeIndex.find(PageBtreeIndex.java:171)
    at org.h2.result.ResultTempTable.find(ResultTempTable.java:208)
    at org.h2.result.ResultTempTable.addRow(ResultTempTable.java:111)
    at org.h2.result.ResultTempTable.addRows(ResultTempTable.java:124)
    at org.h2.result.LocalResult.addRow(LocalResult.java:263)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:191)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.index.ViewIndex.find(ViewIndex.java:253)
    at org.h2.index.BaseIndex.find(BaseIndex.java:126)
    at org.h2.index.IndexCursor.find(IndexCursor.java:155)
    at org.h2.table.TableFilter.next(TableFilter.java:328)
    at org.h2.command.dml.Select.queryFlat(Select.java:518)
    at org.h2.command.dml.Select.queryWithoutCache(Select.java:625)
    at org.h2.command.dml.Query.query(Query.java:314)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:184)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.Query.query(Query.java:36)
    at org.h2.command.CommandContainer.query(CommandContainer.java:91)
    at org.h2.command.Command.executeQuery(Command.java:195)
    at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:177)
    at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:156)
    ...

09-19 12:15:55 jdbc[7]: exception
org.h2.jdbc.JdbcSQLException: General error: "java.lang.RuntimeException: Undo entry not written"; SQL statement:
  <-- as described in previous posts -->
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:331)
    at org.h2.message.DbException.get(DbException.java:160)
    at org.h2.message.DbException.convert(DbException.java:283)
    at org.h2.table.RegularTable.addRow(RegularTable.java:143)
    at org.h2.result.ResultTempTable.addRow(ResultTempTable.java:113)
    at org.h2.result.LocalResult.addRow(LocalResult.java:267)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:191)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:183)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.index.ViewIndex.find(ViewIndex.java:253)
    at org.h2.index.BaseIndex.find(BaseIndex.java:126)
    at org.h2.index.IndexCursor.find(IndexCursor.java:155)
    at org.h2.table.TableFilter.next(TableFilter.java:328)
    at org.h2.command.dml.Select.queryFlat(Select.java:518)
    at org.h2.command.dml.Select.queryWithoutCache(Select.java:625)
    at org.h2.command.dml.Query.query(Query.java:314)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.SelectUnion.queryWithoutCache(SelectUnion.java:184)
    at org.h2.command.dml.SelectUnion.query(SelectUnion.java:408)
    at org.h2.command.dml.Query.query(Query.java:284)
    at org.h2.command.dml.Query.query(Query.java:36)
    at org.h2.command.CommandContainer.query(CommandContainer.java:91)
    at org.h2.command.Command.executeQuery(Command.java:195)
    at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:177)
    at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:156)
    ...
Caused by: java.lang.RuntimeException: Undo entry not written
    at org.h2.message.DbException.throwInternalError(DbException.java:230)
    at org.h2.store.PageLog.addUndo(PageLog.java:485)
    at org.h2.store.PageStore.update(PageStore.java:1061)
    at org.h2.index.PageBtreeLeaf.addRow(PageBtreeLeaf.java:165)
    at org.h2.index.PageBtreeLeaf.addRowTry(PageBtreeLeaf.java:101)
    at org.h2.index.PageBtreeNode.addRowTry(PageBtreeNode.java:201)
    at org.h2.index.PageBtreeNode.addRowTry(PageBtreeNode.java:201)
    at org.h2.index.PageBtreeIndex.addRow(PageBtreeIndex.java:95)
    at org.h2.index.PageBtreeIndex.add(PageBtreeIndex.java:86)
    at org.h2.table.RegularTable.addRow(RegularTable.java:125)
    ... 46 more

  

[Noel Grandin]

Did you start from a clean DB?

That looks like left-over corruption from the earlier problem.
You may have to do a dump and restore.

[Uli]

Yes, I started with a clean DB using 1.3.173.

Also yes that the database need to be restored.

I will email you the trace log. Maybe you can find some more information about the problem.

Uli

[Noel Grandin]

Hi

OK, I can't see anything useful in the trace log.

So let's run through Thomas' checklist - sorry but this going to be a
little long winded:

- What is your database URL?
- Did you use LOG=0 or LOG=1? Did you read the FAQ about it?
- Did the system ever run out of disk space?
- Could you send the full stack trace of the exception including message
text?
- Did you use SHUTDOWN DEFRAG or the database setting DEFRAG_ALWAYS with
H2 version 1.3.159 or older?
- How many connections does your application use concurrently?
- Do you use temporary tables?
- With which version of H2 was this database created?
You can find it out using:
select * from information_schema.settings where name='CREATE_BUILD'
or have a look in the SQL script created by the recover tool.
- Did the application run out of memory (once, or multiple times)?
- Do you use any settings or special features (for example cache settings,
two phase commit, linked tables)?
- Do you use any H2-specific system properties?
- Is the application multi-threaded?
- What operating system, file system, and virtual machine
(java -version) do you use?
- How did you start the Java process (java -Xmx... and so on)?
- Is it (or was it at some point) a networked file system?
- How big is the database (file sizes)?
- How much heap memory does the Java process have?
- Is the database usually closed normally, or is process terminated
forcefully or the computer switched off?
- Is it possible to reproduce this problem using a fresh database
(sometimes, or always)?
- Are there any other exceptions (maybe in the .trace.db file)?
Could you send them please?
- Do you still have any .trace.db files, and if yes could you send them?
- Could you send the .h2.db file where this exception occurs?

[Uli]

OK, I can't see anything useful in the trace log.

So let's run through Thomas' checklist - sorry but this going to be a
little long winded:

That's ok :-)
 

- What is your database URL?

jdbc:h2:~/data/rcl;LOCK_TIMEOUT=86400000;MULTI_THREADED=1
The database is embedded in my application.

- Did you use LOG=0 or LOG=1? Did you read the FAQ about it?

No, I don't use LOG
 

- Did the system ever run out of disk space?

No. (For sure, 3 TB free disk space :-)
 

- Could you send the full stack trace of the exception including message
text?

I've sent the trace log to you so you should already have it.
 

- Did you use SHUTDOWN DEFRAG or the database setting DEFRAG_ALWAYS with
H2 version 1.3.159 or older?

The database was created with 1.3.173, so 1.3.159 was never used with the database.
I am using SHUTDOWN DEFRAG once per week.
 

- How many connections does your application use concurrently?

I am using the JdbcConnectionPool. Using this pool there are not more than 10 parallel connections.
I am also running the H2 webserver. So there might be one additional connection.
 

- Do you use temporary tables?

No.
 

- With which version of H2 was this database created?
     You can find it out using:
     select * from information_schema.settings where name='CREATE_BUILD'
     or have a look in the SQL script created by the recover tool.

The recovered SQL file tells:
INSERT INTO O_0 VALUES(1, 0, 6, 'SET CREATE_BUILD 173');
 

- Did the application run out of memory (once, or multiple times)?

No.
 

- Do you use any settings or special features (for example cache settings,
     two phase commit, linked tables)?

Usually not. With this database I created one linked table once. But removed it instantly.
 

- Do you use any H2-specific system properties?

No
 

- Is the application multi-threaded?

Yes

 

- What operating system, file system, and virtual machine
     (java -version) do you use?

Filesytem is ext4 on OpenSUSE 11.2, 64 bit kernel 2.6.31.14-0.8-default #1 SMP

Java(TM) SE Runtime Environment (build 1.7.0_11-b21)
Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)

 

- How did you start the Java process (java -Xmx... and so on)?

Here are the interesting parts:

java -Xmx4g -XX:-UseSplitVerifier -cp myapp.jar com.xxx.xxx.xxx.MyApp <some parameter of the app>
 

- Is it (or was it at some point) a networked file system?

No
 

- How big is the database (file sizes)?

~ 125 GB

- How much heap memory does the Java process have?

4 GB
 

- Is the database usually closed normally, or is process terminated
     forcefully or the computer switched off?

Usually the database is closed once a week by running SHUTDOWN DEFRAG.
It is possible that the database gets closed due to shutdown the server.
 

- Is it possible to reproduce this problem using a fresh database
     (sometimes, or always)?

I've seen this with a fresh database but I cannot give a testcase that triggers the problem.
 

- Are there any other exceptions (maybe in the .trace.db file)?
     Could you send them please?
- Do you still have any .trace.db files, and if yes could you send them?
- Could you send the .h2.db file where this exception occurs?

125GB... Do you really want to have it?



Thanks for having a look on it!
Uli

[Noel Grandin]

Thanks for the extra info.

Could you perhaps try running your system with CHECK2 enabled?

Add an extra command line parameter like this:
    -Dh2.check2=true

And see if you can trigger the problem?

Note that it will slow things down a little, but hopefully not so much that it hides the problem.

[Uli]

Ok, I will do that.

But first I have to recover the database and this will take some time...

I will post again if the error occurs again.

Uli

[Noel Grandin]

Yeah, sorry, I have to warn you that I'm not hopeful of a quick fix -
these kinds of things are quite difficult to track down.
Without a test-case I'm shooting in the dark.

If you have limited time now you want to just try running your app with
MULTI_THREADED off, which is going to be more reliable.

[Uli]

Hi Noel,

I did have some time today and I managed to write a test case (for 1.3.174) :-)

When running the application you should get:

General error: "java.lang.RuntimeException: Undo entry not written"; SQL statement:

UPDATE values_7 SET timeend = 21332841064754 WHERE timeend = 9223372036854775807 [50000-174]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:332)
    at org.h2.message.DbException.get(DbException.java:161)
    at org.h2.message.DbException.convert(DbException.java:284)
    at org.h2.table.RegularTable.removeRow(RegularTable.java:394)
    at org.h2.table.Table.updateRows(Table.java:449)
    at org.h2.command.dml.Update.update(Update.java:144)
    at org.h2.command.CommandContainer.update(CommandContainer.java:79)
    at org.h2.command.Command.executeUpdate(Command.java:253)
    at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:181)
    at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:156)
    at com.gematronik.analyst.db.test.H2MultiThreadTest1$Inserter.run(H2MultiThreadTest1.java:121)
    at java.lang.Thread.run(Thread.java:744)

Caused by: java.lang.RuntimeException: Undo entry not written

    at org.h2.message.DbException.throwInternalError(DbException.java:231)

    at org.h2.store.PageLog.addUndo(PageLog.java:485)
    at org.h2.store.PageStore.update(PageStore.java:1061)

    at org.h2.index.PageBtreeLeaf.remove(PageBtreeLeaf.java:239)
    at org.h2.index.PageBtreeNode.remove(PageBtreeNode.java:332)
    at org.h2.index.PageBtreeNode.remove(PageBtreeNode.java:332)
    at org.h2.index.PageBtreeIndex.remove(PageBtreeIndex.java:239)
    at org.h2.table.RegularTable.removeRow(RegularTable.java:376)
    ... 8 more

If the problem does not occur after 10 minutes just remove the database file and restart the application. Usually I get the error around each 3 attempts...

The test case seems to be quite similar to the problem described here: https://groups.google.com/forum/#!searchin/h2-database/Undo$20entry$20not$20written/h2-database/E_2gUWb-iGc/rkbaekWMUw4J

I'll be off until next year but I hope the test case will give you some more hints!

Uli

[Uli]

I managed to strip it down a bit more...

[Noel Grandin]

On 2013-12-13 17:38, Uli wrote:
> I managed to strip it down a bit more...

Thanks a lot for the test-case!

Unfortunately, it proves to me that our MULTI_THREADED=1 support is not at all safe.

For example, in the stacktrace below:

java.lang.RuntimeException: Undo entry not written
at org.h2.message.DbException.throwInternalError(DbException.java:231)

at org.h2.store.PageLog.addUndo(PageLog.java:495)
at org.h2.store.PageStore.update(PageStore.java:1061)
at org.h2.index.PageBtreeLeaf.remove(PageBtreeLeaf.java:239) // synched on PageStore
at org.h2.index.PageBtreeNode.remove(PageBtreeNode.java:332) // not safe
at org.h2.index.PageBtreeNode.remove(PageBtreeNode.java:332) // not safe
at org.h2.index.PageBtreeIndex.remove(PageBtreeIndex.java:239) // not safe
at org.h2.table.RegularTable.removeRow(RegularTable.java:376) // not safe
at org.h2.command.dml.Delete.update(Delete.java:94)
at org.h2.command.CommandContainer.update(CommandContainer.java:79)
at org.h2.command.Command.executeUpdate(Command.java:253) // synched on Session

at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:181)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:156)

at org.h2.test.H2MultiThreadTest2$Deletor.run(H2MultiThreadTest2.java:100)
at java.lang.Thread.run(Thread.java:662)

The layers up to org.h2.command.CommandContainer are synchronized around the Session object.
And the layers from org.h2.store.PageStore down are synchronized around the PageStore object.

But the layers in between have no synchronization, but they are shared between multiple sessions.

I am afraid I have no idea how to fix this, and it looks like quite a big job.

Regards, Noel.

[Uli]

Hi Noel,

what a pity! Thanks for digging into the synching problems!

It sounds to me as if MULTI_THREADED=1 should never be used and its usage is strongly inadvisable. Can you add some strong warnings to the documentation so that other users do not fall into the same trap?

Thanks!
Uli

Unfortunately, it proves to me that our MULTI_THREADED=1 support is not at all safe.

For example, in the stacktrace below:

...

[Ryan How]

I've been using it in a production environment for several years without issue. But it isn't very high concurrency, so I've probably just been lucky that it hasn't had issues.

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/groups/opt_out.

[Thomas Mueller]

Hi,

The operations on RegularTable should be protected with the table level locking mechanism, and each b-tree belongs to exactly one table. So in theory it should work. But I understand there might be some bugs. My priority is to get the MVStore ready, and then make multi-threading work.

Regards,

Thomas

[Uli]

Hi,

for the sake of completeness I have posted issue #539 that contains the testcase and a link to this thread.

Uli