Concurrency at org.h2.store.fs.FileBase.read()
129 views
Skip to first unread message
Anatolii K
Dec 20, 2016, 8:08:00 AM12/20/16
to H2 Database
Hi
I see a lot of thread blocked at
java.lang.Thread.State: BLOCKED (on object monitor)
at org.h2.store.fs.FileBase.read(FileBase.java:39)
- waiting to lock <0x00000006f1dfcdd0> (a org.h2.store.fs.FileSplit)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:421)
at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
at org.h2.mvstore.Page.read(Page.java:190)
at org.h2.mvstore.MVStore.readPage(MVStore.java:1952)
at org.h2.mvstore.MVMap.readPage(MVMap.java:800)
at org.h2.mvstore.Page.getChildPage(Page.java:217)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:489)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:490)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:490)
at org.h2.mvstore.MVMap.get(MVMap.java:471)
at org.h2.mvstore.db.TransactionStore$TransactionMap.getValue(TransactionStore.java:1036)
- locked <0x00000006f1dd0608> (a org.h2.mvstore.MVMap)
at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:1013)
at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:981)
at org.h2.mvstore.db.MVPrimaryIndex.getRow(MVPrimaryIndex.java:215)
at org.h2.mvstore.db.MVTable.getRow(MVTable.java:463)
at org.h2.mvstore.db.MVSecondaryIndex$MVStoreCursor.get(MVSecondaryIndex.java:489)
at org.h2.index.IndexCursor.get(IndexCursor.java:288)
at org.h2.table.TableFilter.get(TableFilter.java:594)
at org.h2.command.dml.Update.update(Update.java:109)
Why the read() method is synchronized?
May be it would be better to use java.util.concurrent.locks.ReadWriteLock?
Noel Grandin
Dec 20, 2016, 8:15:44 AM12/20/16
to h2-da...@googlegroups.com
Not without changing the structure considerably - FileBase inherits from java.nio.channels.FileChannel, which has a
position field, and multiple things changing that at the same time would cause problems.
We'd probably need to switch to using Java's asynchronous IO API to avoid that, which is a fairly large change.
You're welcome to submit patches, of course :-)
position field, and multiple things changing that at the same time would cause problems.
We'd probably need to switch to using Java's asynchronous IO API to avoid that, which is a fairly large change.
You're welcome to submit patches, of course :-)
Noel Grandin
Dec 21, 2016, 5:55:32 AM12/21/16
to h2-da...@googlegroups.com
It appears I was wrong about this, we already implement reading/writing without synchronisation for all of our other
file implementations, I just missed that you are using the "split:" stuff.
I have pushed a fix for this.
file implementations, I just missed that you are using the "split:" stuff.
I have pushed a fix for this.
Anatolii K
Dec 21, 2016, 6:01:40 AM12/21/16
to H2 Database
Thank you for quick help!
Anatolii K
Dec 21, 2016, 9:40:43 AM12/21/16
to H2 Database
After applying this patch I'm getting following error with nioMemLZF (url= jdbc:h2:split:30:nioMemLZF:db):
org.h2.jdbc.JdbcSQLException: General error: "java.nio.BufferUnderflowException"; SQL statement:
UPDATE account set balance = ? where id = ? [50000-193]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.message.DbException.convert(DbException.java:295)
at org.h2.command.Command.executeUpdate(Command.java:266)
at org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:201)
at org.radixware.imdb_test.Arte.updateAcct(Arte.java:137)
at org.radixware.imdb_test.Arte.call(Arte.java:68)
at org.radixware.imdb_test.Arte.call(Arte.java:11)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:500)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
at org.h2.compress.CompressLZF.expand(CompressLZF.java:427)
at org.h2.store.fs.FileNioMemData.expand(FilePathNioMem.java:555)
at org.h2.store.fs.FileNioMemData.readWrite(FilePathNioMem.java:659)
at org.h2.store.fs.FileNioMem.read(FilePathNioMem.java:332)
at org.h2.store.fs.FileSplit.read(FilePathSplit.java:296)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:421)
at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
at org.h2.mvstore.Page.read(Page.java:190)
at org.h2.mvstore.MVStore.readPage(MVStore.java:1952)
at org.h2.mvstore.MVMap.readPage(MVMap.java:738)
at org.h2.mvstore.Page.getChildPage(Page.java:217)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:470)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:471)
at org.h2.mvstore.MVMap.get(MVMap.java:452)
Noel Grandin
Dec 21, 2016, 9:43:13 AM12/21/16
to h2-da...@googlegroups.com
I'm sorry, but that's just a silly combination to be using.
You should only using be "split:" for FAT filesystems, and you should certainly not be using it in combination with
"nioMemLZF:"
You should only using be "split:" for FAT filesystems, and you should certainly not be using it in combination with
"nioMemLZF:"
Anatolii K
Dec 21, 2016, 9:49:00 AM12/21/16
to H2 Database
I have very big in-memory database ~ 1TB. Without split H2 crashes
Noel Grandin
Dec 21, 2016, 9:50:31 AM12/21/16
to h2-da...@googlegroups.com
then let's fix that problem, but I'm not really keen on trying to fix a combination crash like that.
Anatolii K
Dec 21, 2016, 9:53:55 AM12/21/16
to H2 Database
Without split it also failed (url jdbc:h2:nioMemLZF:db):
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:500)
Noel Grandin
Dec 21, 2016, 9:57:41 AM12/21/16
to h2-da...@googlegroups.com
Can we get a full stack trace for that failure.
Thomas Mueller Graf
Dec 21, 2016, 10:41:44 AM12/21/16
to H2 Google Group
Hi,
I guess that bug would be on my plate... I recently made changes there (trying to improve concurrency). I'm sorry about that. Having a test case would be great!
Regards,
Thomas
On Wed, Dec 21, 2016 at 3:56 PM, Noel Grandin <noelg...@gmail.com> wrote:
Can we get a full stack trace for that failure.
--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscribe@googlegroups.com.
To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at https://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.
Anatolii K
Dec 21, 2016, 12:06:41 PM12/21/16
to H2 Database
org.h2.jdbc.JdbcSQLException: General error: "java.nio.BufferUnderflowException"; SQL statement:
UPDATE account set balance = ? where id = ? [50000-193]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.message.DbException.convert(DbException.java:295)
at org.h2.command.Command.executeUpdate(Command.java:266)
at org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:201)
at org.radixware.imdb_test.Arte.updateAcct(Arte.java:137)
at org.radixware.imdb_test.Arte.call(Arte.java:68)
at org.radixware.imdb_test.Arte.call(Arte.java:11)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:500)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
at org.h2.compress.CompressLZF.expand(CompressLZF.java:427)
at org.h2.store.fs.FileNioMemData.expand(FilePathNioMem.java:555)
at org.h2.store.fs.FileNioMemData.readWrite(FilePathNioMem.java:659)
at org.h2.store.fs.FileNioMem.read(FilePathNioMem.java:332)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:421)
at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
at org.h2.mvstore.Page.read(Page.java:190)
at org.h2.mvstore.MVStore.readPage(MVStore.java:1959)
at org.h2.mvstore.MVMap.readPage(MVMap.java:800)
at org.h2.mvstore.Page.getChildPage(Page.java:217)
at org.h2.mvstore.Cursor.min(Cursor.java:129)
at org.h2.mvstore.Cursor.hasNext(Cursor.java:36)
at org.h2.mvstore.db.TransactionStore$TransactionMap$1.fetchNext(TransactionStore.java:1221)
at org.h2.mvstore.db.TransactionStore$TransactionMap$1.<init>(TransactionStore.java:1217)
at org.h2.mvstore.db.TransactionStore$TransactionMap.keyIterator(TransactionStore.java:1212)
at org.h2.mvstore.db.TransactionStore$TransactionMap.keyIterator(TransactionStore.java:1200)
at org.h2.mvstore.db.MVSecondaryIndex.find(MVSecondaryIndex.java:309)
at org.h2.mvstore.db.MVSecondaryIndex.find(MVSecondaryIndex.java:254)
at org.h2.index.BaseIndex.find(BaseIndex.java:128)
at org.h2.index.IndexCursor.find(IndexCursor.java:169)
at org.h2.table.TableFilter.next(TableFilter.java:467)
at org.h2.command.dml.Update.update(Update.java:102)
at org.h2.command.CommandContainer.update(CommandContainer.java:98)
at org.h2.command.Command.executeUpdate(Command.java:258)
... 8 more
Anatolii K
Dec 21, 2016, 12:08:58 PM12/21/16
to H2 Database
The bug is reproduced only in multi-thread application
On Wednesday, December 21, 2016 at 8:41:44 PM UTC+5, Thomas Mueller Graf wrote:
Hi,I guess that bug would be on my plate... I recently made changes there (trying to improve concurrency). I'm sorry about that. Having a test case would be great!Regards,Thomas
On Wed, Dec 21, 2016 at 3:56 PM, Noel Grandin <noelg...@gmail.com> wrote:
Can we get a full stack trace for that failure.
--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
Anatolii K
Dec 21, 2016, 12:24:51 PM12/21/16
to H2 Database
Test attached
On Wednesday, December 21, 2016 at 8:41:44 PM UTC+5, Thomas Mueller Graf wrote:
Hi,I guess that bug would be on my plate... I recently made changes there (trying to improve concurrency). I'm sorry about that. Having a test case would be great!Regards,Thomas
On Wed, Dec 21, 2016 at 3:56 PM, Noel Grandin <noelg...@gmail.com> wrote:
Can we get a full stack trace for that failure.
--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
Noel Grandin
Dec 22, 2016, 4:13:57 AM12/22/16
to h2-da...@googlegroups.com
On 2016/12/21 7:24 PM, Anatolii K wrote:
> Test attached
>
Are you definitely testing with the latest HEAD? Because I ran your test for 10 minutes without problems.
On 64-bit JDK 1.8.0_111
Thomas Mueller Graf
Dec 22, 2016, 4:36:35 AM12/22/16
to H2 Google Group
Hi,
I didn't see a failure either.
Regards,
Thomas
--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscribe@googlegroups.com.
Anatolii K
Dec 22, 2016, 8:06:58 AM12/22/16
to H2 Database
Please try attached test. It fails after 10-60 seconds run
org.h2.jdbc.JdbcSQLException: General error: "java.lang.IllegalStateException: File corrupted in chunk 28, expected page length 4..32, got -1621686845 [1.4.193/6]"; SQL statement:
update account set balance = ? where id = ? [50000-193]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.message.DbException.convert(DbException.java:295)
at org.h2.command.Command.executeUpdate(Command.java:266)
at org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:201)
at Bug_H2_4$Arte.call(Bug_H2_4.java:99)
at Bug_H2_4$Arte.call(Bug_H2_4.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: File corrupted in chunk 28, expected page length 4..32, got -1621686845 [1.4.193/6]
at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:766)
at org.h2.mvstore.Page.read(Page.java:649)
at org.h2.mvstore.Page.read(Page.java:195)
at org.h2.mvstore.MVStore.readPage(MVStore.java:1952)
at org.h2.mvstore.MVMap.readPage(MVMap.java:738)
at org.h2.mvstore.Page.getChildPage(Page.java:217)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:470)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:471)
at org.h2.mvstore.MVMap.binarySearch(MVMap.java:471)
at org.h2.mvstore.MVMap.get(MVMap.java:452)
at org.h2.mvstore.db.TransactionStore$TransactionMap.getValue(TransactionStore.java:1203)
at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:1180)
at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:1148)
at org.h2.mvstore.db.MVPrimaryIndex.getRow(MVPrimaryIndex.java:215)
at org.h2.mvstore.db.MVTable.getRow(MVTable.java:463)
at org.h2.mvstore.db.MVSecondaryIndex$MVStoreCursor.get(MVSecondaryIndex.java:489)
at org.h2.index.IndexCursor.get(IndexCursor.java:288)
at org.h2.table.TableFilter.get(TableFilter.java:594)
at org.h2.command.dml.Update.update(Update.java:109)
at org.h2.command.CommandContainer.update(CommandContainer.java:98)
On Thursday, December 22, 2016 at 2:36:35 PM UTC+5, Thomas Mueller Graf wrote:
Hi,I didn't see a failure either.Regards,Thomas
On Thu, Dec 22, 2016 at 10:13 AM, Noel Grandin <noelg...@gmail.com> wrote:
On 2016/12/21 7:24 PM, Anatolii K wrote:
Test attached
Are you definitely testing with the latest HEAD? Because I ran your test for 10 minutes without problems.
On 64-bit JDK 1.8.0_111
--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
Anatolii K
Jan 2, 2017, 7:03:43 AM1/2/17
to H2 Database
I created separate topic for this bug https://groups.google.com/forum/#!topic/h2-database/rhSoRhzeZzk
Reply all
Reply to author
Forward
0 new messages