DB Corrupt
61 views
Skip to first unread message
Behroz Sikander
Apr 24, 2018, 6:22:40 AM4/24/18
to H2 Database
We are using h2, 1.3.176 and updating the version to the latest one is a bit
difficult now but if there is no option then we can consider it.
Our connection string looks like
jdbc:h2:tcp://<address>//my/path/to/db;MAX_LOG_SIZE=300;AUTO_SERVER=TRUE
I have an H2 database with 5 tables, 1 of the table is storing BLOBs. Last
day, out of the blue the DB stopped responding. We see the following
exception.
General error: "java.lang.RuntimeException: page[2837974] data leaf table:10 T10 entries:1 parent:2837272 overflow:2837451 keys:[2234666] offsets:[21] parent 2837272 expected 2816832" [50000-176]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.message.DbException.convert(DbException.java:294)
at org.h2.engine.Database.openDatabase(Database.java:291)
at org.h2.engine.Database.<init>(Database.java:254)
at org.h2.engine.Engine.openSession(Engine.java:57)
at org.h2.engine.Engine.openSession(Engine.java:164)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:142)
at org.h2.engine.Engine.createSession(Engine.java:125)
at org.h2.server.TcpServerThread.run(TcpServerThread.java:150)
at java.lang.Thread.run(Thread.java:808)
Caused by: java.lang.RuntimeException: page[2837974] data leaf table:10 T10 entries:1 parent:2837272 overflow:2837451 keys:[2234666] offsets:[21] parent 2837272 expected 2816832
at org.h2.message.DbException.throwInternalError(DbException.java:241)
at org.h2.index.PageDataIndex.getPage(PageDataIndex.java:247)
at org.h2.index.PageDataNode.getLastKey(PageDataNode.java:215)
at org.h2.index.PageDataNode.getLastKey(PageDataNode.java:215)
at org.h2.index.PageDataNode.getLastKey(PageDataNode.java:215)
at org.h2.index.PageDataIndex.<init>(PageDataIndex.java:88)
at org.h2.table.RegularTable.<init>(RegularTable.java:84)
at org.h2.store.PageStore.addMeta(PageStore.java:1693)
at org.h2.store.PageStore.readMetaData(PageStore.java:1624)
at org.h2.store.PageStore.recover(PageStore.java:1406)
at org.h2.store.PageStore.openExisting(PageStore.java:368)
at org.h2.store.PageStore.open(PageStore.java:289)
at org.h2.engine.Database.getPageStore(Database.java:2366)
at org.h2.engine.Database.open(Database.java:657)
at org.h2.engine.Database.openDatabase(Database.java:260)
... 7 more
at org.h2.engine.SessionRemote.done(SessionRemote.java:606)
at org.h2.engine.SessionRemote.initTransfer(SessionRemote.java:129)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:430)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:311)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:107)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:91)
at org.h2.Driver.connect(Driver.java:74)
at slick.jdbc.DriverDataSource.getConnection(DriverDataSource.scala:98)
at slick.jdbc.DataSourceJdbcDataSource.createConnection(JdbcDataSource.scala:64)
at slick.jdbc.JdbcBackend$BaseSession.conn$lzycompute(JdbcBackend.scala:415)
at slick.jdbc.JdbcBackend$BaseSession.conn(JdbcBackend.scala:414)
at slick.jdbc.JdbcBackend$BaseSession.startInTransaction(JdbcBackend.scala:437)
at slick.driver.JdbcActionComponent$StartTransaction$.run(JdbcActionComponent.scala:41)
at slick.driver.JdbcActionComponent$StartTransaction$.run(JdbcActionComponent.scala:38)
at slick.backend.DatabaseComponent$DatabaseDef$$anon$2.liftedTree1$1(DatabaseComponent.scala:237)
at slick.backend.DatabaseComponent$DatabaseDef$$anon$2.run(DatabaseComponent.scala:237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:808)
Here is the transaction log statistics outputted by the recover tool
---- Transaction log ----
---- Statistics ----
-- page count: 2877737, free: 17860
-- page data bytes: head 7550672, empty 17835363, rows 596255693 (98% full)
-- free 0%, 1825 page(s)
-- data leaf 10%, 303536 page(s)
-- data node 0%, 2694 page(s)
-- data overflow 88%, 2544303 page(s)
-- btree leaf 0%, 6618 page(s)
-- btree node 0%, 189 page(s)
-- free list 0%, 176 page(s)
-- stream trunk 0%, 41 page(s)
-- stream data 0%, 18352 page(s)
I tried to recover the database using the recover tool + runscript but it also ended up with an exception.
04-23 16:25:45 jdbc[3]: exception
org.h2.jdbc.JdbcSQLException: Column count does not match; SQL statement:
-- page 2161970: data overflow (last)
-- page 2161971: data overflow
-- page 2161972: data overflow
-- page 2161973: data overflow
-- page 2161974: data overflow
-- page 2161975: data overflow
-- page 2161976: data overflow
-- page 2161977: data overflow
-- page 2161978: data overflow
-- page 2161979: data leaf parent: 2164239 table: 10 entries: 1 columns: 3
-- next: 2161980
-- ERROR [2161979] child[0]: 2161980 parent: 2160633
-- ERROR: type: 1 storageId: 10 recordLength: 3 valueId: 3
.......
.......
-- dump: 01 a2 fe 00 20 f7 f9 0a 03 00 01 00 20 fd 3a 86 8b 4e 00 14 05 86 8b 4e 20 0c a0 9c 01 55 f8 83 5a e7 9f 36 90 0f 6a 1f 30 e8 00 5e c3 65 7a 6d c0 55 eb 65 bc a1 56 4d c1 31 c9 98 ce e2 04 56 60 31 e4 17 09 c5 25 14 09 c4 31 a4 51 42 85 a7 18 ef 16 58 ef 37 c1 fa 0d 81 af c2 a4 f6 6b 4c 8a bc a1 71 af 01 6e 43 db 2a b5 35 74 94 29 93 e0 4e ab c4 9f 22 82 a7 e9 d3 19 4a cf 12 db 41 4e e2
.......
.......
-- ERROR: exception java.lang.ArrayIndexOutOfBoundsException: while trying to copy from index 20029 of a type array with length 2048 storageId: 10 recordLength: 3 valueId: 2
.......
INSERT INTO O_10 VALUES(1272066, 0) [21002-176]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:178)
at org.h2.message.DbException.get(DbException.java:154)
at org.h2.message.DbException.get(DbException.java:143)
at org.h2.command.dml.Insert.prepare(Insert.java:266)
at org.h2.command.Parser.prepareCommand(Parser.java:248)
at org.h2.engine.Session.prepareLocal(Session.java:442)
at org.h2.engine.Session.prepareCommand(Session.java:384)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1188)
at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:172)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:160)
at org.h2.tools.RunScript.process(RunScript.java:261)
at org.h2.tools.RunScript.process(RunScript.java:191)
at org.h2.tools.RunScript.process(RunScript.java:329)
at org.h2.tools.RunScript.runTool(RunScript.java:142)
at org.h2.tools.RunScript.main(RunScript.java:69)
Could some please guide me what went wrong? how to avoid this in future and how to recover the database?
difficult now but if there is no option then we can consider it.
Our connection string looks like
jdbc:h2:tcp://<address>//my/path/to/db;MAX_LOG_SIZE=300;AUTO_SERVER=TRUE
I have an H2 database with 5 tables, 1 of the table is storing BLOBs. Last
day, out of the blue the DB stopped responding. We see the following
exception.
General error: "java.lang.RuntimeException: page[2837974] data leaf table:10 T10 entries:1 parent:2837272 overflow:2837451 keys:[2234666] offsets:[21] parent 2837272 expected 2816832" [50000-176]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.message.DbException.convert(DbException.java:294)
at org.h2.engine.Database.openDatabase(Database.java:291)
at org.h2.engine.Database.<init>(Database.java:254)
at org.h2.engine.Engine.openSession(Engine.java:57)
at org.h2.engine.Engine.openSession(Engine.java:164)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:142)
at org.h2.engine.Engine.createSession(Engine.java:125)
at org.h2.server.TcpServerThread.run(TcpServerThread.java:150)
at java.lang.Thread.run(Thread.java:808)
Caused by: java.lang.RuntimeException: page[2837974] data leaf table:10 T10 entries:1 parent:2837272 overflow:2837451 keys:[2234666] offsets:[21] parent 2837272 expected 2816832
at org.h2.message.DbException.throwInternalError(DbException.java:241)
at org.h2.index.PageDataIndex.getPage(PageDataIndex.java:247)
at org.h2.index.PageDataNode.getLastKey(PageDataNode.java:215)
at org.h2.index.PageDataNode.getLastKey(PageDataNode.java:215)
at org.h2.index.PageDataNode.getLastKey(PageDataNode.java:215)
at org.h2.index.PageDataIndex.<init>(PageDataIndex.java:88)
at org.h2.table.RegularTable.<init>(RegularTable.java:84)
at org.h2.store.PageStore.addMeta(PageStore.java:1693)
at org.h2.store.PageStore.readMetaData(PageStore.java:1624)
at org.h2.store.PageStore.recover(PageStore.java:1406)
at org.h2.store.PageStore.openExisting(PageStore.java:368)
at org.h2.store.PageStore.open(PageStore.java:289)
at org.h2.engine.Database.getPageStore(Database.java:2366)
at org.h2.engine.Database.open(Database.java:657)
at org.h2.engine.Database.openDatabase(Database.java:260)
... 7 more
at org.h2.engine.SessionRemote.done(SessionRemote.java:606)
at org.h2.engine.SessionRemote.initTransfer(SessionRemote.java:129)
at org.h2.engine.SessionRemote.connectServer(SessionRemote.java:430)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:311)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:107)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:91)
at org.h2.Driver.connect(Driver.java:74)
at slick.jdbc.DriverDataSource.getConnection(DriverDataSource.scala:98)
at slick.jdbc.DataSourceJdbcDataSource.createConnection(JdbcDataSource.scala:64)
at slick.jdbc.JdbcBackend$BaseSession.conn$lzycompute(JdbcBackend.scala:415)
at slick.jdbc.JdbcBackend$BaseSession.conn(JdbcBackend.scala:414)
at slick.jdbc.JdbcBackend$BaseSession.startInTransaction(JdbcBackend.scala:437)
at slick.driver.JdbcActionComponent$StartTransaction$.run(JdbcActionComponent.scala:41)
at slick.driver.JdbcActionComponent$StartTransaction$.run(JdbcActionComponent.scala:38)
at slick.backend.DatabaseComponent$DatabaseDef$$anon$2.liftedTree1$1(DatabaseComponent.scala:237)
at slick.backend.DatabaseComponent$DatabaseDef$$anon$2.run(DatabaseComponent.scala:237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:808)
Here is the transaction log statistics outputted by the recover tool
---- Transaction log ----
---- Statistics ----
-- page count: 2877737, free: 17860
-- page data bytes: head 7550672, empty 17835363, rows 596255693 (98% full)
-- free 0%, 1825 page(s)
-- data leaf 10%, 303536 page(s)
-- data node 0%, 2694 page(s)
-- data overflow 88%, 2544303 page(s)
-- btree leaf 0%, 6618 page(s)
-- btree node 0%, 189 page(s)
-- free list 0%, 176 page(s)
-- stream trunk 0%, 41 page(s)
-- stream data 0%, 18352 page(s)
I tried to recover the database using the recover tool + runscript but it also ended up with an exception.
04-23 16:25:45 jdbc[3]: exception
org.h2.jdbc.JdbcSQLException: Column count does not match; SQL statement:
-- page 2161970: data overflow (last)
-- page 2161971: data overflow
-- page 2161972: data overflow
-- page 2161973: data overflow
-- page 2161974: data overflow
-- page 2161975: data overflow
-- page 2161976: data overflow
-- page 2161977: data overflow
-- page 2161978: data overflow
-- page 2161979: data leaf parent: 2164239 table: 10 entries: 1 columns: 3
-- next: 2161980
-- ERROR [2161979] child[0]: 2161980 parent: 2160633
-- ERROR: type: 1 storageId: 10 recordLength: 3 valueId: 3
.......
.......
-- dump: 01 a2 fe 00 20 f7 f9 0a 03 00 01 00 20 fd 3a 86 8b 4e 00 14 05 86 8b 4e 20 0c a0 9c 01 55 f8 83 5a e7 9f 36 90 0f 6a 1f 30 e8 00 5e c3 65 7a 6d c0 55 eb 65 bc a1 56 4d c1 31 c9 98 ce e2 04 56 60 31 e4 17 09 c5 25 14 09 c4 31 a4 51 42 85 a7 18 ef 16 58 ef 37 c1 fa 0d 81 af c2 a4 f6 6b 4c 8a bc a1 71 af 01 6e 43 db 2a b5 35 74 94 29 93 e0 4e ab c4 9f 22 82 a7 e9 d3 19 4a cf 12 db 41 4e e2
.......
.......
-- ERROR: exception java.lang.ArrayIndexOutOfBoundsException: while trying to copy from index 20029 of a type array with length 2048 storageId: 10 recordLength: 3 valueId: 2
.......
INSERT INTO O_10 VALUES(1272066, 0) [21002-176]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:178)
at org.h2.message.DbException.get(DbException.java:154)
at org.h2.message.DbException.get(DbException.java:143)
at org.h2.command.dml.Insert.prepare(Insert.java:266)
at org.h2.command.Parser.prepareCommand(Parser.java:248)
at org.h2.engine.Session.prepareLocal(Session.java:442)
at org.h2.engine.Session.prepareCommand(Session.java:384)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1188)
at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:172)
at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:160)
at org.h2.tools.RunScript.process(RunScript.java:261)
at org.h2.tools.RunScript.process(RunScript.java:191)
at org.h2.tools.RunScript.process(RunScript.java:329)
at org.h2.tools.RunScript.runTool(RunScript.java:142)
at org.h2.tools.RunScript.main(RunScript.java:69)
Could some please guide me what went wrong? how to avoid this in future and how to recover the database?
Behroz Sikander
Apr 25, 2018, 7:23:27 AM4/25/18
to H2 Database
Did anybody get the chance to look at this problem? Would be nice to get some pointers. Any help would be much appreciated.
Noel Grandin
Apr 25, 2018, 7:43:10 AM4/25/18
to h2-da...@googlegroups.com, Behroz Sikander
There is not much to say, your DB is corrupt, and unfortunately too corrupt for the Recover tool to be able to do much.
Doing regular backups is the only real change you can make to protect against this.
Behroz Sikander
Apr 25, 2018, 7:54:56 AM4/25/18
to H2 Database
Ok.
What I can do to figure out why it got corrupted?
Any tool or technique that I can use on the .sql file generated by recover tool to understand?
Behroz Sikander
Apr 25, 2018, 7:56:53 AM4/25/18
to H2 Database
Also, is there any documentation that I can use understand the following output:
---- Transaction log ----
---- Statistics ----
-- page count: 2877737, free: 17860
-- page data bytes: head 7550672, empty 17835363, rows 596255693 (98% full)
-- free 0%, 1825 page(s)
-- data leaf 10%, 303536 page(s)
-- data node 0%, 2694 page(s)
-- data overflow 88%, 2544303 page(s)
-- btree leaf 0%, 6618 page(s)
-- btree node 0%, 189 page(s)
-- free list 0%, 176 page(s)
-- stream trunk 0%, 41 page(s)
-- stream data 0%, 18352 page(s)
---- Statistics ----
-- page count: 2877737, free: 17860
-- page data bytes: head 7550672, empty 17835363, rows 596255693 (98% full)
-- free 0%, 1825 page(s)
-- data leaf 10%, 303536 page(s)
-- data node 0%, 2694 page(s)
-- data overflow 88%, 2544303 page(s)
-- btree leaf 0%, 6618 page(s)
-- btree node 0%, 189 page(s)
-- free list 0%, 176 page(s)
-- stream trunk 0%, 41 page(s)
-- stream data 0%, 18352 page(s)
On Wednesday, 25 April 2018 13:43:10 UTC+2, Noel Grandin wrote:
Noel Grandin
Apr 25, 2018, 8:00:19 AM4/25/18
to h2-da...@googlegroups.com, Behroz Sikander
On 2018/04/25 1:56 PM, Behroz Sikander wrote:
> Also, is there any documentation that I can use understand the following output:
>
Other than reading the code, and the documentation like:
> Also, is there any documentation that I can use understand the following output:
>
http://h2database.com/html/architecture.html
no, I'm afraid not
> What I can do to figure out why it got corrupted?
If it is a bug, unless you can reproduce the failure mode, finding it is almost impossible.
Reply all
Reply to author
Forward
0 new messages