Database corrupt after computer power failure.

[Mikael Nordenberg]

Hi,

We are using H2 version 1.4.185 with mv-store.

After an unclean shutdown the database file was corrupt. The database is attached.

Opening the database results in http://pastebin.com/PXceifPx

The database is used from a single thread.

Hope this gives anything...

/Mikael

[Mikael Nordenberg]

Same corruption happened again today on a clients computer when power disappeared.

Will try to reproduce tomorrow in our lab.

/Mikael

[Thomas Mueller]

Hi,

Thanks a lot! I have analyzed the database file, and I think I know what the problem is. It looks like the disk (or operating system) re-ordered write operations, so that changes later in time (and later in the file) were written, but one earlier change (both in time and in the file) was not written.

(Detail for later reference: various entries (in chunks 'aad4' to 'aadb', from 16:05:10.378 to 16:05:13.916) think that chunk 'aad3' (later than 16:05:09.369, earlier than 16:05:10.378) is in block 2, but block 2 actually contains the earlier chunk 'aa7b' from 16:04:24.742).

The MVStore should detect this and automatically discard chunks that were written later than 16:05:10.378. It does not currently do that, this is a problem. I will implement this for the next release.

I think what also can happen is write re-ordering causes a file to be truncated too early. This would explain a different corruption problem. The fix for that would need to be different however (truncation would need to wait for 45 seconds).

Regards,

Thomas

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.
To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

[Mikael Nordenberg]

Thank you for your response and time to investigate this.

That is very interesting!

I attach another case when this happened, in case it can be used to verify your assumption.

I have not been able to reproduce this on a test machine yet.

But it has happened twice in just a matter of days on one client machine, so it may be reproducible on that PC.

(The affected PC runs Windows 7, and uses a Samsung 850 EVO SSD drive if that matters.)

/Mikael

Den fredag 8 maj 2015 kl. 18:12:07 UTC+2 skrev Thomas Mueller:

Hi,

Thanks a lot! I have analyzed the database file, and I think I know what the problem is. It looks like the disk (or operating system) re-ordered write operations, so that changes later in time (and later in the file) were written, but one earlier change (both in time and in the file) was not written.

(Detail for later reference: various entries (in chunks 'aad4' to 'aadb', from 16:05:10.378 to 16:05:13.916) think that chunk 'aad3' (later than 16:05:09.369, earlier than 16:05:10.378) is in block 2, but block 2 actually contains the earlier chunk 'aa7b' from 16:04:24.742).

The MVStore should detect this and automatically discard chunks that were written later than 16:05:10.378. It does not currently do that, this is a problem. I will implement this for the next release.

I think what also can happen is write re-ordering causes a file to be truncated too early. This would explain a different corruption problem. The fix for that would need to be different however (truncation would need to wait for 45 seconds).

Regards,

Thomas

On Thursday, May 7, 2015, Mikael Nordenberg <mik...@ikanos.se> wrote:

Same corruption happened again today on a clients computer when power disappeared.

Will try to reproduce tomorrow in our lab.

/Mikael

Den torsdag 7 maj 2015 kl. 15:42:48 UTC+2 skrev Mikael Nordenberg:

Hi,

We are using H2 version 1.4.185 with mv-store.

After an unclean shutdown the database file was corrupt. The database is attached.

Opening the database results in http://pastebin.com/PXceifPx

The database is used from a single thread.

Hope this gives anything...

/Mikael

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscribe@googlegroups.com.

[Thomas Mueller]

Hi,

Until this problem is fixed, it might make sense to use the PageStore storage engine, by appending ";mv_store=false" to the database URL.

Regards,

Thomas

To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.

To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "H2 Database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to h2-database...@googlegroups.com.

[Thomas Mueller]

Hi,

I forgot to write: this should be fixed now (actually since quite a long time).

Regards,

Thomas

[Alexander Lauper]

Hi Thomas

I think I have the same problem with 1.4.190 using MVStore.

Stacktrace:

http://pastebin.com/jqZr0iXf

Corrupt DB attached.

Regards,

Alex

[Gautham Sampath]

Hi,

We are currently using h2 version: 1.3.174 in our customer environment. We are facing the same problem as Mikael Nordenberg. Is this problem resolved in the latest release version of H2 DB? Will H2 DB upgrade from PageStore to MVstore will solve this problem?
Even in our case below exception was thrown due the power shutdown:

2018-04-27 06:03:38,595 [ERROR] [        ] UpdateThread hibernate.util.JDBCExceptionReporter - File corrupted while reading record: "[16227] stream data key:16326 pos:11 remaining:0". Possible solution: use the recovery tool; SQL statement:
.
.
2018-04-27 06:03:38,595 [ERROR] [        ] UpdateThread dao.jpa.Impl - Error in Impl:getAll()
javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not execute query
    at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1214)
    at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1147)
    at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:255)
    .
    .

We connect with the db through this URL: jdbc:h2:db/Axxx;AUTO_SERVER=TRUE

Please let me your views on the same. Thanks in advance.

Regards,
Gautham

To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscribe@googlegroups.com.

To post to this group, send email to h2-da...@googlegroups.com.
Visit this group at http://groups.google.com/group/h2-database.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscribe@googlegroups.com.

[Mikael Nordenberg]

No offence Thomas, but after reading the source code for H2 when we investigated our problems, we decided to use a different database. To us, it seemed an overwhelming task to get that code in order to make it make it stable enough for our needs.

We switched to Postgres, and all our problems went away.

/Mikael

--
You received this message because you are subscribed to a topic in the Google Groups "H2 Database" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/h2-database/ajy6DUWfTbU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to h2-database+unsubscribe@googlegroups.com.

To post to this group, send email to h2-da...@googlegroups.com.

Visit this group at https://groups.google.com/group/h2-database.

[Noel Grandin]

On 2018/05/02 10:48 AM, Mikael Nordenberg wrote:
> No offence Thomas, but after reading the source code for H2 when we investigated our problems, we decided to use a
> different database. To us, it seemed an overwhelming task to get that code in order to make it make it stable enough for
> our needs.
> We switched to Postgres, and all our problems went away.
>

Did you identifier any problems in particular?

[Mikael Nordenberg]

We were mainly looking at the code quality / neatness. 

But if I remember correctly we identified places where things would not clean up correctly if an exception were thrown. It was a few years back now though, so I don't remember exactly what we discovered.

We did however rescue almost all our data. It turned out it was only one or two records (of millions) which were broken when the driver crashed and refused to read the database.

The recovery tool was also useless for us, since it also crashed (or refused to read the file).

If I remember correctly we selected all records "surrounding" the faulty ones, and recovered the data that way.

/Mikael