Question about org.h2.store.PageStore's checkpoint() method

[Steve McLeod]

Hi Thomas,

When running TRUNCATE TABLE on a large table, it is time-consuming. I've done some profiling of the H2 source, and found that a step in PageStore.checkpoint() that consumes one-third of the time needed for TRUNCATE TABLE. In my specific case it uses 69 seconds out of seconds for the whole TRUNCATE execution.

This line is the one consuming the time:

                file.readFully(test, 0, 16); 

which is org.h2.store.PageStore.java: line 451 in the current SVN trunk.

It is part of a process of zeroing freed pages on disk. I'm wondering if zeroing is an important part of the workings of the PageStore. Or is it for security reasons? If so, is this step something I could potentially make optional by introducing a "INSECURE_FREE" database option? 

I'm running on Mac OS X Lion, using an SSD drive.

Regards,

Steve McLeod

[Noel Grandin]

I can't be sure, but it looks like it's zero-ing out the header of a
page, so I would assume that it's a little necessary.

you could try increasing the page size of your database, that might
speed up this operation.

> --
> You received this message because you are subscribed to the Google
> Groups "H2 Database" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/h2-database/-/KpuvExBPO2sJ.
> To post to this group, send email to h2-da...@googlegroups.com.
> To unsubscribe from this group, send email to
> h2-database...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/h2-database?hl=en.

[Steve McLeod]

Hi Noel,

Thanks for the suggestion to increase page size. Something I'll consider.

I took another look at code I asked about. It seems to me this is what happens:

1) load first 16 bytes of a data page from disk

2) if the first of those 16 loaded bytes is not 0, then zero the entire page, as follows: (comments are mine)

    // move to start of page i

    file.seek((long) i << pageSizeShift);

    // zero out entire contents of page i

    file.write(empty, 0, pageSize);

Experience has taught me that sections of code like this usually have a very good reason. Alas, I can't work out that reason yet.

Thomas, can you illuminate?

Regards

Steve

> h2-database+unsubscribe@googlegroups.com.

[Thomas Mueller]

Hi,

Yes, a larger page size should help.

The header of the freed pages is zeroed out so that in case of a crash, the free pages can be easily re-used.

It's unlikely the checkpoint can be easily improved I'm afraid. I'm working on a new storage engine, but it will take a while until this is available.

Regards,

Thomas

[Steve McLeod]

Ah, I see.

However, the code for checkpoint() seems to zero not just the header but the entire page:

   byte[] empty = new byte[pageSize];

   file.write(empty, 0, pageSize);

This is writing <pageSize> bytes, right?

Regards,

Steve

On Wednesday, 4 April 2012 20:06:57 UTC+2, Thomas Mueller wrote:

Hi,

Yes, a larger page size should help.

The header of the freed pages is zeroed out so that in case of a crash, the free pages can be easily re-used.

It's unlikely the checkpoint can be easily improved I'm afraid. I'm working on a new storage engine, but it will take a while until this is available.

Regards,

Thomas

On Tuesday, April 3, 2012, Noel Grandin wrote:

I can't be sure, but it looks like it's zero-ing out the header of a page, so I would assume that it's a little necessary.

you could try increasing the page size of your database, that might speed up this operation.


On 2012-04-02 17:15, Steve McLeod wrote:

Hi Thomas,

When running TRUNCATE TABLE on a large table, it is time-consuming. I've done some profiling of the H2 source, and found that a step in PageStore.checkpoint() that consumes one-third of the time needed for TRUNCATE TABLE. In my specific case it uses 69 seconds out of seconds for the whole TRUNCATE execution.

This line is the one consuming the time:

               file.readFully(test, 0, 16);

which is org.h2.store.PageStore.java: line 451 in the current SVN trunk.

It is part of a process of zeroing freed pages on disk. I'm wondering if zeroing is an important part of the workings of the PageStore. Or is it for security reasons? If so, is this step something I could potentially make optional by introducing a "INSECURE_FREE" database option?

I'm running on Mac OS X Lion, using an SSD drive.

Regards,

Steve McLeod



--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To view this discussion on the web visit https://groups.google.com/d/msg/h2-database/-/KpuvExBPO2sJ.
To post to this group, send email to h2-da...@googlegroups.com.

To unsubscribe from this group, send email to h2-database+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/h2-database?hl=en.

--
You received this message because you are subscribed to the Google Groups "H2 Database" group.
To post to this group, send email to h2-da...@googlegroups.com.

To unsubscribe from this group, send email to h2-database+unsubscribe@googlegroups.com.

[Noel Grandin]

I'm not sure, but I think you are correct, and you could change it to only overwrite the first 16 bytes.
It would have security concerns, because it wouldn't erase old data.
But more importantly, it probably wouldn't buy you much in terms of IO because the filesystem is probably writing in at least 2K chunks.
But it might be worth a try. If you have a filesystem formatted to 512 byte sectors, and you crank up the pagesize of your H2 database to 16K, you may well gain some speed.

Try it by making the change and running the unit tests - if there is a problem, the unit tests will catch it.

To view this discussion on the web visit https://groups.google.com/d/msg/h2-database/-/Z2ZMtLk5qWsJ.

To post to this group, send email to h2-da...@googlegroups.com.

To unsubscribe from this group, send email to h2-database...@googlegroups.com.