Questions about pmemobj and pmemlog

[Haoyu Huang]

Hi, 

I'm trying to use pmemlog to build a database logging subsystem. 

1) Is pmem_append atomic?  

I read the source code on the pmemlog_append function in the pmemlog library. It seems to me that the append function is not atomic which means that higher level application, e.g. database, still needs to handle scenarios like partial writes. 

I just want to confirm that if this is the case or not. If not, can you help point me to the code where atomicity is provided?

2) does pmem_log use any redo/undo logging?

3) If I understand correctly, pmemobj uses redo/undo logging to provide transaction, is there any document that discusses details of pmemobj's logging subsystem?

Thanks!

[Piotr Balcer]

Hi,
Answer inline.

W dniu poniedziałek, 30 października 2017 06:04:41 UTC+1 użytkownik Haoyu Huang napisał:

Hi, 

I'm trying to use pmemlog to build a database logging subsystem. 

1) Is pmem_append atomic?  

I read the source code on the pmemlog_append function in the pmemlog library. It seems to me that the append function is not atomic which means that higher level application, e.g. database, still needs to handle scenarios like partial writes. 

I just want to confirm that if this is the case or not. If not, can you help point me to the code where atomicity is provided?

This function is atomic in terms of both visibility and fail-safety. Notice that the metadata (`write_offset` field) is written after the data.

 

2) does pmem_log use any redo/undo logging?

No, it's a simple append-only log.

 

3) If I understand correctly, pmemobj uses redo/undo logging to provide transaction, is there any document that discusses details of pmemobj's logging subsystem?

Yes, you are right - pmemobj uses a combination of undo and redo logging to ensure fail-safety.

Unfortunately, there's currently no publicly available detailed documentation of internal details of libpmemobj.

 

Thanks!

Piotr

[Haoyu Huang]

Thanks for your answer! 

I noticed that metadata write_offset is written after data. 

The write_offset is written using pmem_persist and the libpmem's man page said pmem_persist is not atomic nor transactional. 

Can you help me clarify this?

[Piotr Balcer]

Sure.

All of our high level pmem libraries (libpmemblk, libpmemlog and libpmemobj) are built on a guarantee
that a single 8 byte store is fail-safe atomic, meaning that if a power failure occurs at any time during that store,
we will either get the new value or the old one.
We combine this guarantee with careful flushing (clflush, clflushopt, clwb) and ordering (sfence) of data stores to
create more complex algorithms.

In the case of libpmemlog, things are fairly straightforward.

We have a contiguous blob of memory. Our task is very simple: atomically append N bytes of data.
First, a little bit of memory is carved out of that block for metadata. The only field we really care about is
the position that marks the end of the region we know contains valid data. Initially, it's 0.
The write offset size is 8 bytes, because that's that's more than plenty to cover the entire address space.

Next, when the append is called, we first write the data we received to the pool_addr+offset location,
and once that memcpy() is completed, and we are sure that the data is out of CPU caches
(that's what pmem_memcpy_persist ensures), we can atomically update the `write_offset`.

If the application crashes in between any of those operations, you will either have the new data or not,
there will never be case in which the pmemlog_walk() function gives you an incomplete part of the log.

Hope this helps,
Piotr

[Haoyu Huang]

Thanks. It totally makes sense. 

Thanks a lot for your quick responses!