Regarding atomicity of 8k memcpy

[Himanshu Chatterjee]

Hi, Can someone please answer this question :-

In writer.c example, i see a code snippet like this

    TX_BEGIN(pop) {

        pmemobj_tx_add_range(root, 0, sizeof(struct my_root));

        memcpy(rootp->buf, buf, strlen(buf));

    } TX_END

My question is, if i attempt to write a 8k block of data to PMEM, then, is it possible, that i can have a torn write (example, power failure while write in progress) into PMEM, if i use PMEM transactions, as shown in the code snippet ? 

If i get atomic write (either redo the work or undo to previous state) happens because of transaction, do we know what is the cost associated with the transaction management ? Do we have any numbers ?

Followup question is, if transactions as per above can handle atomic writes then why we need sector atomicity by BTT implementation.

Thanks a lot

Himanshu

[Andy Rudoff]

Hi Himanshu,

Let me answer the BTT question first:  the kernel driver uses the BTT to implement a block device interface on top of persistent memory.  The BTT algorithm prevents a torn write of a single sector (so if the sector size is configured to 4k, an 8k write can still be torn by a power failure, but each of the two 4k blocks being written could not be torn).

So that's for block usages, like accessing a block device directly or using a traditional file system on top of a block device.

The code that you see in writer.c is for direct access to pmem from user space.  So there would be no BTT involved on Linux and access would be provided via the DAX feature (the "dax" mount option with ext4, for example).  In that case, when an application mmaps() a file, it gets direct load/store access.  No kernel code is called to copy data to the persistent memory, memcpy() is doing it directly. So there's no BTT (and no opportunity for something like BTT) to add atomicity.

So that's why we have the library.  The TX_BEGIN/TX_END block you quoted uses an undo log to make the stuff inside the block happen atomically with respect to power failure (or other interruption like a program crash).  The next time the library is used with that file, any partial changes are rolled back before the call to pmemobj_open() returns.  If you don't use the transactions provided by the library, a memcpy() to persistent memory can certainly get torn by an interruption.

What's the overhead?  It will depend on the performance of the persistent memory product you're using, but it is at least the time to make an extra copy of the data being changed into the undo log, plus the time to update some undo log data structures, plus the time to flush those changes from the CPU caches.  So the overhead is non-trivial, but it is the cost of providing transactions.  The good news is that pmem is byte-addressable, so often times the overhead of the transactions is significantly less than that of a transaction on a traditional storage device since that requires full blocks to be transferred via the driver, no matter how small the transaction.

Hope that helps.

-andy