Failure Atomicity

78 views
Skip to first unread message

Nabeel Mohamed

unread,
Oct 27, 2021, 7:48:03 PM10/27/21
to pm...@googlegroups.com
Hi,

I'm modifying a software stack to be persistent memory aware and have the following query on failure atomicity:

The USENIX article (Vol. 42, 2017) from Andy Rudoff mentions - "On Intel, only an 8-byte store, aligned on an 8-byte boundary, is guaranteed to be failure atomic. Anything larger than 8 bytes can be torn by power failure".

What about naturally aligned stores that are smaller than 8 bytes, for instance, a 4-byte store, aligned on a 4-byte boundary?

The Intel Software Developer's Manual (Volume 3, section 8.1.1) provides a list of memory operations that will be carried out atomically by various Intel processor families. What is the applicability of these rules for a Intel Optane DC PMM sitting on the memory bus?

Please clarify...

Thanks,
Nabeel

Andy Rudoff

unread,
Oct 27, 2021, 11:53:09 PM10/27/21
to pmem
Hi Nabeel,

Yes, stores to pmem that are smaller than 8 bytes, but do not cross an 8-byte boundary are also architecturally guaranteed to be power fail atomic.  There are lots of places that mention atomic operations in the SDM, but those refer to atomicity with respect to visibility, not persistence.  When systems with eADR are available, those two types of atomicity become the same thing, but the ADR systems that are available today have only the 8-byte power fail atomicity as the article you quoted says.

-andy

Nabeel Mohamed

unread,
Oct 28, 2021, 3:22:10 PM10/28/21
to pmem
Hi Andy,

Thanks very much for providing clarity on this! I assume that an unaligned load/store or a load/store crossing a cache line behaves the same way as it behaves today on a DRAM. Also, atomic operations on a pmem memory-map provide the same visibility guarantees as atomics on a DRAM.

I've another question on the behavior of O_DIRECT + O_SYNC and persistence -

When an application opens a file on a DAX mounted filesystem (xfs/ext4) with O_DIRECT + O_SYNC flags, can it safely assume file integrity completion guarantees?

In other words, on return from the write(2) system call on such a file (O_DIRECT + O_SYNC), is the written user data and its associated filesystem metadata persistent on the pmem device such that the application need not follow the write(2) with an fsync(2) to ensure persistence?

Thanks,
Nabeel
Reply all
Reply to author
Forward
0 new messages