Global visibility with SFENCE.

[Naresh Kumar]

Hi,

Does the SFENCE instruction ensure visibility of the stores globally, or only of the stores on the specific CPU/socket that it is executed on? Put differently, if pmem_flush() and pmem_drain() end up running on different CPUs/sockets due to a context switch in between them (and due to a lack of CPU affinity of the process), would the latter ensure global visibility of the former?

Thanks,

Naresh.

[Andy Rudoff]

Hi Naresh,

It seems like you are asking two questions.  The first is about stores, and when they become globally-visible.  The second is about flush instructions, like CLWB, and when they are known to be completed.  Let's focus on the second question.

For the CLWB case, the Intel Software Developer's Manual says this:

Executions of the CLWB instruction are ordered with respect to fence instructions and to locked read-modify-write instructions

which means that grabbing any sort of lock will essentially do the same thing as an SFENCE, ensuring the CLWBs before it are complete.  In your example, if some CLWB instructions happened on one CPU and then the process was rescheduled to another CPU, the locks used by the first CPU as part of the context switch would order the CLWBs like an SFENCE.  Then, when the process began running on the new CPU, the SFENCE would apply to the CLWBs that were executed on that CPU (if any).  Of course, if someone invented an OS that could reschedule a process without grabbing any locks, that OS would be responsible for issuing a fence before the context switch to handle cases like this.  (In reality, I think there are lots of things that happen during a context switch that end up ordering operations like this.)

For the general case on how SFENCE is used, I found a great blog entry about SFENCE a while back with an impressive amount of detail and research behind it:

https://hadibrais.wordpress.com/2019/02/26/the-significance-of-the-x86-sfence-instruction/

-andy