C++ paper by Tomasz Kapela

[Andy Rudoff]

Hello pmem-interested folks,

Just wanted to draw attention to a new paper written by Tomasz Kapela, a member of the PMDK team, about his experiences implementing our C++ support.  The paper gives some nice background on why our C++ support works the way it works, and also discusses some interesting areas that are still under investigation.  Comments/questions back to this list are welcome!

https://software.intel.com/en-us/cpp-extensions-for-persistent-memory-programming

-andy

[Niall Douglas]

Just wanted to draw attention to a new paper written by Tomasz Kapela, a member of the PMDK team, about his experiences implementing our C++ support.  The paper gives some nice background on why our C++ support works the way it works, and also discusses some interesting areas that are still under investigation.  Comments/questions back to this list are welcome!

https://software.intel.com/en-us/cpp-extensions-for-persistent-memory-programming

This was useful, thanks.

The Rapperswill meeting in June will be the beginning of the big push for standardisation of persistent memory support. Out of the areas described in the paper above, the rough plan of division (currently unapproved) is expected to be:

1. SG1 Concurrency is to take on adjusting the C++ memory model for RDMA and persistent memory. I doubt that this would refer to object lifetimes personally, rather coherency ordering.
2. SG5 Transactional Memory is to take on pmem i/o transactions as part of the general C++ memory transactions support.
3. SG14 Low Latency is to take on making C++ exception throws deterministic using a new value-based rather than type-based EH implementation, such that aborted transactions become deterministic. This is currently expected to be coordinated with WG14, the C programming language, so C code will be able to catch and throw these new C++ exceptions!
4. The new study group I am proposing in June (paper P1026 A call for a Data Persistence (iostream v2) study group) would take on persistent allocation, file system, kernel support, object serialisation and deserialisation, persistent algorithms, and all the other stuff not covered by the other SGs.

All of the above is unapproved by WG21, and is my current understanding from private conversations with the WG21 leadership. We will know much more after the Rapperswil meeting.

Niall

[Piotr Balcer]

That's really great Niall. Let us know if we can be of any help with the committees.
I know I wasn't very responsive as of late, but I will try to remedy that in the future.

Persistent Memory /w RDMA is actively being worked on by the industry, and there are new protocol extensions being proposed that address the remote durability/ordering guarantees. It would be great to see these semantics standardized in C++ as well. But I realize we need the basic building blocks first ;)

Would transactional memory for persistence be a separate proposal to the current TM work? I believe we need to recognize that while there's a significant amount of overlap, isolation (what the current proposal addresses) and durability are two separate things that require different approaches. We were thinking about writing our own proposal and presenting it to the SG5 in the future, but I don't want to step on anyone toes here, especially since we are the newcomers here.

Thanks!

Piotr

[Niall Douglas]

Would transactional memory for persistence be a separate proposal to the current TM work?

In the pure sense, probably not. Current TM is based on the C/C++ memory model, which currently only specifies visibility of ordering of changes between threads of execution on a cache coherent system. So, C++ TM currently is only about effectively doing scatter-gather atomics where more than one location can be updated in a single shot, without tearing the updates, with respect to other threads only.

Nothing is said about any other form of visibility ordering because the C/C++ memory model does not specify that other forms are possible. So if you update the memory model, that goes hand in hand with the transactional support. They are two sides of the same coin, as it were.

 

I believe we need to recognize that while there's a significant amount of overlap, isolation (what the current proposal addresses) and durability are two separate things that require different approaches. We were thinking about writing our own proposal and presenting it to the SG5 in the future, but I don't want to step on anyone toes here, especially since we are the newcomers here.

I'd get onto the SG1 mailing list and see what people are thinking in terms of updating the memory model for persistence. I've already explained what is needed to them, and it hurt everybody's heads, and I think they decided to stop hurting their heads.

But I'd start as soon as you can if I were you. Getting a wording acceptable to WG21 is a multi-year effort. I would expect at least five years of effort would be needed, so the C++ 25 standard. So sooner to start, the better.

Let me know if I can be more specific in advice.

Niall