thread safety in non-transactional persistent atomic lists

[Clark Laughlin]

I am looking for information the thread safety of the non-transactional persistent atomic list functions.  The intro blog posts don't mention anything about it, yet I can see the macros to construct a list add a mutex to the structure.  Is there any additional documentation on this?

[Piotr Balcer]

W dniu wtorek, 13 września 2016 20:38:30 UTC+2 użytkownik Clark Laughlin napisał:

I am looking for information the thread safety of the non-transactional persistent atomic list functions.  The intro blog posts don't mention anything about it, yet I can see the macros to construct a list add a mutex to the structure.  Is there any additional documentation on this?

Hi,
In one of the blog posts I've mentioned that all of our functions (except for open and create, and we intend to change this too) are thread-safe. Macro's are a little bit different story because they usually consist of several operations which individually are thread safe but when put together it's a completely different story. To illustrate that, let's take a loot at the POBJ_LIST_INSERT_HEAD:

pmemobj_list_insert((pop),\
    TOID_OFFSETOF(POBJ_LIST_FIRST(head), field),\
    (head), OID_NULL,\
    POBJ_LIST_DEST_HEAD, (elm).oid) // << we extract oid from the macro subject, this CANNOT change in a different thread.

The lock inside the structure guarantees that this function will add the new object into the list correctly and the data structure itself will be correct, but the object we've extracted in the fourth line of this snippet might no longer be valid. I think this is pretty common sense.

Long story short, you can use those macros without additional locks but you have to keep your data separate.

Piotr

[Clark Laughlin]

So, since the macros are presented as part of the API, then it seems the persistent atomic list API as a whole is not thread safe, correct?

It's not clear from the documentation as to which functions / macros are safe and which aren't... which ones use the mutex and which don't.  One of my main concerns was the POBJ_LIST_FOREACH macro.  It does not appear to be thread safe, which seems problematic.  You can have another thread inserting or deleting items while enumerating through the list in another thread.

[Piotr Balcer]

Yes, the FOREACH macros are not thread safe - that would require a lock outside of the loop. On the other hand the insert/remove/move functions and macros are atomic in terms of visibility to other threads, you just need to protect the objects you operate on.
We've had an internal debate about this exact topic quite some time ago, I don't remember what was the conclusion but I understand how confusing the list API must be :)

You are right that we did a poor job documenting the thread safety of the API, we will remedy that ASAP. This information should be included in the man pages.

Piotrek

[Clark Laughlin]

Thank you!  What are your thoughts on providing an implementation of FOREACH that is safe without requiring an additional external lock?

- Clark

[Piotr Balcer]

Honestly, I'm wondering if we should just get rid of the lock from the list altogether and let the programmer deal with the problem (like in the vast majority of data structure libraries).
But that's not a backward compatible change ;)

I agree that the FOREACH macro is kinda out of place, and a synchronized variant should be available. We will probably do it soonish.
Feel free to create a bug in our issues database, https://github.com/pmem/issues/issues, that will at least make sure we won't forget about it ;)

Piotrek

[Clark Laughlin]

Issue #246 created: https://github.com/pmem/issues/issues/246

I think that ensuring all of the operations are thread safe is better than leaving it up to the programmer.  Ensuring safety of the data and data structures in persistent memory is so much more important that for volatile memory, given that you can't just reboot the system to fix a problem ;) Providing that within the library ensures that it is solved once -- correctly.

Thanks!