Persistent memory in MPI applications

[Anders S]

Hi,

I am new in this subject and has two very basic questions:

1. Is use of persistent memory a way to avoid hard disk access problems when threads want to read and write files?

2. If a software is developed with use of persistent memory, is it possible to run it on hardware without persistent memory (file read/write

takes place on the hard disk instead)?

Best regards

Anders S

[steve]

On Tue, 10 Mar 2020 09:07:20 -0700 (PDT), Anders S <anders...@gmail.com> wrote:

>Hi,
>I am new in this subject and has two very basic questions:
>
>1. Is use of persistent memory a way to avoid hard disk access problems
>when threads want to read and write files?

I don't understand this question.

>2. If a software is developed with use of persistent memory, is it possible
>to run it on hardware without persistent memory (file read/write
>takes place on the hard disk instead)?

Yes, that is possible. I know because I have done it.
However, I don't believe the usual libraries like pmem support it.
------------
Steve Heller

[Andy Rudoff]

Hi Anders,

1. Is use of persistent memory a way to avoid hard disk access problems when threads want to read and write files?

If you are asking about avoiding locks that might be taken during the path through a traditional I/O stack, yes, you can avoid those locks because you get direct load/store access to pmem once you have memory mapped it.  But of course it then becomes your problem to coordinate access between multiple application threads (or use an MT safe library like those in PMDK).  In general, you can do much finer-grained locking with the memory mapped pmem but the specifics of your use case will dictate if this is true for you.

 

2. If a software is developed with use of persistent memory, is it possible to run it on hardware without persistent memory (file read/write

takes place on the hard disk instead)?

The pmem programming model builds on top of the standard file APIs.  If you use those APIs, your code will work on both pmem and traditional storage.  But a single persistence algorithm for both is hardly ever what you want because that way, you end up using the persistence according to the least common denominator of storage capabilities.  While we often use traditional storage for developing and debugging pmem-aware code, a real-world application is much more likely to detect when real pmem is present (there's a function to do this in libpmem called pmem_is_pmem()) and switch to pmem-resident data structures, fine grain updates and flushing when pmem is detected.  If you use those pmem-optimized operations on traditional storage, your performance suffers because a small flush becomes a block flush and a small read become a block read, etc.  Therefore my best advice is not to treat pmem and storage as identical in your program but instead use them both for what they're good at, when you detect they are available.

We cover these aspects in more detail in the first three chapters of the book at pmem.io/book (it is free).

Thanks,

-andy

[Anders Salwén]

Thanks Steve and Andy!

Anders S

--
You received this message because you are subscribed to the Google Groups "pmem" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pmem+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pmem/45b727d0-4587-49be-8776-bb14a0031e59%40googlegroups.com.