pmalloc perf

[Qing Chen]

HI,

We want to test this persist allocator in the libpmemobj.dll performance in our latest Nvm enabled Windows server, we found pmalloc/pfree performance is very bad , > 1000 times slower than malloc,   here is what i am using. 

// simple logic is like this :

pop = pmemobj_create(L"pmem.test.fs", NULL, 4 * 1024 * 1024 * 1024ULL, 0);

// in other threads 

pmalloc(pop, &m->offset, size, 0, 0);

pfree(pop, &m->offset);

Anything i missed ? 

btw: do you have any performance number compared with jemalloc/tcmalloc ? 

Thanks!

[Piotr Balcer]

Hi,

libpmemobj is *not* a persistent replacement of stdlib. You cannot compare transient malloc() implementations with persistent memory management.
The algorithms required to maintain correctness and fail safety are vastly different than the ones traditionally used in transient memory allocators such as jemalloc/tcmalloc.

The performance characteristics are also heavily affected by the hardware you are running your benchmarks on, and on the software platform you are using.
Currently the only publicly available numbers we post on our website come from running pmem emulation on regular DRAM.
On such configuration, pmalloc() is capable of millions of allocations per second. See the last two charts on this blog post: http://pmem.io/2017/03/29/progress-report-q1-17.html

Also, if you are really just doing:
void *ptr = malloc(size);
free(ptr);

It's very likely that the compiler is optimizing out the malloc() because it is never used.

It's also worth noting that allocating and freeing the exact same size in a loop is the best case scenario for jemalloc/tcmalloc as you create only one run/span and then remove and insert the exact same entry repeatedly,
whereas in the case of pmalloc it's a case we specifically had to optimize for because the algorithms tries to keep the persistent layout of the heap in the "cleanest" possible state, and that includes deleting runs/spans immediately
after they are entirely free.

Piotr

[Qing Chen]

ok, i just stepped into pmemobj_create function,  found on Windows, this function always return 0 , does it mean it is NOT supported to run on Windows for DAX ? 

int

util_file_is_device_dax(const char *path)

{

LOG(3, "path \"%s\"", path);

#ifdef _WIN32

return 0;

#else

..................

}

we have got real Nvm hardware, and latest Windows Server, could you please tell me how can i run/configure this correctly on a real DAX volume ?   

[Andy Rudoff]

The routine you are talking about is detecting if Device DAX is in use.  Device DAX is a Linux-only feature that allows applications to map full persistent memory devices without any file system on them.  Device DAX was invented for some very specific use cases where things like file names & permissions are not required.

What you want (and what is available in Windows) is file system DAX.  That's where an NTFS file system is created with the DAX option so that when PMDK maps a file, it gets direct access to the persistent memory.

You may find the information that Neal Christiansen presented recently at the PM Summit helpful: https://www.snia.org/pm-summit

If you scroll down, you'll find his slides and a video of his very informative talk about the current state of Windows persistent memory support.

-andy

[Qing Chen]

Thank you for information, yes i have gone through the PPT and video before, and the file system of the volume is NTFS and it is formatted to 2MB cluster size, also on the server the volume is shown a persistent memory and the driver for the DAX volume is pmem.sys same as the  one described in the PPT, so we should have right configuration for file system DAX. 

What i did not understand why this pmalloc and pfree is so slow on the DAX volume ?  here is the number 

allocate/deallocate about 30 times in a single thread with size 64, it takes about  85470 milliseconds, 

BTW, i check the return value of these two routine, both of them are zero, which indicates no error ? 

could you advise what's wrong here ? 

[Piotr Balcer]

That's a ... unrealistically big number ;) Are you sure you are not missing a comma? Could you confirm the same results on a different operating system (i.e., Linux)?

You could also try setting the PMEM_IS_PMEM_FORCE=1 and PMEM_NO_FLUSH=1 environment variables.
If the first one helps, it means that you are not really running on a DAX file system. And if the second helps, it means that your are simply seeing the hardware characteristics.

And yes, 0 indicates a successful allocation.

Piotr

[Czurylo, Krzysztof]

Hi,

 

You can quickly check if you have your system configured properly by running the following PowerShell script (assuming you've built PMDK from source):

 

# path to the DAX-enabled volume

$PMEMPATH = "p:\"

 

# path to pmemdetect utility in PMDK repo

$PMEMDETECT = "c:\github\pmdk\src\x64\Debug\pmemdetect.exe"

 

# path to the log file

$Env:PMEM_LOG_FILE = "c:\temp\pmemdetect.log"

$Env:PMEM_LOG_LEVEL = 4

$Env:PMEM_IS_PMEM_FORCE = 0

$Global:LASTEXITCODE = 0

 

& $PMEMDETECT $PMEMPATH

 

echo $Global:LASTEXITCODE

if ($Global:LASTEXITCODE -eq 0) {

                Write-output "'$PMEMPATH' points to DAX-enabled file system"

} else {

                Write-output "'$PMEMPATH' does NOT point to DAX-enabled file system"

}

 

# uncomment next line to print libpmem log

# Get-Content $Env:PMEM_LOG_FILE

 

K.

 

--
You received this message because you are subscribed to the Google Groups "pmem" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pmem+uns...@googlegroups.com.
To post to this group, send email to pm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pmem/222e26c4-46c8-46b2-b988-388637b05496%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Qing Chen]

PS C:\Users\chingc> $PMEMPATH = "d:\"

PS C:\Users\chingc> $PMEMDETECT = "F:\pmdk-master\src\x64\Release\pmemdetect.exe"

PS C:\Users\chingc> $Env:PMEM_LOG_FILE = "c:\temp\pmemdetect.log"

PS C:\Users\chingc>

PS C:\Users\chingc> $Env:PMEM_LOG_LEVEL = 4

PS C:\Users\chingc>

PS C:\Users\chingc> $Env:PMEM_IS_PMEM_FORCE = 0

PS C:\Users\chingc>

PS C:\Users\chingc> $Global:LASTEXITCODE = 0

PS C:\Users\chingc> echo $Global:LASTEXITCODE

0

PS C:\Users\chingc>

PS C:\Users\chingc> if ($Global:LASTEXITCODE -eq 0) {

>>

>>                 Write-output "'$PMEMPATH' points to DAX-enabled file system"

>>

>> } else {

>>

>>                 Write-output "'$PMEMPATH' does NOT point to DAX-enabled file system"

>>

>> }

'd:\' points to DAX-enabled file system

PS C:\Users\chingc>

[Qing Chen]

sorry, i missed the one command 

here is the result, 

'd:\' does NOT point to DAX-enabled file system