How to specify a file, that is a dax device file?

[fengyule...@gmail.com]

Hi

I'm playing with nvml library recently, and tried to run simple_copy.

[root@grantleyIPDC03 libpmem]# ./simple_copy
usage: ./simple_copy src-file dst-file

In util_fd_is_device_dax,  it works like this:
1. Check whether the fd opened with dst-file is char device
2. Get the real path of /sys/dev/char/major_n:minor_n/subsystem
3. Check whether real path is identical with /sys/class/dax

The code is self-explanatory, while my newbie-question is how to specify the dst-file
to match the criteria?



 

[Wojciech Uss]

Hi,

Do you have any Device DAX configured on your machine?

If yes, its path like `/dev/dax0.0` should exist and that is what you can pass for dst-file to match the criteria.

If not, you have to create a Device DAX first. Here you can find a procedure for creating a Device DAX.

Hope that helps.

Wojtek

[fengyule...@gmail.com]

Yes, I have /dev/dax0.0 on my server, but specifying the dst-file as /dev/dax0.0 seems make no sense to me.
/dev/dax0.0 can be opened and mmaped, nvml sits on top of /dev/dax0.0, which carves the persistent memory
in various ways.

I have tested using /dev/dax0.0 before, it pops error:
[root@purley-2s libpmem]# ./simple_copy /tmp/2.txt /dev/dax0.0
=================================
path "/dev/dax0.0" size 4096 flags 3 mode 666 mapped_lenp 0x7ffca5b9c978 is_pmemp 0x7ffca5b9c974
pmem_map_file: Invalid argument

Or am I missing something else where?

在 2017年5月23日星期二 UTC+8下午6:51:51，Wojciech Uss写道：

[Piotr Balcer]

Hi,

Notice that this example tries to create the destination file with a size of 4 kilobytes.
This is impossible on device dax, as it does not provide fs-like capabilities.
Limitations of the pmem_map_file functions with regards to device dax are listed
in the libpmem man page.

You could modify the example to support device dax by removing the PMEM_FILE_EXCL
flag and the size of the buffer from the arguments to the pmem_map_file.

Piotr

[fengyule...@gmail.com]

hmm, reset the len to ZERO will also clear all the valid flags, then

./simple_copy /tmp/2.txt /dev/dax0.0

will succeed, which means content in file is store in /dev/dax0.0 in raw mode, with any semantics of file system.
I have one more general question about how libpmem and all other supplementary libs in pmem.io:

Currently the NVDIMM kernel design allow user to define block devices like pmem or blk, or device like /dev/dax0.0,

config1:
a. configure pmem or blk block device
b. mkfs the corresponding partition
c. mount the partition with "-o dax" to by pass page cache stuff
d. create files in daxed file system

config2:
a. open /dev/dax0.0
b. mmap with /dev/dax0.0 size
c. Carve out the memory space as libpmem own style.

config3:
something else...

which one does libpmem and all other libs use ?



在 2017年5月24日星期三 UTC+8下午4:38:53，Piotr Balcer写道：

[Piotr Balcer]

Hi,

Our libraries strive to be hardware/configuration agnostic, which means
we try to make sure that the software does the right thing regardless of
what's underneath the stack.

This means that you can use NVML with whatever exposes the
necessary semantics. This includes, but is not limited to,
regular file systems, dax-enabled file systems and raw dax devices.

Does this answer your question?

Piotr

[fengyule...@gmail.com]

Thanks, very clear explanation. :)

在 2017年5月24日星期三 UTC+8下午6:05:52，Piotr Balcer写道：

[Rukhsana Ansari]

Hi,

I posted a query this morning on a  related topic. Hadn't seen this, but  apart from the configs below, is the following a  valid config too:

1. Open pmem device (block device in "raw"mode) in direct io mode

if ((fd = open(“/dev/pmem0”, O_RDWR | O_DIRECT))< 0) {

perror(“open failed.”);

exit(1);

2. mmap the device to ap addr space.

mmapp = mmap(NULL,length,PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset);

3.

// Write to the device using NVML api

pmem_memcpy_persist(mmap, buf, buf_length)

I've tried the above and it works during normal operation. 

However, will  this ensure persistence, during a  AC power loss for a legacy pmem, type 12 NVDIMM (not an emulated one)

[Jeff Moyer]

Rukhsana Ansari <ruk.a...@gmail.com> writes:

> Hi,
>
> I posted a query this morning on a related topic. Hadn't seen this, but
> apart from the configs below, is the following a valid config too:
> 1. Open pmem device (block device in "raw"mode) in direct io mode
> if ((fd = open(“/dev/pmem0”, O_RDWR | O_DIRECT))< 0) {
> perror(“open failed.”);
> exit(1);
>
> 2. mmap the device to ap addr space.
> mmapp = mmap(NULL,length,PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset);
>
>
> 3.
> // Write to the device using NVML api
> pmem_memcpy_persist(mmap, buf, buf_length)
>
> I've tried the above and it works during normal operation.
> However, will this ensure persistence, during a AC power loss for a
> legacy pmem, type 12 NVDIMM (not an emulated one)

Today, Linux does not support DAX on the block device. If you need
direct access to raw pmem (without a file system), then you need to use
device dax (/dev/dax*).

Cheers,
Jeff

>>>>>> <https://gist.github.com/wojtuss/ababfaa0d867d97f9cbd65d5a336a532> you

>>>>>> can find a procedure for creating a Device DAX.
>>>>>>
>>>>>> Hope that helps.
>>>>>> Wojtek
>>>>>>
>>>>>>
>>>>>> On Tuesday, May 23, 2017 at 9:45:14 AM UTC+2, fengyule...@gmail.com
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> I'm playing with nvml library recently, and tried to run simple_copy

>>>>>>> <https://github.com/pmem/nvml/blob/master/src/examples/libpmem/simple_copy.c>
>>>>>>> .

>>>>>>>
>>>>>>> [root@grantleyIPDC03 libpmem]# ./simple_copy
>>>>>>> usage: ./simple_copy src-file dst-file
>>>>>>>
>>>>>>> In util_fd_is_device_dax,

>>>>>>> <https://github.com/pmem/nvml/blob/master/src/common/file.c#L131> it

[Rukhsana Ansari]

Thanks Jeff.