persisting lvm with fsdax device

[chris....@gmail.com]

Hi experts,

I am trying to make a lvm using the fsdax device. I took reference from the blog: https://pmem.io/2018/05/15/using_persistent_memory_devices_with_the_linux_device_mapper.html

While I can make a linear mapped device, But I got problem regarding how I can restore the lvm.

Just like in this blog, I use systemd but with a much simpler script as I just try to test the lvm restoring. Because the pmem device names usually do not change across reboot or power cycle as I observed. 

I wrote the script just like this:

echo -e "0 `blockdev --getsz /dev/pmem8` linear /dev/pmem8 0 "\\n"`blockdev --getsz /dev/pmem8` `blockdev --getsz /dev/pmem9` linear /dev/pm    em9 0" |sudo dmsetup create linear-pmem

mount -o dax /dev/mapper/linear-pmem /mnt/pmeml/

and save it as /opt/pmem/pmem-mapper

Then I created a service in the /etc/systemd/system/pmem-dev-mapper.service with content:

[Unit]

Description=Create Persistent Memory liner map and mount it

[Service]

Type=simple

ExecStart=/opt/pmem/pmem-mapper

[Install]

WantedBy=multi-user.target

after change it to excutable, I start and enable the service.

But after I reboot the machine, the mapping failed and sysctl showed :

Bash# systemctl status pmem-dev-mapper.service

● pmem-dev-mapper.service - Create Persistent Memory liner map and mount it

   Loaded: loaded (/etc/systemd/system/pmem-dev-mapper.service; enabled; vendor preset: disabled)

   Active: failed (Result: exit-code) since Tue 2018-11-27 16:48:49 CST; 1min 17s ago

  Process: 1919 ExecStart=/opt/pmem/pmem-mapper (code=exited, status=32)

 Main PID: 1919 (code=exited, status=32)

Nov 27 16:48:48 tserver-1 systemd[1]: Starting Create Persistent Memory liner map and mount it...

Nov 27 16:48:48 tserver-1 pmem-dev-mapper[1919]: blockdev: cannot open /dev/pmem8: No such file or d...ory

Nov 27 16:48:48 tserver-1 pmem-dev-mapper[1919]: blockdev: cannot open /dev/pmem8: No such file or d...ory

Nov 27 16:48:48 tserver-1 pmem-dev-mapper[1919]: blockdev: cannot open /dev/pmem9: No such file or d...ory

Nov 27 16:48:49 tserver-1 pmem-dev-mapper[1919]: Invalid format on line 1 of table on stdin

Nov 27 16:48:49 tserver-1 pmem-dev-mapper[1919]: Command failed

Nov 27 16:48:49 tserver-1 pmem-dev-mapper[1919]: mount: special device /dev/mapper/linear-pmem does ...ist

Nov 27 16:48:49 tserver-1 systemd[1]: pmem-dev-mapper.service: main process exited, code=exited, sta.../n/a

Nov 27 16:48:49 tserver-1 systemd[1]: Unit pmem-dev-mapper.service entered failed state.

Nov 27 16:48:49 tserver-1 systemd[1]: pmem-dev-mapper.service failed.

Hint: Some lines were ellipsized, use -l to show in full.

I can remap the pmem devices with the instructions in the script after the reboot, but why I cannot restore it through the systemd service?

I see that in the blog, it created link using uuid considering that the pmem device name might change after reboot but not the uuid. But if the pmem device names do not change across the reboot, I can write the service like this and I think it should work. 

Can experts here understand this issue and take a look? thanks a lot.

 

[Jeff Moyer]

chris....@gmail.com writes:

> Hi experts,
> I am trying to make a lvm using the fsdax device. I took reference from the
> blog: https://pmem.io/2018/05/15/using_persistent_memory_devices_with_the_linux_device_mapper.html
> While I can make a linear mapped device, But I got problem regarding how I
> can restore the lvm.
>
> Just like in this blog, I use systemd but with a much simpler script as I
> just try to test the lvm restoring. Because the pmem device names usually
> do not change across reboot or power cycle as I observed.
> I wrote the script just like this:
>
> echo -e "0 `blockdev --getsz /dev/pmem8` linear /dev/pmem8 0 "\\n"`blockdev
> --getsz /dev/pmem8` `blockdev --getsz /dev/pmem9` linear /dev/pm em9 0" |sudo
> dmsetup create linear-pmem
>
> mount -o dax /dev/mapper/linear-pmem /mnt/pmeml/
>
> and save it as /opt/pmem/pmem-mapper

Why not just use the lvm tools that exist to make this easy for you?

> Then I created a service in the /etc/systemd/system/pmem-dev-mapper.service
> with content:
>
> [Unit]
>
> Description=Create Persistent Memory liner map and mount it
>
>
> [Service]
>
> Type=simple
>
> ExecStart=/opt/pmem/pmem-mapper
>
>
> [Install]
>
> WantedBy=multi-user.target
>
> after change it to excutable, I start and enable the service.
>
> But after I reboot the machine, the mapping failed and sysctl showed :
>
> Bash# systemctl status pmem-dev-mapper.service
>
> ● pmem-dev-mapper.service - Create Persistent Memory liner map and mount it
>
> Loaded: loaded (/etc/systemd/system/pmem-dev-mapper.service; enabled;
> vendor preset: disabled)
>
> Active: failed (Result: exit-code) since Tue 2018-11-27 16:48:49 CST;
> 1min 17s ago
>
> Process: 1919 ExecStart=/opt/pmem/pmem-mapper (code=exited, status=32)
>
> Main PID: 1919 (code=exited, status=32)
>
>
> Nov 27 16:48:48 tserver-1 systemd[1]: Starting Create Persistent Memory
> liner map and mount it...
>
> Nov 27 16:48:48 tserver-1 pmem-dev-mapper[1919]: blockdev: cannot open /dev/
> pmem8: No such file or d...ory

You're likely running into a problem with timing. The devices aren't
available when your service is started. Your service file probably
needs to add a dependency on those devices. I'm not sure how to do
that. I'd recommend just using lvm, and see if that takes care of this
issue for you automatically.

Cheers,
Jeff

[chris....@gmail.com]

Thanks Jeff.

Thanks for your suggestions.

I use the dmsetup just to follow the blog https://pmem.io/2018/05/15/using_persistent_memory_devices_with_the_linux_device_mapper.html

I tried many method to make the timing right, such as adding "Requires=systemd-udevd.service" or changing "Type=idle", but failed with the same result.

Then I switched to use the lvm command, and it works good. 

I think there is some mistake in the blog. For example it said the uuid would not change across system restarts, so in the blog, it use the uuid from the "ndctl list" to address the device in dir /dev/disk/by-uuid/.

Well, as I observed locally, the uuid in the /dev/disk/by-uuid/ which linked to the pmem device is totally different with the uuid in the "ndctl list".

I think this is pretty wired. I don't know if someone else here meet the same issue, but it does happen on my local machine. 

So in this case, address the device with uuid in the "ndctl list"  would definitely not work.

 

在 2018年11月27日星期二 UTC+8下午10:53:26，Jeff Moyer写道：