lvmcache problem after reboot

[Robert SK]

Hello,

I try to use lvmcache for LVM, the problem is that after restart LVM is "NOT available". Probably at startup something is wrong.

I tried to change this but it didn't help.
/etc/rc.conf
rc.dmcache_enable=YES

I tried the following versions:
3.0.14
2.1.5


I have the same problem with lvmcache on the test VM
===

after reboot

  LV Path                /dev/VG_-_DATA/lv_-_data
  LV Name                lv_-_data
  VG Name                VG_-_DATA
  LV UUID                pB1uec-Tifa-AYFy-h1OA-DTFY-1kGt-JXUnJI
  LV Write Access        read/write
  LV Creation host, time localhost, 2023-01-03 10:04:04 +0000
  LV Cache pool name     lv1_cache
  LV Cache origin name   lv_-_data_corig
  LV Status              NOT available
  LV Size                7.00 GiB
  Current LE             1792
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto

===



Version 4 (tested: 4.0.8 , 4.0.12) on my old server doesn't boot at all, it quits right at the beginning, only the "GRUB" message appears.

thanks

[Robert SK]

lsmod

LVM works fine, before reboot

===

[root@localhost ~]# lsmod
Module                  Size  Used by    Tainted: P
dm_cache_smq           20480  1
scst_qla2xxx           24576  0
fcst                   28672  0
scst_changer           16384  0
scst_tape              16384  0
scst_vdisk             90112  0
scst_disk              16384  0
ib_srpt                40960  0
isert_scst             49152  4
iscsi_scst             81920  4 isert_scst
scst                 1286144  8 scst_qla2xxx,fcst,scst_changer,scst_tape,scst_vdisk,scst_disk,ib_srpt,iscsi_scst
zfs                  3182592  6
zlua                  131072  1 zfs
zcommon                69632  1 zfs
znvpair                53248  2 zfs,zcommon
icp                   253952  1 zfs
zunicode              327680  1 zfs
zavl                   16384  1 zfs
spl                    81920  5 zfs,zcommon,znvpair,icp,zavl
qla2xxx               663552  1 scst_qla2xxx
bonding               110592  0
cls_switchtec          36864  0
mlx5_core             307200  0
bna                   126976  0
ib_umad                20480  0
rdma_ucm               24576  0
rdma_cm                45056  3 ib_srpt,isert_scst,rdma_ucm
iw_cm                  36864  1 rdma_cm
ib_uverbs              73728  1 rdma_ucm
ib_srp                 40960  0
ib_cm                  40960  3 ib_srpt,rdma_cm,ib_srp
iw_nes                151552  0
iw_cxgb4              118784  0
cxgb4                 262144  1 iw_cxgb4
iw_cxgb3               69632  0
ib_qib                315392  0
rdmavt                 69632  1 ib_qib
mlx4_ib               139264  0
mlx4_core             266240  1 mlx4_ib
ib_mthca              122880  0
ib_core               184320 16 ib_srpt,isert_scst,ib_umad,rdma_ucm,rdma_cm,iw_cm,ib_uverbs,ib_srp,ib_cm,iw_nes,iw_cxgb4,iw_cxgb3,ib_qib,rdmavt,mlx4_ib,ib_mthca

===

after reboot = LVM failed

===

[root@localhost ~]# lsmod
Module                  Size  Used by    Tainted: P
dm_cache_smq           20480  0
zfs                  3182592  6
zlua                  131072  1 zfs
zcommon                69632  1 zfs
znvpair                53248  2 zfs,zcommon
icp                   253952  1 zfs
zunicode              327680  1 zfs
zavl                   16384  1 zfs
spl                    81920  5 zfs,zcommon,znvpair,icp,zavl
qla2xxx               663552  0
bonding               110592  0
cls_switchtec          36864  0
mlx5_core             307200  0
bna                   126976  0
ib_umad                20480  0
rdma_ucm               24576  0
rdma_cm                45056  1 rdma_ucm
iw_cm                  36864  1 rdma_cm
ib_uverbs              73728  1 rdma_ucm
ib_srp                 40960  0
ib_cm                  40960  2 rdma_cm,ib_srp
iw_nes                151552  0
iw_cxgb4              118784  0
cxgb4                 262144  1 iw_cxgb4
iw_cxgb3               69632  0
ib_qib                315392  0
rdmavt                 69632  1 ib_qib
mlx4_ib               139264  0
mlx4_core             266240  1 mlx4_ib
ib_mthca              122880  0
ib_core               184320 14 ib_umad,rdma_ucm,rdma_cm,iw_cm,ib_uverbs,ib_srp,ib_cm,iw_nes,iw_cxgb4,iw_cxgb3,ib_qib,rdmavt,mlx4_ib,ib_mthca

===

[Marc Smith]

I don't believe the newer version of LVM2 was added until the 4.x
version (and current master builds) to support 'lvmcache'. I use
'lvmcache' (dm-writecache and dm-cache) regularly with ESOS 4.x and
nothing special needs to load on boot. Enable 'rc.lvm2' in
'/etc/rc.conf' so it activates volume groups on boot.

If you're having GRUB boot loader issues, you could try changing your
USB emulation type (eg, on Dell servers). Or perhaps updating your
BIOS may help (latest version).

--Marc

> --
> You received this message because you are subscribed to the Google Groups "esos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to esos-users+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/esos-users/46d62d86-6cc0-4a18-866f-16b43231d65en%40googlegroups.com.

[Robert SK]

Hello,

When I changed the emulation for USB Disk in BIOS, version 4 starts correctly.

I tried dm-cache so according to everything it works.


I tried dm-writecache and there i have a problem, when i create a new device (TUI) with Block Size 512 so when i want to add dm-writecache so when i want to enter the size and offer the option 512/4096 so 512 does not take and when I leave 4096 so there are errors in the dmesg.

When I make a new device and give the sector size 4096, the dm-writecache is created without error but then ESXi does not display this disk.


Can you write how you have it created on your server ?

thanks

[Marc Smith]

On Sat, Jan 14, 2023 at 11:32 AM Robert SK <sk.dh....@gmail.com> wrote:
>
> Hello,
>
> When I changed the emulation for USB Disk in BIOS, version 4 starts correctly.
>
> I tried dm-cache so according to everything it works.
>
>
> I tried dm-writecache and there i have a problem, when i create a new device (TUI) with Block Size 512 so when i want to add dm-writecache so when i want to enter the size and offer the option 512/4096 so 512 does not take and when I leave 4096 so there are errors in the dmesg.
>
> When I make a new device and give the sector size 4096, the dm-writecache is created without error but then ESXi does not display this disk.
>
>
> Can you write how you have it created on your server ?

All of my storage is 4Kn (4096 physical / 4096 logical) block size
(the backing/origin device AND the cache device). With at least
dm-writecache you need the same logical block size for both the cache
LV and the origin/backing LV. The logical block size chosen in the
SCST device config must be a multiple of the underlying block device
(for vdisk_blockio) logical block size OR match the logical block size
of that device (I always make mine match).

--Marc

> To view this discussion on the web visit https://groups.google.com/d/msgid/esos-users/29a3c97a-536c-4574-a6c8-b40c1ced98b9n%40googlegroups.com.