rook-ceph-operator missing rook-ceph.conf and keyring

[Samuel Giroux]

sh-4.2# rook version

 rook: v1.2.0

ceph version 14.2.6

sh-4.2# ceph -s

  cluster:

    id:     89f14a13-935f-4502-ab8b-acf6ded7022a

    health: HEALTH_WARN

            mon j is low on available space

  services:

    mon: 3 daemons, quorum g,i,j (age 16h)

    mgr: a(active, since 2w)

    mds: cephfs:1 {0=cephfs-b=up:active} 1 up:standby-replay

    osd: 29 osds: 24 up (since 18h), 24 in (since 3w)

    rgw: 3 daemons active (object.store.a, object.store.b, object.store.c)

  data:

    pools:   10 pools, 1096 pgs

    objects: 561.44k objects, 2.1 TiB

    usage:   6.1 TiB used, 33 TiB / 39 TiB avail

    pgs:     1096 active+clean

  io:

    client:   12 KiB/s rd, 13 MiB/s wr, 7 op/s rd, 90 op/s wr

After rebooting two nodes from our cluster, the rook-operator is unable to create PVCs: 

```failed to provision volume with StorageClass "rook-ceph-block": failed to create rook block image replicapool/pvc-125a3447-1899-11eb-90b0-52e0b1ad3548: failed to create image pvc-125a3447-1899-11eb-90b0-52e0b1ad3548 in pool replicapool of size 4398046511104, output: : Failed to complete '': exit status 1. global_init: unable to open config file from search list /var/lib/rook/rook-ceph/rook-ceph.config```

/var/lib is also missing from the rook-operator container. 

We tried redeploying the operator but the issue is still present. Not really sure what is supposed to create /var/lib/rook/rook-ceph/rook-ceph.config and /var/lib/rook/rook-ceph/client.admin.keyring on the operator? We temporarily patch this issue by adding the files manually to the rook-operator container but it would be great to know how this process works to fix it permanently.

Thank you for the help!

[Travis Nielsen]

Hi Samuel,

Th operator is expected to generate those files for the Ceph configuration. If restarting the operator pod (deleting the pod) doesn’t fix it, this is certainly unexpected. If you hit it again, please open a GitHub issue and include the operator log.

Thanks!

Travis

--
You received this message because you are subscribed to the Google Groups "rook-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rook-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rook-dev/0310a667-8aed-473d-9590-6c75032c566bn%40googlegroups.com.