PV VolumeSnapShot Restore issue

[Saddique Khan]

Hello Team,

I want to take volumsnapshot backup for pvc and pv. It looks fine but when I restore it, it skips the pv backup and create the pv with new name. Let me explain what I have enabled in my cluster.

1. I have kubernets cluster and my velero v1.12 is deployed in the velero namespace.

2. I have crd enabled on the kubernets cluster. here is the list:

volumesnapshotclasses vsclass,vsclasses napshot.storage.k8s.io/v1 false VolumeSnapshotClass

volumesnapshotcontents vsc,vscs snapshot.storage.k8s.io/v1 false VolumeSnapshotContent

volumesnapshots vs snapshot.storage.k8s.io/v1 true VolumeSnapshot

3. I have enabled CSI plugin in velero and also --features=EnableCSI shows up in the velero deployment.

4. I have created the volumesnapshotclass with the label to true to my storage class.

5. The backendstoragelocation and volumstoragelocation are minio.

6. Now I have a deployment running in nginx-test1 with pvc and pv.

7. I took the volumesnapshot using the following command:

velero backup create nginx-test1 --include-resources pvc,pv --snapshot-move-data --data-mover velero --include-namespaces=nginx-test --snapshot-volumes=true --wait.

Here is output.

" velero backup describe nginx-test1 --details

Name: nginx-test1

Namespace: velero

Labels: velero.io/storage-location=minio

Annotations: velero.io/resource-timeout=10m0s

velero.io/source-cluster-k8s-gitversion=v1.28.12

velero.io/source-cluster-k8s-major-version=1

velero.io/source-cluster-k8s-minor-version=28

Phase: Completed

Namespaces:

Included: nginx-test

Excluded: <none>

Resources:

Included: pvc, pv

Excluded: <none>

Cluster-scoped: auto

Label selector: <none>

Or label selector: <none>

Storage Location: minio

Velero-Native Snapshot PVs: true

Snapshot Move Data: true

Data Mover: velero

TTL: 720h0m0s

CSISnapshotTimeout: 10m0s

ItemOperationTimeout: 4h0m0s

Hooks: <none>

Backup Format Version: 1.1.0

Started: 2024-08-09 11:01:31 +0200 CEST

Completed: 2024-08-09 11:02:00 +0200 CEST

Expiration: 2024-09-08 11:01:31 +0200 CEST

Total items to be backed up: 2

Items backed up: 2

Backup Item Operations:

Operation for persistentvolumeclaims nginx-test/nginx-pvc:

Backup Item Action Plugin: velero.io/csi-pvc-backupper

Operation ID: du-880a3a03-adde-43f3-a105-55aa4843c227.eb1bcf49-f9f4-434d2c2bc

Items to Update:

datauploads.velero.io velero/nginx-test1-jcf99

Phase: Completed

Progress description: Completed

Created: 2024-08-09 11:01:36 +0200 CEST

Started: 2024-08-09 11:01:36 +0200 CEST

Updated: 2024-08-09 11:01:56 +0200 CEST

Resource List:

v1/PersistentVolume:

- pvc-8b152436-235c-413d-abd4-974cafbed453

v1/PersistentVolumeClaim:

- nginx-test/nginx-pvc

Velero-Native Snapshots: <none included"

8. after this i stopped the nginx-test1 pod and delete the pvc. PV is also get deleted as it has Policy "Delete".

9. I am restoring it by following command

velero restore create --from-backup nginx-test1 --include-cluster-resources --wait

Here is the logs:

"velero restore describe nginx-test1-20240809110357 --details

Name: nginx-test1-20240809110357

Namespace: velero

Labels: <none>

Annotations: <none>

Phase: Completed

Total items to be restored: 3

Items restored: 3

Started: 2024-08-09 11:03:57 +0200 CEST

Completed: 2024-08-09 11:04:19 +0200 CEST

Backup: nginx-test1

Namespaces:

Included: all namespaces found in the backup

Excluded: <none>

Resources:

Included: *

Excluded: nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io, csinodes.storage.k8s.io, volumeattachments.storage.k8s.io, backuprepositories.velero.io

Cluster-scoped: included

Namespace mappings: <none>

Label selector: <none>

Or label selector: <none>

Restore PVs: auto

Existing Resource Policy: <none>

ItemOperationTimeout: 4h0m0s

Preserve Service NodePorts: auto

Restore Item Operations:

Operation for persistentvolumeclaims nginx-test/nginx-pvc:

Restore Item Action Plugin: velero.io/csi-pvc-restorer

Operation ID: dd-b4be3454-42fe-4636-b32b-be05e35cef8d.eb1bcf49-f9f4-434217395

Phase: Completed

Progress description: Completed

Created: 2024-08-09 11:03:58 +0200 CEST

Started: 2024-08-09 11:03:58 +0200 CEST

Updated: 2024-08-09 11:04:11 +0200 CEST

Resource List:

v1/PersistentVolume:

- pvc-8b152436-235c-413d-abd4-974cafbed453(skipped)

v1/PersistentVolumeClaim:

- nginx-test/nginx-pvc(created)

velero.io/v2alpha1/DataUpload:

- velero/nginx-test1-jcf99(skipped)

"

11. it creates the new pv and there is no data in it.

Now I don't what is the issue and why it is skipping the pv restore.

[Scott Seago]

The new PV on restore is expected. The restored PVC is expected to get
a new one provisioned by the storage class on restore. If the data is
not there, something must have gone wrong on the data mover restore
(or possibly on the backup).

Next steps would be:
1) check the status of the DataDownload (restoring data from BSL to
volume) and DataUpload (backing up data to BSL) CRs in the velero
namespace -- this should include the amount of volume data processed
(bytesDone, totalBytes), as well as phase and any errors encountered.
2) Check backup/restore logs for your backup and restore as well as
the node agent logs for the node agent that processed the data upload
and the data download.

Scott

> --
> You received this message because you are subscribed to the Google Groups "Project Velero" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to projectveler...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/projectvelero/3ade11e3-733d-4553-8d30-1134a657b8dfn%40googlegroups.com.