Details
-
Bug
-
Resolution: Done
-
Major
-
OADP 1.2.0
-
None
-
None
-
False
-
-
False
-
ToDo
-
No
-
0
-
0
-
Very Likely
-
0
-
None
-
Unset
-
Unknown
Description
Description of problem:
running restore using Restic on single ns with 1000 pods , PVC size 32MB
on ceph-rbd
the restore failed on the following 2 different errors
time="2023-05-11T20:40:52Z" level=error msg="unable to successfully complete pod volume restores of pod's volumes" error="error getting persistent volume claim for volume: pe rsistentvolumeclaims \"pvc-busybox-perf-single-ns-1000-pods-105\" not found" logSource="/remote-source/velero/app/pkg/restore/restore.go:1699" restore=openshift-adp/restore-r estic-busybox-perf-single-1000-pods-rbd-rbd time="2023-05-11T20:40:55Z" level=error msg="unable to successfully complete pod volume restores of pod's volumes" error="error getting persistent volume claim for volume: pe rsistentvolumeclaims \"pvc-busybox-perf-single-ns-1000-pods-106\" not found" logSource="/remote-source/velero/app/pkg/restore/restore.go:1699" restore=openshift-adp/restore-r estic-busybox-perf-single-1000-pods-rbd-rbd time="2023-05-11T20:40:58Z" level=error msg="unable to successfully complete pod volume restores of pod's volumes" error="error getting persistent volume claim for volume: pe rsistentvolumeclaims \"pvc-busybox-perf-single-ns-1000-pods-107\" not found" logSource="/remote-source/velero/app/pkg/restore/restore.go:1699" restore=openshift-adp/restore-r
time="2023-05-11T21:34:02Z" level=error msg="Namespace busybox-perf-single-ns-1000-pods, resource restore error: error preparing persistentvolumeclaims/busybox-perf-single-ns-1000-pods/pvc-busybox-perf-single-ns-1000-pods-997: rpc error: code = Unknown desc = Failed to get Volumesnapshot busybox-perf-single-ns-1000-pods/velero-pvc-busybox-perf-single-ns-1000-pods-997-8hxmj to restore PVC busybox-perf-single-ns-1000-pods/pvc-busybox-perf-single-ns-1000-pods-997: volumesnapshots.snapshot.storage.k8s.io \"velero-pvc-busybox-perf-single-ns-1000-pods-997-8hxmj\" not found" logSource="/remote-source/velero/app/pkg/controller/restore_controller.go:498" restore=openshift-adp/restore-restic-busybox-perf-single-1000-pods-rbd-rbd time="2023-05-11T21:34:02Z" level=error msg="Namespace busybox-perf-single-ns-1000-pods, resource restore error: error preparing persistentvolumeclaims/busybox-perf-single-ns-1000-pods/pvc-busybox-perf-single-ns-1000-pods-998: rpc error: code = Unknown desc = Failed to get Volumesnapshot busybox-perf-single-ns-1000-pods/velero-pvc-busybox-perf-single-ns-1000-pods-998-vrns8 to restore PVC busybox-perf-single-ns-1000-pods/pvc-busybox-perf-single-ns-1000-pods-998: volumesnapshots.snapshot.storage.k8s.io \"velero-pvc-busybox-perf-single-ns-1000-pods-998-vrns8\" not found" logSource="/remote-source/velero/app/pkg/controller/restore_controller.go:498" restore=openshift-adp/restore-restic-busybox-perf-single-1000-pods-rbd-rbd time="2023-05-11T21:34:02Z" level=error msg="Namespace busybox-perf-single-ns-1000-pods, resource restore error: error preparing persistentvolumeclaims/busybox-perf-single-ns-1000-pods/pvc-busybox-perf-single-ns-1000-pods-999: rpc error: code = Unknown desc = Failed to get Volumesnapshot busybox-perf-single-ns-1000-pods/velero-pvc-busybox-perf-single-ns-1000-pods-999-stpqw to restore PVC busybox-perf-single-ns-1000-pods/pvc-busybox-perf-single-ns-1000-pods-999: volumesnapshots.snapshot.storage.k8s.io \"velero-pvc-busybox-perf-single-ns-1000-pods-999-stpqw\" not found" logSource="/remote-source/velero/app/pkg/controller/restore_controller.go:498" restore=openshift-adp/restore-restic-busybox-perf-single-1000-pods-rbd-rbd
Version-Release number of selected component (if applicable):
OCP: 4.11.7
ODF: 4.11.7
OADP: 1.2.0-69
Steps to Reproduce:
1.create a backup-restic-busybox-perf-single-1000-pods-rbd
2.after backup completed delete the ns
3.run restore restore-restic-busybox-perf-single-1000-pods-rbd-rbd with the same backup CR : backup-restic-busybox-perf-single-1000-pods-rbd
Actual results:
"status": { "completionTimestamp": "2023-05-11T21:34:02Z", "errors": 2000, "phase": "PartiallyFailed", "progress": { "itemsRestored": 9025, "totalItems": 9025 }, "startTimestamp": "2023-05-11T20:28:24Z", "warnings": 11
Expected results:
Additional info:
this bug may be related to this one :
https://issues.redhat.com/browse/OADP-1844