-
Bug
-
Resolution: Done
-
Major
-
OADP 1.2.0
-
False
-
-
False
-
QE - Ack
-
oadp-operator-container-1.2.0-44
-
ToDo
-
-
-
0
-
0
-
Very Likely
-
0
-
None
-
Unset
-
Unknown
-
Yes
Backup using "restic" failed on single namespace with 5000 pods pvc size 32MB
on "ocs-storagecluster-ceph-rbd"
adp version: iib:387699 - v1.2.0 , velero v1.9.2-OADP
the error "
the main error : "Fatal: invalid id \"${ID}\": no matching ID found for prefix" appears multiple times for more then one ID
the full ID's list can be found on the attached file ID's.list
update ( 9-1-23)
reproduce again , failed in total two cycles out of two
exit status 1 stderr: Fatal: invalid id \"6f87e619\": no matching ID found for prefix \"6f87e619\"
the full error from the velero log command :
time="2023-01-05T17:02:21Z" level=info msg="1 errors encountered backup up item" backup=openshift-adp/single-ns-5000-pods-restic-rbd-backup2 logSource="/remote-source/velero/app/pkg/backup/ba ckup.go:421" name=busybox-perf-single-ns-5000-pods-587 time="2023-01-05T17:02:21Z" level=error msg="Error backing up item" backup=openshift-adp/single-ns-5000-pods-restic-rbd-backup2 error="pod volume backup failed: running backup, stderr=error r unning restic backup command restic backup --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/oadp-bucket/velero/restic/busybox-perf-single-ns-5000-pods --password-f ile=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic . --tag=backup-uid=1a2b4416-23f5-423f-b095-ebb3e3d7b9c1 --tag=ns=busybox-perf -single-ns-5000-pods --tag=pod=busybox-perf-single-ns-5000-pods-587 --tag=pod-uid=2e9aa1d4-5f55-4781-91fe-ea0915612c7c --tag=pvc-uid=73fca5e8-2675-4019-b899-ea0764de099d --tag=volume=busybox- perf-single-ns-5000-pods-587 --tag=backup=single-ns-5000-pods-restic-rbd-backup2 --host=velero --json --insecure-tls=true --parent=6f87e619 with error: exit status 1 stderr: Fatal: invalid id \"6f87e619\": no matching ID found for prefix \"6f87e619\"\n: error running restic backup command restic backup --repo=s3:https://s3-openshift-storage.apps.vlan608.rdu2.scalelab.redhat.com/o adp-bucket/velero/restic/busybox-perf-single-ns-5000-pods --password-file=/tmp/credentials/openshift-adp/velero-repo-credentials-repository-password --cache-dir=/scratch/.cache/restic . --tag =backup-uid=1a2b4416-23f5-423f-b095-ebb3e3d7b9c1 --tag=ns=busybox-perf-single-ns-5000-pods --tag=pod=busybox-perf-single-ns-5000-pods-587 --tag=pod-uid=2e9aa1d4-5f55-4781-91fe-ea0915612c7c -- tag=pvc-uid=73fca5e8-2675-4019-b899-ea0764de099d --tag=volume=busybox-perf-single-ns-5000-pods-587 --tag=backup=single-ns-5000-pods-restic-rbd-backup2 --host=velero --json --insecure-tls=true --parent=6f87e619 with error: exit status 1 stderr: Fatal: invalid id \"6f87e619\": no matching ID found for prefix \"6f87e619\"\n" error.file="/remote-source/velero/app/pkg/podvolume/backup per.go:223" error.function="github.com/vmware-tanzu/velero/pkg/podvolume.(*backupper).BackupPodVolumes" logSource="/remote-source/velero/app/pkg/backup/backup.go:425" name=busybox-perf-single -ns-5000-pods-587
velero backup describe :
Name: single-ns-5000-pods-restic-rbd-backup2 Namespace: openshift-adp Labels: velero.io/storage-location=example-velero-1 Annotations: velero.io/source-cluster-k8s-gitversion=v1.24.0+3882f8f velero.io/source-cluster-k8s-major-version=1 velero.io/source-cluster-k8s-minor-version=24Phase: PartiallyFailed (run `velero backup logs single-ns-5000-pods-restic-rbd-backup2` for more information)Errors: 2365 Warnings: 0Namespaces: Included: busybox-perf-single-ns-5000-pods Excluded: <none>Resources: Included: * Excluded: <none> Cluster-scoped: autoLabel selector: <none>Storage Location: example-velero-1Velero-Native Snapshot PVs: falseTTL: 720h0m0sCSISnapshotTimeout: 10m0sHooks: <none>Backup Format Version: 1.1.0Started: 2023-01-05 08:26:05 +0000 UTC Completed: 2023-01-05 17:33:12 +0000 UTCExpiration: 2023-02-04 08:26:05 +0000 UTCTotal items to be backed up: 40022 Items backed up: 40022Velero-Native Snapshots: <none included>restic Backups (specify --details for more information): Completed: 2635 Failed: 2365
