Details
-
Bug
-
Resolution: Done
-
Major
-
None
-
False
-
-
False
-
Release Notes
-
Passed
-
No
-
Note: OADP/Velero currently does not support backup/restore of AWS EFS Volumes
-
Known Issue
-
0
-
0
-
Very Likely
-
0
-
Customer Escalated, Customer Facing
-
None
-
Unset
-
Unknown
Description
Description of problem:
Restic restoration fails on AWS EFS storage with the following error:
time="2022-08-30T20:37:48Z" level=error msg="unable to successfully complete restic restores of pod's volumes" error="pod volume restore failed: error restoring volume: error running restic restore, cmd=restic restore --repo=s3:s3-us-east-2.amazonaws.com/cs-app-backup-bucket-dev/velero/restic/ckretler-sandbox --password-file=/tmp/credentials/openshift-adp/velero-restic-credentials-repository-password --cache-dir=/scratch/.cache/restic e272f797 --target=. --delete --skip-unchanged, stdout=Skip Unchanged True\nrestoring <Snapshot e272f797 of [/host_pods/5a2244f0-9ce4-428d-8aff-8af799a015cc/volumes/kubernetes.io~csi/pvc-7ac3424c-bc50-4119-a88d-6382f7f8d866/mount] at 2022-08-30 20:35:22.652520881 +0000 UTC by root@velero> to .\n, stderr=ignoring error for /bigfile: Lchown: lchown /host_pods/fc1194b7-f306-4dac-9543-d9c5dcf91c26/volumes/kubernetes.io~csi/pvc-6130721b-2d5f-4897-a2b3-e5890e9d7a78/mount/bigfile: operation not permitted\nignoring error for /file.txt: Lchown: lchown /host_pods/fc1194b7-f306-4dac-9543-d9c5dcf91c26/volumes/kubernetes.io~csi/pvc-6130721b-2d5f-4897-a2b3-e5890e9d7a78/mount/file.txt: operation not permitted\nFatal: There were 2 errors\n\n: exit status 1" logSource="pkg/restore/restore.go:1464" restore=openshift-adp/ckretler-restore
Version-Release number of selected component (if applicable):
OADP operator 1.0.3
How reproducible:
All the time.
Steps to Reproduce:
- OADP operator installed
- EFS CSI driver installed
- Create a SC using EFS
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: annotations: storageclass.kubernetes.io/is-default-class: "true" name: efs parameters: basePath: /pvs directoryPerms: "700" fileSystemId: fs-0886f245d447f0cf5 provisioningMode: efs-ap provisioner: efs.csi.aws.com reclaimPolicy: Delete volumeBindingMode: Immediate - Some workload that is using the EFS SC
- Run a restic backup
- Delete the workload
- Restore the backup
Actual results:
Restoration fails with following error:
time="2022-08-30T20:37:48Z" level=error msg="unable to successfully complete restic restores of pod's volumes" error="pod volume restore failed: error restoring volume: error running restic restore, cmd=restic restore --repo=s3:s3-us-east-2.amazonaws.com/cs-app-backup-bucket-dev/velero/restic/ckretler-sandbox --password-file=/tmp/credentials/openshift-adp/velero-restic-credentials-repository-password --cache-dir=/scratch/.cache/restic e272f797 --target=. --delete --skip-unchanged, stdout=Skip Unchanged True\nrestoring <Snapshot e272f797 of [/host_pods/5a2244f0-9ce4-428d-8aff-8af799a015cc/volumes/kubernetes.io~csi/pvc-7ac3424c-bc50-4119-a88d-6382f7f8d866/mount] at 2022-08-30 20:35:22.652520881 +0000 UTC by root@velero> to .\n, stderr=ignoring error for /bigfile: Lchown: lchown /host_pods/fc1194b7-f306-4dac-9543-d9c5dcf91c26/volumes/kubernetes.io~csi/pvc-6130721b-2d5f-4897-a2b3-e5890e9d7a78/mount/bigfile: operation not permitted\nignoring error for /file.txt: Lchown: lchown /host_pods/fc1194b7-f306-4dac-9543-d9c5dcf91c26/volumes/kubernetes.io~csi/pvc-6130721b-2d5f-4897-a2b3-e5890e9d7a78/mount/file.txt: operation not permitted\nFatal: There were 2 errors\n\n: exit status 1" logSource="pkg/restore/restore.go:1464" restore=openshift-adp/ckretler-restore
Expected results:
A successful backup restoration
