-
Bug
-
Resolution: Unresolved
-
Undefined
-
OADP 1.2.3
-
False
-
-
False
-
ToDo
-
No
-
-
-
0
-
0
-
Very Likely
-
0
-
None
-
Unset
-
Unknown
Description of problem:
I found this bug during manual testing on OADP 1.2.3. This issue happens when the CSI plugin is added in the defaultPluginsList. DataSource and dataSourceRef should get removed in case of filesystem backup and restore.
upstream PR were we reset this values at the time of backup.
https://github.com/vmware-tanzu/velero/issues/6106
Version-Release number of selected component (if applicable):
OADP 1.2.3 (This issue only happens with OADP 1.2)
How reproducible:
Always
Steps to Reproduce:
1. Create a DPA with CSI enabled.
apiVersion: oadp.openshift.io/v1alpha1 kind: DataProtectionApplication metadata: namespace: openshift-adp name: ts-dpa spec: configuration: velero: defaultPlugins: - openshift - aws - csi backupLocations: - velero: provider: aws default: true objectStorage: bucket: miniobucket prefix: velero config: insecureSkipTLSVerify: 'true' profile: minio region: minio s3ForcePathStyle: 'true' s3Url: http://minio-minio.apps.oadp-67130.qe.gcp.devcluster.openshift.com
2. Deploy a stateful application
3. Execute CSI backup
$ oc get backup test-backup -o yaml apiVersion: velero.io/v1 kind: Backup metadata: annotations: velero.io/source-cluster-k8s-gitversion: v1.27.8+4fab27b velero.io/source-cluster-k8s-major-version: "1" velero.io/source-cluster-k8s-minor-version: "27" creationTimestamp: "2024-01-10T06:25:09Z" generation: 6 labels: velero.io/storage-location: ts-dpa-1 name: test-backup namespace: openshift-adp resourceVersion: "43589" uid: 6eed31ea-fbaa-4433-a7a4-bca659b7e792 spec: csiSnapshotTimeout: 10m0s defaultVolumesToFsBackup: false includedNamespaces: - ocp-todolist-mariadb itemOperationTimeout: 1h0m0s storageLocation: ts-dpa-1 ttl: 720h0m0s status: completionTimestamp: "2024-01-10T06:25:40Z" csiVolumeSnapshotsAttempted: 1 csiVolumeSnapshotsCompleted: 1 expiration: "2024-02-09T06:25:09Z" formatVersion: 1.1.0 phase: Completed progress: itemsBackedUp: 69 totalItems: 69 startTimestamp: "2024-01-10T06:25:10Z" version: 1
4. Delete app namespace and trigger restore.
$ oc get restore test-restore -o yaml
apiVersion: velero.io/v1 kind: Restore metadata: creationTimestamp: "2024-01-10T08:34:51Z" generation: 7 name: test-restore namespace: openshift-adp resourceVersion: "102826" uid: 9650269a-0e2a-4051-b0a0-f04fa8057d41 spec: backupName: test-backup excludedResources: - nodes - events - events.events.k8s.io - backups.velero.io - restores.velero.io - resticrepositories.velero.io - csinodes.storage.k8s.io - volumeattachments.storage.k8s.io - backuprepositories.velero.io itemOperationTimeout: 1h0m0s status: completionTimestamp: "2024-01-10T08:35:11Z" phase: Completed progress: itemsRestored: 44 totalItems: 44 startTimestamp: "2024-01-10T08:34:51Z" warnings: 13
5. Enable Restic on DPA, don't remove CSI plugin
apiVersion: oadp.openshift.io/v1alpha1 kind: DataProtectionApplication metadata: namespace: openshift-adp name: ts-dpa spec: configuration: restic: enable: true velero: defaultPlugins: - openshift - aws - csi backupLocations: - velero: provider: aws default: true objectStorage: bucket: miniobucket prefix: velero config: insecureSkipTLSVerify: 'true' profile: minio region: minio s3ForcePathStyle: 'true' s3Url: http://minio-minio.apps.oadp-67130.qe.gcp.devcluster.openshift.com
6. Trigger a fileSytemBackup with restic
oc get backup test-backup1 -o yaml apiVersion: velero.io/v1 kind: Backup metadata: annotations: velero.io/source-cluster-k8s-gitversion: v1.27.8+4fab27b velero.io/source-cluster-k8s-major-version: "1" velero.io/source-cluster-k8s-minor-version: "27" creationTimestamp: "2024-01-10T11:46:13Z" generation: 6 labels: velero.io/storage-location: ts-dpa-1 name: test-backup1 namespace: openshift-adp resourceVersion: "192079" uid: 2975748f-910c-40cc-97b0-6d171c64cd5e spec: csiSnapshotTimeout: 10m0s defaultVolumesToFsBackup: true includedNamespaces: - ocp-todolist-mariadb itemOperationTimeout: 1h0m0s storageLocation: ts-dpa-1 ttl: 720h0m0s status: completionTimestamp: "2024-01-10T11:46:20Z" expiration: "2024-02-09T11:46:13Z" formatVersion: 1.1.0 phase: Completed progress: itemsBackedUp: 76 totalItems: 76 startTimestamp: "2024-01-10T11:46:13Z" version: 1
7. Delete app namespace and trigger restore.
Polarion case:- https://polarion.engineering.redhat.com/polarion/redirect/project/OADP/workitem?id=OADP-522
Actual results:
Restored PVC points to the restored snapshot which might be problematic for the cross cluster scenario(Specifically when the snapshot is stored in the cluster itself).
PVC after restore
$ oc get pvc -n ocp-todolist-mariadb mysql -o yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: backup.velero.io/must-include-additional-items: "true" pv.kubernetes.io/bind-completed: "yes" pv.kubernetes.io/bound-by-controller: "yes" velero.io/backup-name: test-backup velero.io/volume-snapshot-name: velero-mysql-cf26s volume.beta.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io volume.kubernetes.io/selected-node: oadp-67120-hp4b2-worker-c-mv9db.c.openshift-qe.internal volume.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io creationTimestamp: "2024-01-10T08:45:05Z" finalizers: - kubernetes.io/pvc-protection labels: app: mysql backup.velero.io/must-include-additional-items: "true" velero.io/backup-name: test-backup1 velero.io/restore-name: test-restore1 velero.io/volume-snapshot-name: velero-mysql-cf26s name: mysql namespace: ocp-todolist-mariadb resourceVersion: "108997" uid: 031c38c3-c059-47e2-a47d-473a5663c0d3 spec: accessModes: - ReadWriteOnce dataSource: apiGroup: snapshot.storage.k8s.io kind: VolumeSnapshot name: velero-mysql-cf26s dataSourceRef: apiGroup: snapshot.storage.k8s.io kind: VolumeSnapshot name: velero-mysql-cf26s resources: requests: storage: 1Gi storageClassName: standard-csi volumeMode: Filesystem volumeName: pvc-031c38c3-c059-47e2-a47d-473a5663c0d3 status: accessModes: - ReadWriteOnce capacity: storage: 1Gi phase: Bound
Expected results:
Velero should not add DataSource and DataSourceRef fields in PVC spec.
Additional info: