Loading...

Details

Type: Bug
Resolution: Unresolved
Priority: Normal
Fix Version/s: OADP 1.3.2
Affects Version/s: OADP 1.3.0
Component/s: data-mover
Labels:
- logs
- triaged

Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
QEStatus:
ToDo
Regression:
No

Cost of Delay:
0
WSJF:
0
Risk Probability:
Very Likely
Risk Score:
0

Workstream:

None

Root Cause:
Unset
Failure Category:
Unknown

SFDC Cases Links:
SFDC Cases Counter:

Description

Description of problem:

When executing any backup with snapshotMoveData: true, the backup status / velero backup describe , does not report any details about the CSI snapshots.

Version-Release number of selected component (if applicable):

1.3.0

How reproducible:

Always

Steps to Reproduce:
1. Create DPA with NodeAgent and CSI enabled

$ oc get dpa -o yaml
apiVersion: v1
items:
- apiVersion: oadp.openshift.io/v1alpha1
  kind: DataProtectionApplication
  metadata:
    creationTimestamp: "2023-11-08T06:30:36Z"
    generation: 3
    name: ts-dpa
    namespace: openshift-adp
    resourceVersion: "105091"
    uid: ef94fc25-fc44-4a26-b02b-ab935bf1e5b5
  spec:
    backupLocations:
    - velero:
        credential:
          key: cloud
          name: cloud-credentials-gcp
        default: true
        objectStorage:
          bucket: oadpbucket245306
          prefix: velero
        provider: gcp
    configuration:
      nodeAgent:
        enable: true
        uploaderType: kopia
      velero:
        defaultPlugins:
        - openshift
        - csi
        - gcp
  status:
    conditions:
    - lastTransitionTime: "2023-11-08T06:30:36Z"
      message: Reconcile complete
      reason: Complete
      status: "True"
      type: Reconciled
kind: List
metadata:
  resourceVersion: ""

2. Deploy any stateful application
3. Perform backup with datamover enabled

$ oc get backup -o yaml testcsi2
apiVersion: velero.io/v1
kind: Backup
metadata:
  annotations:
    velero.io/resource-timeout: 10m0s
    velero.io/source-cluster-k8s-gitversion: v1.26.9+636f2be
    velero.io/source-cluster-k8s-major-version: "1"
    velero.io/source-cluster-k8s-minor-version: "26"
  creationTimestamp: "2023-11-08T08:58:11Z"
  generation: 7
  labels:
    velero.io/storage-location: ts-dpa-1
  name: testcsi2
  namespace: openshift-adp
  resourceVersion: "107202"
  uid: d6f7caab-cc41-48de-80a1-eaac9174fabe
spec:
  csiSnapshotTimeout: 10m0s
  defaultVolumesToFsBackup: false
  includedNamespaces:
  - testcsi
  itemOperationTimeout: 4h0m0s
  snapshotMoveData: true
  storageLocation: ts-dpa-1
  ttl: 720h0m0s
status:
  backupItemOperationsAttempted: 2
  backupItemOperationsCompleted: 2
  completionTimestamp: "2023-11-08T08:59:39Z"
  expiration: "2023-12-08T08:58:11Z"
  formatVersion: 1.1.0
  phase: Completed
  progress:
    itemsBackedUp: 54
    totalItems: 54
  startTimestamp: "2023-11-08T08:58:11Z"
  version: 1

Velero Describe backup:

$ velero describe backup testcsi2 -n openshift-adp --details
Name:         testcsi2
Namespace:    openshift-adp
Labels:       velero.io/storage-location=ts-dpa-1
Annotations:  velero.io/resource-timeout=10m0s
              velero.io/source-cluster-k8s-gitversion=v1.26.9+636f2be
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=26

Phase:  Completed


Namespaces:
  Included:  testcsi
  Excluded:  <none>

Resources:
  Included:        *
  Excluded:        <none>
  Cluster-scoped:  auto

Label selector:  <none>

Or label selector:  <none>

Storage Location:  ts-dpa-1

Velero-Native Snapshot PVs:  auto
Snapshot Move Data:          true
Data Mover:                  velero

TTL:  720h0m0s

CSISnapshotTimeout:    10m0s
ItemOperationTimeout:  4h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2023-11-08 14:28:11 +0530 IST
Completed:  2023-11-08 14:29:39 +0530 IST

Expiration:  2023-12-08 14:28:11 +0530 IST

Total items to be backed up:  54
Items backed up:              54

Backup Item Operations:
  Operation for persistentvolumeclaims testcsi/mysql:
    Backup Item Action Plugin:  velero.io/csi-pvc-backupper
    Operation ID:               du-d6f7caab-cc41-48de-80a1-eaac9174fabe.c13148e4-5fe4-4007d4a44
    Items to Update:
                           datauploads.velero.io openshift-adp/testcsi2-cwv84
    Phase:                 Completed
    Progress:              107854713 of 107854713 complete (Bytes)
    Progress description:  Completed
    Created:               2023-11-08 14:28:21 +0530 IST
    Started:               2023-11-08 14:28:21 +0530 IST
    Updated:               2023-11-08 14:29:31 +0530 IST
  Operation for persistentvolumeclaims testcsi/mysql-1:
    Backup Item Action Plugin:  velero.io/csi-pvc-backupper
    Operation ID:               du-d6f7caab-cc41-48de-80a1-eaac9174fabe.9ec5bb33-f4f9-46df5e397
    Items to Update:
                           datauploads.velero.io openshift-adp/testcsi2-b78jb
    Phase:                 Completed
    Progress description:  Completed
    Created:               2023-11-08 14:28:31 +0530 IST
    Started:               2023-11-08 14:28:31 +0530 IST
    Updated:               2023-11-08 14:29:31 +0530 IST
Resource List:
  apps/v1/Deployment:
    - testcsi/mysql
  apps/v1/ReplicaSet:
    - testcsi/mysql-66865fdf8c
  authorization.openshift.io/v1/RoleBinding:
    - testcsi/admin
    - testcsi/system:deployers
    - testcsi/system:image-builders
    - testcsi/system:image-pullers
  discovery.k8s.io/v1/EndpointSlice:
    - testcsi/mysql-dsbf5
  rbac.authorization.k8s.io/v1/RoleBinding:
    - testcsi/admin
    - testcsi/system:deployers
    - testcsi/system:image-builders
    - testcsi/system:image-pullers
  v1/ConfigMap:
    - testcsi/kube-root-ca.crt
    - testcsi/openshift-service-ca.crt
  v1/Endpoints:
    - testcsi/mysql
  v1/Event:
    - testcsi/mysql-1.17959a0431737f10
    - testcsi/mysql-1.17959a04605405fc
    - testcsi/mysql-1.17959a04605b883e
    - testcsi/mysql-1.17959a054a3fd5e0
    - testcsi/mysql-66865fdf8c-288wz.17959a054ed84662
    - testcsi/mysql-66865fdf8c-288wz.17959a067ef743cc
    - testcsi/mysql-66865fdf8c-288wz.17959a074a658787
    - testcsi/mysql-66865fdf8c-288wz.17959a07e7cabc00
    - testcsi/mysql-66865fdf8c-288wz.17959a07e9e45ca5
    - testcsi/mysql-66865fdf8c-288wz.17959a07f0da72e7
    - testcsi/mysql-66865fdf8c-288wz.17959a07f22a8344
    - testcsi/mysql-66865fdf8c.17959a045ed62335
    - testcsi/mysql.17959a0410075c37
    - testcsi/mysql.17959a045c8e52f3
    - testcsi/mysql.17959a045fc414bd
    - testcsi/mysql.17959a045fc92232
    - testcsi/mysql.17959a054b9a0e98
    - testcsi/velero-mysql-1-wrb2j.17959a1300d53522
    - testcsi/velero-mysql-1-wrb2j.17959a145a147ede
    - testcsi/velero-mysql-1-wrb2j.17959a17cc5b2946
    - testcsi/velero-mysql-jkscj.17959a11b30728e0
    - testcsi/velero-mysql-jkscj.17959a12d0ec78b9
    - testcsi/velero-mysql-jkscj.17959a1813b0e2a7
  v1/Namespace:
    - testcsi
  v1/PersistentVolume:
    - pvc-9ec5bb33-f4f9-46de-9eaf-642f5fa77b1c
    - pvc-c13148e4-5fe4-4009-b33c-752cd04c0931
  v1/PersistentVolumeClaim:
    - testcsi/mysql
    - testcsi/mysql-1
  v1/Pod:
    - testcsi/mysql-66865fdf8c-288wz
  v1/Secret:
    - testcsi/builder-dockercfg-2tr88
    - testcsi/builder-token-bhxsh
    - testcsi/default-dockercfg-n849k
    - testcsi/default-token-mjtzb
    - testcsi/deployer-dockercfg-72fgd
    - testcsi/deployer-token-2gdt5
    - testcsi/mysql
  v1/Service:
    - testcsi/mysql
  v1/ServiceAccount:
    - testcsi/builder
    - testcsi/default
    - testcsi/deployer

Velero-Native Snapshots: <none included>

Actual results:

Backup status does not have any details about the CSI snapshots performed. Velero describe does not have any backup item operations for the volume snapshots performed.

Backup does not show any details about CSI snapshots in case of DataMover

Details

Description

Description of problem:

When executing any backup with snapshotMoveData: true, the backup status / velero backup describe , does not report any details about the CSI snapshots.

1.3.0

Always

Actual results:

Backup status does not have any details about the CSI snapshots performed. Velero describe does not have any backup item operations for the volume snapshots performed.

All the details regrding volume snapshots should be present in backup status.

Attachments

Activity

People

Dates