Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-5460 Datamover Restore fails when OADP performing restore is installed in a different namespace
  3. OADP-5505

[RedHat QE] Verify Bug OADP-5460 - Datamover Restore fails when OADP performing restore is installed in a different namespace

XMLWordPrintable

    • Icon: Sub-task Sub-task
    • Resolution: Done
    • Icon: Undefined Undefined
    • OADP 1.4.2
    • None
    • QE-Task
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • ToDo
    • 0
    • 0
    • Very Likely
    • 0
    • None
    • Unset
    • Unknown

      Description of problem:

      When we perform a datamover backup of any workload (lets say, in namespace W) using OADP installed in namespace A (lets say), but restore the workload from an OADP installed in the namespace B (lets say), The restore fails.

      Version-Release number of selected component (if applicable):

      OADP 1.4.2

      How reproducible:

      Always

      Steps to Reproduce:
      1. Install OADP in namespace openshift-adp
      2. Install OADP in namespace oadp2
      3. Deploy workload in namespace mysql-test
      4. Perform backup using OADP in openshift-adp
      5. Perform restore using OADP in oadp2

      Actual results:

      Restore results in Partially Failed.

      Expected results:

      Restore should be successfull.

      Additional info:

      The Application to Backup / Restore

      $ oc get all -n mysql-test
      Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
      NAME                         READY   STATUS    RESTARTS   AGE
      pod/mysql-5f7547b97f-xxtz6   1/1     Running   0          3m36s
      
      NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
      service/mysql   ClusterIP   172.30.39.13   <none>        3306/TCP   3m39s
      
      NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
      deployment.apps/mysql   1/1     1            1           3m37s
      
      NAME                               DESIRED   CURRENT   READY   AGE
      replicaset.apps/mysql-5f7547b97f   1         1         1       3m38s
      

      OADP installed in openshift-adp with DPA created.

      $ oc get pods -n openshift-adp
      NAME                                                READY   STATUS    RESTARTS   AGE
      node-agent-4qp9r                                    1/1     Running   0          12s
      node-agent-dlgmz                                    1/1     Running   0          12s
      node-agent-vk829                                    1/1     Running   0          12s
      openshift-adp-controller-manager-5f9c4dbdf4-f6dj2   1/1     Running   0          3h37m
      velero-5dcbc86f75-7sgjq                             1/1     Running   0          12s
      

      DPA :

      apiVersion: v1
      items:
      - apiVersion: oadp.openshift.io/v1alpha1
        kind: DataProtectionApplication
        metadata:
          creationTimestamp: "2025-01-09T09:58:05Z"
          generation: 1
          name: ts-dpa
          namespace: openshift-adp
          resourceVersion: "135714"
          uid: 38b93cc2-78ec-4666-931e-630098dd60a7
        spec:
          backupLocations:
          - velero:
              credential:
                key: cloud
                name: cloud-credentials-gcp
              default: true
              objectStorage:
                bucket: oadp107141nr57r
                prefix: velero
              provider: gcp
          configuration:
            nodeAgent:
              enable: true
              uploaderType: kopia
            velero:
              defaultPlugins:
              - csi
              - gcp
              - openshift
        status:
          conditions:
          - lastTransitionTime: "2025-01-09T09:58:05Z"
            message: Reconcile complete
            reason: Complete
            status: "True"
            type: Reconciled
      kind: List
      metadata:
        resourceVersion: ""
      

      Backup CR

      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        annotations:
          velero.io/resource-timeout: 10m0s
          velero.io/source-cluster-k8s-gitversion: v1.30.7
          velero.io/source-cluster-k8s-major-version: "1"
          velero.io/source-cluster-k8s-minor-version: "30"
        creationTimestamp: "2025-01-09T10:16:55Z"
        generation: 8
        labels:
          velero.io/storage-location: ts-dpa-1
        name: backup1
        namespace: openshift-adp
        resourceVersion: "144209"
        uid: e760c716-7851-4cc3-9316-75fc3a6288b4
      spec:
        csiSnapshotTimeout: 10m0s
        defaultVolumesToFsBackup: false
        includedNamespaces:
        - mysql-test
        itemOperationTimeout: 0h3m0s
        snapshotMoveData: true
        storageLocation: ts-dpa-1
        ttl: 720h0m0s
      status:
        backupItemOperationsAttempted: 2
        backupItemOperationsCompleted: 2
        completionTimestamp: "2025-01-09T10:18:50Z"
        expiration: "2025-02-08T10:16:55Z"
        formatVersion: 1.1.0
        hookStatus: {}
        phase: Completed
        progress:
          itemsBackedUp: 45
          totalItems: 45
        startTimestamp: "2025-01-09T10:16:56Z"
        version: 1
      

      Restore CR in oadp2 namespace:

      $ oc get restore restore1 -o yaml
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        creationTimestamp: "2025-01-09T10:25:30Z"
        finalizers:
        - restores.velero.io/external-resources-finalizer
        generation: 7
        name: restore1
        namespace: oadp2
        resourceVersion: "146609"
        uid: 3ce81300-46c1-4806-902e-79fc6318a3f0
      spec:
        backupName: backup1
        excludedResources:
        - nodes
        - events
        - events.events.k8s.io
        - backups.velero.io
        - restores.velero.io
        - resticrepositories.velero.io
        - csinodes.storage.k8s.io
        - volumeattachments.storage.k8s.io
        - backuprepositories.velero.io
        itemOperationTimeout: 1h0m0s
      status:
        completionTimestamp: "2025-01-09T10:25:33Z"
        errors: 2
        hookStatus: {}
        phase: PartiallyFailed
        progress:
          itemsRestored: 30
          totalItems: 30
        startTimestamp: "2025-01-09T10:25:30Z"
        warnings: 6
      

      Restore CR (With Namespace Mapping) in oadp2:

      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        creationTimestamp: "2025-01-09T10:28:01Z"
        finalizers:
        - restores.velero.io/external-resources-finalizer
        generation: 7
        name: restore2
        namespace: oadp2
        resourceVersion: "147521"
        uid: c12a9323-6e99-44d7-b59e-7cb01065b35a
      spec:
        backupName: backup1
        excludedResources:
        - nodes
        - events
        - events.events.k8s.io
        - backups.velero.io
        - restores.velero.io
        - resticrepositories.velero.io
        - csinodes.storage.k8s.io
        - volumeattachments.storage.k8s.io
        - backuprepositories.velero.io
        itemOperationTimeout: 1h0m0s
        namespaceMapping:
          openshift-adp: oadp2
      status:
        completionTimestamp: "2025-01-09T10:28:04Z"
        errors: 2
        hookStatus: {}
        phase: PartiallyFailed
        progress:
          itemsRestored: 30
          totalItems: 30
        startTimestamp: "2025-01-09T10:28:01Z"
        warnings: 6
      

      Describe restore output:

       velero describe restore restore1 -n oadp2 --details -n oadp2
      Name:         restore1
      Namespace:    oadp2
      Labels:       <none>
      Annotations:  <none>
      
      Phase:                       PartiallyFailed (run 'velero restore logs restore1' for more information)
      Total items to be restored:  30
      Items restored:              30
      
      Started:    2025-01-09 15:55:30 +0530 IST
      Completed:  2025-01-09 15:55:33 +0530 IST
      
      Warnings:
        Velero:     <none>
        Cluster:    <none>
        Namespaces:
          mysql-test:  could not restore, ConfigMap "kube-root-ca.crt" already exists. Warning: the in-cluster version is different than the backed-up version
                       could not restore, ConfigMap "openshift-service-ca.crt" already exists. Warning: the in-cluster version is different than the backed-up version
                       could not restore, RoleBinding "admin" already exists. Warning: the in-cluster version is different than the backed-up version
                       could not restore, RoleBinding "system:deployers" already exists. Warning: the in-cluster version is different than the backed-up version
                       could not restore, RoleBinding "system:image-builders" already exists. Warning: the in-cluster version is different than the backed-up version
                       could not restore, RoleBinding "system:image-pullers" already exists. Warning: the in-cluster version is different than the backed-up version
      
      Errors:
        Velero:     <none>
        Cluster:    <none>
        Namespaces:
          mysql-test:  error preparing persistentvolumeclaims/mysql-test/mysql-data: rpc error: code = Unknown desc = fail get DataUploadResult for restore: restore1: no DataUpload result cm found with labels velero.io/pvc-namespace-name=mysql-test.mysql-data,velero.io/restore-uid=3ce81300-46c1-4806-902e-79fc6318a3f0,velero.io/resource-usage=DataUpload
                       error preparing persistentvolumeclaims/mysql-test/mysql-data1: rpc error: code = Unknown desc = fail get DataUploadResult for restore: restore1: no DataUpload result cm found with labels velero.io/pvc-namespace-name=mysql-test.mysql-data1,velero.io/restore-uid=3ce81300-46c1-4806-902e-79fc6318a3f0,velero.io/resource-usage=DataUpload
      
      Backup:  backup1
      
      Namespaces:
        Included:  all namespaces found in the backup
        Excluded:  <none>
      
      Resources:
        Included:        *
        Excluded:        nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io, csinodes.storage.k8s.io, volumeattachments.storage.k8s.io, backuprepositories.velero.io
        Cluster-scoped:  auto
      
      Namespace mappings:  <none>
      

       

              rhn-support-ssingla Sachin Singla
              akarol@redhat.com Aziza Karol
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: