Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-3598

Restore with OADP 1.3.1 is failing on OCP 4.15 when image registry is not running

    XMLWordPrintable

Details

    • False
    • Hide

      None

      Show
      None
    • False
    • oadp-operator-bundle-container-1.3.1-49
    • ToDo
    • No
    • Important
    • 8
    • 0
    • Very Likely
    • 0
    • None
    • Unset
    • Unknown

    Description

      Description of problem:

      Restore is partially failing on OCP 4.15 with OADP 1.3.1, verified this with build 35 which is the latest and with older builds as well.

      Velero logs contains following error.

       

      oc logs velero-57596cdc85-846lj -n openshift-adp  | grep error

      Defaulted container "velero" out of: velero, openshift-velero-plugin (init), velero-plugin-for-aws (init), velero-plugin-for-csi (init)

      time="2024-02-28T05:43:35Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/0/1)" controller=backup-storage-location logSource="/remote-source/velero/app/pkg/controller/backup_storage_location_controller.go:194"

      time="2024-02-28T06:38:53Z" level=warning msg="active indexes [] deletion watermark 0001-01-01 00:00:00 +0000 UTC" logModule=kopia/kopia/format logSource="/remote-source/velero/app/pkg/kopia/kopia_log.go:101" sublevel=error

      time="2024-02-28T06:38:54Z" level=warning msg="active indexes [] deletion watermark 0001-01-01 00:00:00 +0000 UTC" logModule=kopia/kopia/format logSource="/remote-source/velero/app/pkg/kopia/kopia_log.go:101" sublevel=error

      time="2024-02-28T06:38:57Z" level=warning msg="active indexes [xn0_f613e240834400bc22c5e0da10f3b8ec-sb03a425b07387896126-c1] deletion watermark 0001-01-01 00:00:00 +0000 UTC" logModule=kopia/kopia/format logSource="/remote-source/velero/app/pkg/kopia/kopia_log.go:101" sublevel=error

      time="2024-02-28T06:46:31Z" level=error msg="Namespace mysql-persistent, resource restore error: error preparing pods/mysql-persistent/mysql-6b49bd67c7-x4nwr: rpc error: code = Unknown desc = Secret is not getting created" logSource="/remote-source/velero/app/pkg/controller/restore_controller.go:567" restore=openshift-adp/restore-mysql

       

      However, the application restore completes and all the resources appears to be in good state.

      [root@m1320001 1.3]# oc get all -n mysql-persistent
      Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
      NAME                         READY   STATUS    RESTARTS   AGE
      pod/mysql-6b49bd67c7-x5kb7   1/1     Running   0          35m
      
      
      NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
      service/mysql   ClusterIP   172.30.187.202   <none>        3306/TCP   35m
      
      
      NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
      deployment.apps/mysql   1/1     1            1           35m
      
      
      NAME                               DESIRED   CURRENT   READY   AGE
      replicaset.apps/mysql-6b49bd67c7   1         1         1       35m
      [root@m1320001 1.3]# oc get secret -n mysql-persistent
      NAME    TYPE     DATA   AGE
      mysql   Opaque   4      42m
      [root@m1320001 1.3]#  

      Version-Release number of selected component (if applicable):

      OADP 1.3.1-35

      [root@m1320001 1.3]# oc version
      Client Version: 4.15.0-rc.7
      Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
      Server Version: 4.15.0-rc.7
      Kubernetes Version: v1.28.6+f1618d5 

       

      How reproducible:

      Always

      Steps to Reproduce:
      1. Deploy mysql app
      2. Take backup using FileSystemBackup (Kopia) 
      3. Restore it either on same cluster after deleting the mysql project or restore it on another cluster

      Actual results:

      oc get restore  -n openshift-adp restore-mysql -o jsonpath='{.status.phase}'

      PartiallyFailed

      Expected results:

      Restore should complete fine.

      Additional info:
      Dev notes:

      asking imageregistry in https://redhat-internal.slack.com/archives/C013VBYBJQH/p1709952439633439

      Attachments

        Activity

          People

            spampatt@redhat.com Shubham Pampattiwar
            manilson-1 Maya Anilson
            Prasad Joshi Prasad Joshi
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: