Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-1289

Restore partially fails with error "Secrets \"deployer-token-rrjqx\" not found"

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • oadp-velero-container-1.1.2-13
    • ToDo
    • 0
    • 0
    • Very Likely
    • 0
    • None
    • Unset
    • Unknown
    • Proposed
    • Yes

      Description of problem:

      We observed this failure in recent runs where restore partially fails with error "secrets \"deployer-token-rrjqx\" not found", before this build oadp-operator-bundle-container-1.1.2-19 test passed as usual. This issue it not consistently reproducible. Attached Report portal links below.

       

      time="2023-02-10T15:05:51Z" level=error msg="error patch for managed fields resource-quota/deployer-token-rrjqx: secrets \"deployer-token-rrjqx\" not found" logSource="/remote-source/velero/app/pkg/restore/restore.go:1429" restore=openshift-adp/resource-quota-e2e-349de3d4-a954-11ed-932b-845cf3eff33a
      time="2023-02-10T15:05:51Z" level=error msg="Namespace resource-quota, resource restore error: secrets \"deployer-token-rrjqx\" not found" logSource="/remote-source/velero/app/pkg/controller/restore_controller.go:510" restore=openshift-adp/resource-quota-e2e-349de3d4-a954-11ed-932b-845cf3eff33a

      [[1]|https://reportportal-migration-qe.apps.ocp-c1.prod.psi.redhat.com/ui/#oadp/launches/all/2749/99342/log?item0Params=filter.eq.hasStats%3Dtrue%26filter.eq.hasChildren%3Dfalse%26filter.in.issueType%3Dti001%252Cti_1ib0x8m5wom04%252Cti_t49j19sd0e9f%252Cti_qdychp4e8gmv%252Cti_skpj4qdaphep]

      https://reportportal-migration-qe.apps.ocp-c1.prod.psi.redhat.com/ui/#oadp/launches/all/2749/99300/log?item0Params=filter.eq.hasStats%3Dtrue%26filter.eq.hasChildren%3Dfalse%26filter.in.issueType%3Dti001%252Cti_1ib0x8m5wom04%252Cti_t49j19sd0e9f%252Cti_qdychp4e8gmv%252Cti_skpj4qdaphep

      Version-Release number of selected component (if applicable):
      OADP 1.1.2 (oadp-operator-bundle-container-1.1.2-19)

       

      How reproducible:
      Intermittent (was able to reproduce it after 5-6 restores)

       

      Steps to Reproduce:
      1. Create a backup with any application

      2. Try to restore same application multiple times until you hit this issue

      Polarion Case :- https://polarion.engineering.redhat.com/polarion/#/project/OADP/workitem?id=OADP-96

      Actual results: 

      Restore partially failed.

      time="2023-02-10T15:05:51Z" level=error msg="error patch for managed fields resource-quota/deployer-token-rrjqx: secrets \"deployer-token-rrjqx\" not found" logSource="/remote-source/velero/app/pkg/restore/restore.go:1429" restore=openshift-adp/resource-quota-e2e-349de3d4-a954-11ed-932b-845cf3eff33a
      time="2023-02-10T15:05:51Z" level=error msg="Namespace resource-quota, resource restore error: secrets \"deployer-token-rrjqx\" not found" logSource="/remote-source/velero/app/pkg/controller/restore_controller.go:510" restore=openshift-adp/resource-quota-e2e-349de3d4-a954-11ed-932b-845cf3eff33a

      Expected results:
      Restore should be successful.

      Additional info:

      $ oc get dpa -o yaml

      spec:
      backupLocations:
      
      velero:
      config:
      profile: default
      region: us-east-2
      credential:
      key: cloud
      name: cloud-credentials
      default: true
      objectStorage:
      bucket: oadpbucket176839
      prefix: velero-e2e-6076bd2a-a953-11ed-932b-845cf3eff33a
      provider: aws
      configuration:
      restic:
      enable: true
      podConfig:
      resourceAllocations: {}
      velero:
      defaultPlugins: openshift aws kubevirt
      podDnsConfig: {}
      snapshotLocations: []
      status:
      conditions: lastTransitionTime: "2023-02-10T15:04:24Z"
      message: Reconcile complete
      reason: Complete
      status: "True"
      type: Reconciled 

      $ oc get bsl

      NAME PHASE LAST VALIDATED AGE DEFAULT
      ts-dpa-1 Available 51s 32m true

      Velero logs:- 
      http://pastebin.test.redhat.com/1091385

              tkaovila@redhat.com Tiger Kaovilai
              rhn-support-prajoshi Prasad Joshi
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: