Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-4231

Backup w/ no pod causes: fail to patch dynamic PV, err: context deadline exceeded

XMLWordPrintable

    • 4
    • False
    • Hide

      None

      Show
      None
    • False
    • oadp-operator-bundle-container-1.4.1-8
    • ToDo
    • 0
    • 0.000
    • Very Likely
    • 0
    • None
    • Unset
    • Unknown
    • No

      Description of problem:

      1. This is simply a clean write up and simple recreate for the bug Amos found in https://issues.redhat.com/browse/OADP-4230
      2.  Environment: OCP 4.16  OADP-1.4 devel install
      3. setup any sample app from https://github.com/openshift/oadp-operator/tree/master/tests/e2e/sample-applications.  In this case I used https://github.com/openshift/oadp-operator/blob/master/tests/e2e/sample-applications/minimal-8csivol/minimal-3csivol.yaml
      4. Ensure the app is running normally
      5.  DPA

       

      1. Version: oadp.openshift.io/v1alpha1
        kind: DataProtectionApplication
        metadata:
          name: dpa-sample
          namespace: openshift-adp
        spec:
          backupLocations:
          - velero:
              config:
                profile: default
                region: us-west-2
              credential:
                key: cloud
                name: cloud-credentials
              default: true
              objectStorage:
                bucket: cvpbucketuswest2
                prefix: velero
              provider: aws
          configuration:
            nodeAgent:
              enable: true
              uploaderType: kopia
            velero:
              defaultPlugins:
              - csi
              - openshift
              - aws

       

      In the deployment config set the replicaset settings to 0

       32 apiVersion: apps/v1
       33 kind: Deployment
       34 metadata:
       35   name: minimal-3csivol
       36   namespace: minimal-3csivol
       37 spec: 
       38   replicas: 1  --> 0
      
      There are additional replica settings in status, update those to 0 as well
      pps/minimal-3csivol -n minimal-3csivol -o yaml | grep -B 5 -A 5 replica
        namespace: minimal-3csivol
        resourceVersion: "6891044"
        uid: 4bcdd8d0-6788-4ef9-8004-0dd7fe257e86
      spec:
        progressDeadlineSeconds: 600
        replicas: 1
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app: minimal-3csivol
        strategy:
      --
          reason: NewReplicaSetAvailable
          status: "True"
          type: Progressing
        observedGeneration: 1
        readyReplicas: 1
        replicas: 1
        updatedReplicas: 1

       

      The number of pods running should move from 1/1 to 0/1

      Notice that the pvc's are all still present.  

       

      oc get all -n minimal-3csivol
      Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
      NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
      deployment.apps/minimal-3csivol   0/0     0            0           3m1s
      NAME                                         DESIRED   CURRENT   READY   AGE
      replicaset.apps/minimal-3csivol-7b779fbd4d   0         0         0       3m2s
      

       

      whayutin@thinkdoe:~/OPENSHIFT/git/OADP/oadp-operator$ oc get pvc -n minimal-3csivol
      NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
      volume1   Bound    pvc-9fdb9241-86db-4e2f-bb8f-76c4ab2ee763   1Gi        RWO            gp3-csi        3m49s
      volume2   Bound    pvc-2853b5c5-2767-4f0f-9284-291d1a2b9329   1Gi        RWO            gp3-csi        3m49s
      volume3   Bound    pvc-4bb128f8-4ced-41dc-99a5-83066597f703   1Gi        RWO            gp3-csi        3m49s
      whayutin@thinkdoe:~/OPENSHIFT/git/OADP/oadp-operator

       

      Execute the backup:

       

      1. 

      velero backup create no-pod1 --include-namespaces minimal-3csivol  --snapshot-volumes=true

      pod backup description:

       

      Name:         no-pod1
      Namespace:    openshift-adp
      Labels:       velero.io/storage-location=dpa-sample-1
      Annotations:  velero.io/resource-timeout=10m0s
                    velero.io/source-cluster-k8s-gitversion=v1.29.5+87992f4
                    velero.io/source-cluster-k8s-major-version=1
                    velero.io/source-cluster-k8s-minor-version=29
      Phase:  Completed
      
      Namespaces:
        Included:  minimal-3csivol
        Excluded:  <none>
      Resources:
        Included:        *
        Excluded:        <none>
        Cluster-scoped:  auto
      Label selector:  <none>
      Or label selector:  <none>
      Storage Location:  dpa-sample-1
      Velero-Native Snapshot PVs:  true
      Snapshot Move Data:          false
      Data Mover:                  velero
      TTL:  720h0m0s
      CSISnapshotTimeout:    10m0s
      ItemOperationTimeout:  4h0m0s
      Hooks:  <none>
      Backup Format Version:  1.1.0
      Started:    2024-06-05 22:25:08 +0000 UTC
      Completed:  2024-06-05 22:26:02 +0000 UTC
      Expiration:  2024-07-05 22:25:08 +0000 UTC
      Total items to be backed up:  67
      Items backed up:              67
      Backup Item Operations:
        Operation for volumesnapshots.snapshot.storage.k8s.io minimal-3csivol/velero-volume1-t5hn4:
          Backup Item Action Plugin:  velero.io/csi-volumesnapshot-backupper
          Operation ID:               minimal-3csivol/velero-volume1-t5hn4/2024-06-05T22:25:14Z
          Items to Update:
                    volumesnapshots.snapshot.storage.k8s.io minimal-3csivol/velero-volume1-t5hn4
                    volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-824ebaba-35b0-4dd9-883d-75393e25940d
          Phase:    Completed
          Created:  2024-06-05 22:25:14 +0000 UTC
          Started:  2024-06-05 22:25:14 +0000 UTC
          Updated:  2024-06-05 22:25:38 +0000 UTC
        Operation for volumesnapshots.snapshot.storage.k8s.io minimal-3csivol/velero-volume2-rq5s9:
          Backup Item Action Plugin:  velero.io/csi-volumesnapshot-backupper
          Operation ID:               minimal-3csivol/velero-volume2-rq5s9/2024-06-05T22:25:19Z
          Items to Update:
                    volumesnapshots.snapshot.storage.k8s.io minimal-3csivol/velero-volume2-rq5s9
                    volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-36493369-50da-476c-9f91-d068baad3531
          Phase:    Completed
          Created:  2024-06-05 22:25:19 +0000 UTC
          Started:  2024-06-05 22:25:19 +0000 UTC
          Updated:  2024-06-05 22:25:48 +0000 UTC
        Operation for volumesnapshots.snapshot.storage.k8s.io minimal-3csivol/velero-volume3-5rb56:
          Backup Item Action Plugin:  velero.io/csi-volumesnapshot-backupper
          Operation ID:               minimal-3csivol/velero-volume3-5rb56/2024-06-05T22:25:24Z
          Items to Update:
                    volumesnapshots.snapshot.storage.k8s.io minimal-3csivol/velero-volume3-5rb56
                    volumesnapshotcontents.snapshot.storage.k8s.io /snapcontent-92af2390-9b2b-45b5-8127-d8e426aa62bb
          Phase:    Completed
          Created:  2024-06-05 22:25:24 +0000 UTC
          Started:  2024-06-05 22:25:24 +0000 UTC
          Updated:  2024-06-05 22:25:58 +0000 UTC
      Resource List:
        apps/v1/Deployment:
          - minimal-3csivol/minimal-3csivol
        apps/v1/ReplicaSet:
          - minimal-3csivol/minimal-3csivol-7b779fbd4d
        authorization.openshift.io/v1/RoleBinding:
          - minimal-3csivol/system:deployers
          - minimal-3csivol/system:image-builders
          - minimal-3csivol/system:image-pullers
        rbac.authorization.k8s.io/v1/RoleBinding:
          - minimal-3csivol/system:deployers
          - minimal-3csivol/system:image-builders
          - minimal-3csivol/system:image-pullers
        snapshot.storage.k8s.io/v1/VolumeSnapshot:
          - minimal-3csivol/velero-volume1-t5hn4
          - minimal-3csivol/velero-volume2-rq5s9
          - minimal-3csivol/velero-volume3-5rb56
        snapshot.storage.k8s.io/v1/VolumeSnapshotClass:
          - csi-aws-vsc
        snapshot.storage.k8s.io/v1/VolumeSnapshotContent:
          - snapcontent-36493369-50da-476c-9f91-d068baad3531
          - snapcontent-824ebaba-35b0-4dd9-883d-75393e25940d
          - snapcontent-92af2390-9b2b-45b5-8127-d8e426aa62bb
        v1/ConfigMap:
          - minimal-3csivol/kube-root-ca.crt
          - minimal-3csivol/openshift-service-ca.crt
        v1/Event:
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba11d6b2bb5
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba29cf81646
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba324780a75
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba32802416f
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba349a18004
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba473b91b81
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba47781be71
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba4edb8081b
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba512db0186
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63ba513c9f20d
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-pjncc.17d63bcdd5de88a8
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-plfxx.17d63bcdd89f25d6
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-plfxx.17d63bcdf06aeee4
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-plfxx.17d63bcdf2097da1
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-plfxx.17d63bce1f6edaca
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-plfxx.17d63bce3d1bb7b7
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-plfxx.17d63bce3e07b9af
          - minimal-3csivol/minimal-3csivol-7b779fbd4d-plfxx.17d63bddfb1d5c4b
          - minimal-3csivol/minimal-3csivol-7b779fbd4d.17d63ba11da3db54
          - minimal-3csivol/minimal-3csivol-7b779fbd4d.17d63bcdd5dfc0d3
          - minimal-3csivol/minimal-3csivol-7b779fbd4d.17d63bcdd81123bb
          - minimal-3csivol/minimal-3csivol-7b779fbd4d.17d63bddfb11fcb2
          - minimal-3csivol/minimal-3csivol.17d63ba11b9c3faf
          - minimal-3csivol/minimal-3csivol.17d63bcdd5e9ea46
          - minimal-3csivol/minimal-3csivol.17d63bddfa340e83
          - minimal-3csivol/volume1.17d63ba11d5be64b
          - minimal-3csivol/volume1.17d63ba172060b57
          - minimal-3csivol/volume1.17d63ba1720e25b8
          - minimal-3csivol/volume1.17d63ba274a90679
          - minimal-3csivol/volume2.17d63ba11fc61fa1
          - minimal-3csivol/volume2.17d63ba1727f25a6
          - minimal-3csivol/volume2.17d63ba172882242
          - minimal-3csivol/volume2.17d63ba27496fc2f
          - minimal-3csivol/volume3.17d63ba122511355
          - minimal-3csivol/volume3.17d63ba172eb531c
          - minimal-3csivol/volume3.17d63ba172eff1da
          - minimal-3csivol/volume3.17d63ba274b5723d
        v1/Namespace:
          - minimal-3csivol
        v1/PersistentVolume:
          - pvc-685c140e-e654-4ce1-9d5c-dcfb6d5bb54a
          - pvc-80e98020-fc44-4326-a80e-ed5890b4c6a8
          - pvc-f5ae8bae-df65-4ffe-82c4-b0771445815f
        v1/PersistentVolumeClaim:
          - minimal-3csivol/volume1
          - minimal-3csivol/volume2
          - minimal-3csivol/volume3
        v1/Secret:
          - minimal-3csivol/builder-dockercfg-826rv
          - minimal-3csivol/default-dockercfg-d22jl
          - minimal-3csivol/deployer-dockercfg-bl4n5
        v1/ServiceAccount:
          - minimal-3csivol/builder
          - minimal-3csivol/default
          - minimal-3csivol/deployer
      Backup Volumes:
        Velero-Native Snapshots: <none included>
        CSI Snapshots:
          minimal-3csivol/volume1:
            Snapshot:
              Operation ID: minimal-3csivol/velero-volume1-t5hn4/2024-06-05T22:25:14Z
              Snapshot Content Name: snapcontent-824ebaba-35b0-4dd9-883d-75393e25940d
              Storage Snapshot ID: snap-05256c003a5db0bb3
              Snapshot Size (bytes): 1073741824
              CSI Driver: ebs.csi.aws.com
          minimal-3csivol/volume2:
            Snapshot:
              Operation ID: minimal-3csivol/velero-volume2-rq5s9/2024-06-05T22:25:19Z
              Snapshot Content Name: snapcontent-36493369-50da-476c-9f91-d068baad3531
              Storage Snapshot ID: snap-0a80ce7b9cb6f513b
              Snapshot Size (bytes): 1073741824
              CSI Driver: ebs.csi.aws.com
          minimal-3csivol/volume3:
            Snapshot:
              Operation ID: minimal-3csivol/velero-volume3-5rb56/2024-06-05T22:25:24Z
              Snapshot Content Name: snapcontent-92af2390-9b2b-45b5-8127-d8e426aa62bb
              Storage Snapshot ID: snap-0a99c20e93d52e57d
              Snapshot Size (bytes): 1073741824
              CSI Driver: ebs.csi.aws.com
        Pod Volume Backups: <none included>
      HooksAttempted:  0
      HooksFailed:     0
      

      The backup completes successfully.

      2.  Delete the app namespace

      oc delete namespace minimal-3csivol

       

      3. Restore the backup

      velero restore create no-pod1  --from-backup no-pod1

       

      4.  The restore will fail w/:

       

      Errors:
        Velero:     <none>
        Cluster:    <none>
        Namespaces:
          minimal-3csivol:  fail to patch dynamic PV, err: context deadline exceeded, PVC: volume2, PV: pvc-80e98020-fc44-4326-a80e-ed5890b4c6a8
                            fail to patch dynamic PV, err: context deadline exceeded, PVC: volume3, PV: pvc-685c140e-e654-4ce1-9d5c-dcfb6d5bb54a
                            fail to patch dynamic PV, err: context deadline exceeded, PVC: volume1, PV: pvc-f5ae8bae-df65-4ffe-82c4-b0771445815f
       
      

      Full Details of restore:

       

      Name:         no-pod1
      Namespace:    openshift-adp
      Labels:       <none>
      Annotations:  <none>
      Phase:                       PartiallyFailed (run 'velero restore logs no-pod1' for more information)
      Total items to be restored:  30
      Items restored:              30
      Started:    2024-06-05 22:28:17 +0000 UTC
      Completed:  2024-06-05 22:38:19 +0000 UTC
      Warnings:
        Velero:     <none>
        Cluster:  could not restore, VolumeSnapshotContent "snapcontent-36493369-50da-476c-9f91-d068baad3531" already exists. Warning: the in-cluster version is different than the backed-up version
                  could not restore, VolumeSnapshotContent "snapcontent-824ebaba-35b0-4dd9-883d-75393e25940d" already exists. Warning: the in-cluster version is different than the backed-up version
                  could not restore, VolumeSnapshotContent "snapcontent-92af2390-9b2b-45b5-8127-d8e426aa62bb" already exists. Warning: the in-cluster version is different than the backed-up version
        Namespaces:
          minimal-3csivol:  could not restore, ConfigMap "kube-root-ca.crt" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, ConfigMap "openshift-service-ca.crt" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, RoleBinding "system:deployers" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, RoleBinding "system:image-builders" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, RoleBinding "system:image-pullers" already exists. Warning: the in-cluster version is different than the backed-up version
      Errors:
        Velero:     <none>
        Cluster:    <none>
        Namespaces:
          minimal-3csivol:  fail to patch dynamic PV, err: context deadline exceeded, PVC: volume2, PV: pvc-80e98020-fc44-4326-a80e-ed5890b4c6a8
                            fail to patch dynamic PV, err: context deadline exceeded, PVC: volume3, PV: pvc-685c140e-e654-4ce1-9d5c-dcfb6d5bb54a
                            fail to patch dynamic PV, err: context deadline exceeded, PVC: volume1, PV: pvc-f5ae8bae-df65-4ffe-82c4-b0771445815f
      Backup:  no-pod1
      Namespaces:
        Included:  all namespaces found in the backup
        Excluded:  <none>
      Resources:
        Included:        *
        Excluded:        nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io, csinodes.storage.k8s.io, volumeattachments.storage.k8s.io, backuprepositories.velero.io
        Cluster-scoped:  auto
      Namespace mappings:  <none>
      Label selector:  <none>
      Or label selector:  <none>
      Restore PVs:  auto
      CSI Snapshot Restores: <none included>
      Existing Resource Policy:   <none>
      ItemOperationTimeout:       4h0m0s
      Preserve Service NodePorts:  auto
      Uploader config:
      
      HooksAttempted:   0
      HooksFailed:      0
      Resource List:
        apps/v1/Deployment:
          - minimal-3csivol/minimal-3csivol(created)
        apps/v1/ReplicaSet:
          - minimal-3csivol/minimal-3csivol-7b779fbd4d(created)
        authorization.openshift.io/v1/RoleBinding:
          - minimal-3csivol/system:deployers(failed)
          - minimal-3csivol/system:image-builders(failed)
          - minimal-3csivol/system:image-pullers(failed)
        rbac.authorization.k8s.io/v1/RoleBinding:
          - minimal-3csivol/system:deployers(created)
          - minimal-3csivol/system:image-builders(created)
          - minimal-3csivol/system:image-pullers(created)
        snapshot.storage.k8s.io/v1/VolumeSnapshot:
          - minimal-3csivol/velero-volume1-t5hn4(created)
          - minimal-3csivol/velero-volume2-rq5s9(created)
          - minimal-3csivol/velero-volume3-5rb56(created)
        snapshot.storage.k8s.io/v1/VolumeSnapshotClass:
          - csi-aws-vsc(skipped)
        snapshot.storage.k8s.io/v1/VolumeSnapshotContent:
          - snapcontent-36493369-50da-476c-9f91-d068baad3531(failed)
          - snapcontent-824ebaba-35b0-4dd9-883d-75393e25940d(failed)
          - snapcontent-92af2390-9b2b-45b5-8127-d8e426aa62bb(failed)
        v1/ConfigMap:
          - minimal-3csivol/kube-root-ca.crt(failed)
          - minimal-3csivol/openshift-service-ca.crt(failed)
        v1/Namespace:
          - minimal-3csivol(created)
        v1/PersistentVolume:
          - pvc-685c140e-e654-4ce1-9d5c-dcfb6d5bb54a(skipped)
          - pvc-80e98020-fc44-4326-a80e-ed5890b4c6a8(skipped)
          - pvc-f5ae8bae-df65-4ffe-82c4-b0771445815f(skipped)
        v1/PersistentVolumeClaim:
          - minimal-3csivol/volume1(created)
          - minimal-3csivol/volume2(created)
          - minimal-3csivol/volume3(created)
        v1/Secret:
          - minimal-3csivol/builder-dockercfg-826rv(created)
          - minimal-3csivol/default-dockercfg-d22jl(created)
          - minimal-3csivol/deployer-dockercfg-bl4n5(created)
        v1/ServiceAccount:
          - minimal-3csivol/builder(skipped)
          - minimal-3csivol/default(skipped)
          - minimal-3csivol/deployer(skipped)
      

       

      Recreates on OCP 4.15 w/ OADP-1.4

      Errors:
        Velero:     <none>
        Cluster:    <none>
        Namespaces:
          minimal-3csivol:  fail to patch dynamic PV, err: context deadline exceeded, PVC: volume1, PV: pvc-93a18e81-83a9-4f27-9e54-88e1364a56aa
                            fail to patch dynamic PV, err: context deadline exceeded, PVC: volume2, PV: pvc-879908a2-3e03-4972-af91-7d64fff40fef
                            fail to patch dynamic PV, err: context deadline exceeded, PVC: volume3, PV: pvc-5753872d-e2dd-4da6-a07c-23cc30f13d51

       

      On OCP 4.15 w/ OADP-1.3.1 , Velero v.1.12.3

      The backup completes, but the PVC's are never bound.

       

      NAME      BACKUP    STATUS    STARTED                         COMPLETED                       ERRORS   WARNINGS   CREATED
                           SELECTOR
      no-pod1   no-pod1   Completed   2024-06-05 23:35:38 +0000 UTC   2024-06-05 23:35:41 +0000 UTC   0        8          2024-06-05
      23:35:38 +0000 UTC   <none>
      no-pod2   no-pod1   Completed   2024-06-05 23:37:09 +0000 UTC   2024-06-05 23:37:12 +0000 UTC   0        8          2024-06-05
      23:37:09 +0000 UTC   <none>

       

       

       

      whayutin@thinkdoe:~/OPENSHIFT/git/OADP/oadp-operator$ remote_velero describe restore no-pod1 --details 
      Name:         no-pod1
      Namespace:    openshift-adp
      Labels:       <none>
      Annotations:  <none>
      Phase:                       Completed
      Total items to be restored:  33
      Items restored:              33
      Started:    2024-06-05 23:35:38 +0000 UTC
      Completed:  2024-06-05 23:35:41 +0000 UTC
      Warnings:
        Velero:     <none>
        Cluster:  could not restore, VolumeSnapshotContent "snapcontent-350013bb-5bf3-4072-82dd-7ffd27668c4a" already exists. Warning: the in-cluster version is different than the backed-up version
                  could not restore, VolumeSnapshotContent "snapcontent-86523191-4762-492e-a92e-e2e3e85898b1" already exists. Warning: the in-cluster version is different than the backed-up version
                  could not restore, VolumeSnapshotContent "snapcontent-fe61819d-e7a2-4455-bf44-7943b7f2c1f1" already exists. Warning: the in-cluster version is different than the backed-up version
        Namespaces:
          minimal-3csivol:  could not restore, ConfigMap "kube-root-ca.crt" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, ConfigMap "openshift-service-ca.crt" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, RoleBinding "system:deployers" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, RoleBinding "system:image-builders" already exists. Warning: the in-cluster version is different than the backed-up version
                            could not restore, RoleBinding "system:image-pullers" already exists. Warning: the in-cluster version is different than the backed-up version
      Backup:  no-pod1
      Namespaces:
        Included:  all namespaces found in the backup
        Excluded:  <none>
      Resources:
        Included:        *
        Excluded:        nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io, csinodes.storage.k8s.io, volumeattachments.storage.k8s.io, backuprepositories.velero.io
        Cluster-scoped:  auto
      Namespace mappings:  <none>
      Label selector:  <none>
      Or label selector:  <none>
      Restore PVs:  auto
      Existing Resource Policy:   <none>
      ItemOperationTimeout:       4h0m0s
      Preserve Service NodePorts:  auto
      Resource List:
        apps/v1/Deployment:
          - minimal-3csivol/minimal-3csivol(created)
        apps/v1/ReplicaSet:
          - minimal-3csivol/minimal-3csivol-7b779fbd4d(created)
        authorization.openshift.io/v1/RoleBinding:
          - minimal-3csivol/system:deployers(failed)
          - minimal-3csivol/system:image-builders(failed)
          - minimal-3csivol/system:image-pullers(failed)
        rbac.authorization.k8s.io/v1/RoleBinding:
          - minimal-3csivol/system:deployers(created)
          - minimal-3csivol/system:image-builders(created)
          - minimal-3csivol/system:image-pullers(created)
        snapshot.storage.k8s.io/v1/VolumeSnapshot:
          - minimal-3csivol/velero-volume1-dqw68(created)
          - minimal-3csivol/velero-volume2-dz5j6(created)
          - minimal-3csivol/velero-volume3-k45p8(created)
        snapshot.storage.k8s.io/v1/VolumeSnapshotClass:
          - csi-aws-vsc(skipped)
        snapshot.storage.k8s.io/v1/VolumeSnapshotContent:
          - snapcontent-350013bb-5bf3-4072-82dd-7ffd27668c4a(failed)
          - snapcontent-86523191-4762-492e-a92e-e2e3e85898b1(failed)
          - snapcontent-fe61819d-e7a2-4455-bf44-7943b7f2c1f1(failed)
        v1/ConfigMap:
          - minimal-3csivol/kube-root-ca.crt(failed)
          - minimal-3csivol/openshift-service-ca.crt(failed)
        v1/Namespace:
          - minimal-3csivol(created)
        v1/PersistentVolume:
          - pvc-2853b5c5-2767-4f0f-9284-291d1a2b9329(skipped)
          - pvc-4bb128f8-4ced-41dc-99a5-83066597f703(skipped)
          - pvc-9fdb9241-86db-4e2f-bb8f-76c4ab2ee763(skipped)
        v1/PersistentVolumeClaim:
          - minimal-3csivol/volume1(created)
          - minimal-3csivol/volume2(created)
          - minimal-3csivol/volume3(created)
        v1/Secret:
          - minimal-3csivol/builder-dockercfg-jx9c7(created)
          - minimal-3csivol/builder-token-t6jm2(skipped)
          - minimal-3csivol/default-dockercfg-zcsrx(created)
          - minimal-3csivol/default-token-t47rk(skipped)
          - minimal-3csivol/deployer-dockercfg-bx7m5(created)
          - minimal-3csivol/deployer-token-vqc2b(skipped)
        v1/ServiceAccount:
          - minimal-3csivol/builder(skipped)
          - minimal-3csivol/default(skipped)
          - minimal-3csivol/deployer(skipped)
      

       

      This is a regression in Velero behavior:

      Likely Cause is https://github.com/vmware-tanzu/velero/pull/7504

            spampatt@redhat.com Shubham Pampattiwar
            amastbau Amos Mastbaum
            Amos Mastbaum Amos Mastbaum
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: