Uploaded image for project: 'OpenShift API for Data Protection'
  1. OpenShift API for Data Protection
  2. OADP-6540

1.4: Unable to pull image from internal registry after OADP restore using namespace mapping

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • OADP 1.4.7
    • OADP 1.4.4
    • OADP
    • Quality / Stability / Reliability
    • 3
    • False
    • Hide

      None

      Show
      None
    • False
    • oadp-operator-bundle-container-1.5.1-23
    • ToDo
    • Important
    • Very Likely
    • 0
    • 8
    • None
    • Unset
    • Unknown
    • None

      ImageStream

      After restoring an entire application (namespace) from an OADP, I am unable to pull images from the internal OpenShift registry. The pods end up in the `ErrImagePull` / `ImagePullBackOff` state. The issue appears when the source namespace and target namespace do not have the same name (e.g. when using Restore.spec.namespaceMappings).

      Version-Release number of selected component (if applicable): OCP 4.17, OADP 1.4.4

      How reproducible: always

      Steps to Reproduce:
      1. Create a namespace with an ImageStream inside
      2. Create a Depployment based on that ImageStream
      3. Take a backup of the namespace
      4. Restore the namespace with a different name (-> Restore.spec.namespaceMappings)

      Actual results: Pods in the restored namespace fail to pull from the internal image registry.

      Failed to pull image "image-registry.openshift-image-registry.svc:5000/test-jack-restore/dtaagent@sha256:9842e45b237735d508e81e15d98aeaa80e1e9863aaeaf84688e48caa4e92731d": reading manifest sha256:9842e45b237735d508e81e15d98aeaa80e1e9863aaeaf84688e48caa4e92731d in image-registry.openshift-image-registry.svc:5000/test-jack-restore/dtaagent: authentication required

      Expected results:

      Applications resume fine after being restored into another namespace.

      Additional info:

      The failure seems to be caused by the fact that OADP / Velero restores the `system:image-puller` role binding (instead of reyling on the one that gets automatically created by OpenShift).

      This role binding still references the old namespace (notice the mismatch between metadata.namespace and subjects[0].name):

      apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        annotations:
          openshift.io/description: Allows all pods in this namespace to pull images from
            this namespace.  It is auto-managed by a controller; remove subjects to disable.
        creationTimestamp: "2025-05-30T10:09:10Z"
        labels:
          velero.io/backup-name: test-backup-3
          velero.io/restore-name: test-restore-4
        name: system:image-pullers
        namespace: test-jack-restore
        resourceVersion: "25908046"
        uid: 7e7fd7d5-558f-48dd-9ca5-3ea7318ff5be
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: system:image-puller
      subjects:
      - apiGroup: rbac.authorization.k8s.io
        kind: Group
        name: system:serviceaccounts:test-jack 

      For extended troubleshooting and additional details, please refer to:

      https://redhat-internal.slack.com/archives/C0144ECKUJ0/p1748434634981579

      https://redhat-internal.slack.com/archives/C013VBYBJQH/p1748594020414759

              tkaovila@redhat.com Tiger Kaovilai
              rh-ee-jhensche Jack Henschel
              Md Nadeem Md Nadeem
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: