Uploaded image for project: 'Multiple Architecture Enablement'
  1. Multiple Architecture Enablement
  2. MULTIARCH-5767

Investagate MTO not scheduling pods in prow

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • mto-1.2
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The multiarch-tuning-operator's image inspector incorrectly processes fully-qualified internal registry images (e.g., `image-registry.openshift-image-registry.svc:5000/namespace/image@sha256:...`) as short names, causing it to attempt pulling from external registries instead of the specified internal registry.

      Environment

      • OpenShift CI (Prow)
      • Cluster with multi-architecture support (amd64 + arm64)

      Expected Behavior

      The operator should:
      1. Recognize the image reference as fully-qualified (contains registry domain with port)
      2. Connect directly to `image-registry.openshift-image-registry.svc:5000`
      3. Successfully retrieve the manifest and inspect the image architectures
      4. Set appropriate node affinity based on the image's supported architectures

      Actual Behavior

      The operator attempts to pull the manifest from unqualified-search registries:

      Failed to retrieve the supported architectures: 2 errors occurred while pulling:

       * reading manifest sha256:d69e35f73b1e5c0bc2e12a95440a0ddb7976b5a69eb0b7c5a0772f0d954b1a57 in registry.access.redhat.com/image-registry.openshift-image-registry.svc: name unknown: Repo not found
       * reading manifest sha256:d69e35f73b1e5c0bc2e12a95440a0ddb7976b5a69eb0b7c5a0772f0d954b1a57 in docker.io/library/image-registry.openshift-image-registry.svc: requested access to the resource is denied
       message: {"level":"error","ts":"2025-11-04T18:57:29Z","logger":"manager","msg":"Unable to set the node affinity for the pod.","controller":"pod","controllerGroup":"","controllerKind":"Pod","Pod":{"name":"install-yamls-deploy-openstack-k8s-operators-deploy","namespace":"ci-op-gdrr450x"},"namespace":"ci-op-gdrr450x","name":"install-yamls-deploy-openstack-k8s-operators-deploy","reconcileID":"da3de0cd-16de-4d5c-af4c-a58f6dd0d126","error":"2 errors occurred while pulling:\n * reading manifest sha256:38064a6677adaaa4741ecbf070ece8137b3963a932f2828e5971ff04f69dfc31 in registry.access.redhat.com/image-registry.openshift-image-registry.svc: name unknown: Repo not found\n * reading manifest sha256:38064a6677adaaa4741ecbf070ece8137b3963a932f2828e5971ff04f69dfc31 in docker.io/library/image-registry.openshift-image-registry.svc: requested access to the resource is denied","stacktrace":"github.com/openshift/multiarch-tuning-operator/controllers/podplacement.(*Pod).handleError\n\t/workspace/controllers/podplacement/pod_model.go:387\ngithub.com/openshift/multiarch-tuning-operator/controllers/podplacement.(*PodReconciler).processPod\n\t/workspace/controllers/podplacement/pod_reconciler.go:130\ngithub.com/openshift/multiarch-tuning-operator/controllers/podplacement.(*PodReconciler).Reconcile\n\t/workspace/controllers/podplacement/pod_reconciler.go:87\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:334\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:294\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:255"} 

       

      This causes the pod to fail scheduling with `ImagePullBackOff`.

      references: https://redhat-internal.slack.com/archives/C06ECDGH4LE/p1762281830657099

      https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_osdc[…]/pull-ci-openshift-osdctl-master-build/1985749621086883840

       

       

       

              tzivkovi@redhat.com Tori Zivkovic
              tzivkovi@redhat.com Tori Zivkovic
              None
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: