In disconnected environment, the Image Registry is configured for pullthrough and the ImageDigestMirrorSet is configured accordingly.

  - mirrors:
    - registry.example.org/quay.io/openshift-release-dev/ocp-v4.0-art-dev
    source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
  - mirrors:
    - registry.example.org/quay.io/openshift-release-dev/ocp-release
    source: quay.io/openshift-release-dev/ocp-release

When the HostedCluster resource is being updated with a new Version (and therefore the Version Image is updated), reconcling of the HostedCluster resource is failing because multiple Image Registry rewrites are taking place, causing an invalid Image Registry URL.

{"level":"error","ts":"2025-09-23T16:38:55Z","msg":"reconciliation is blocked","controller":"hostedcluster","controllerGroup":"hypershift.openshift.io","controllerKind":"HostedCluster","HostedCluster":{"name":"demo","namespace":"clusters"},"namespace":"clusters","name":"demo","reconcileID":"4cc37694-2b8e-4205-a3df-e21c9256d7fb","message":"failed to lookup release image: failed to extract release metadata: failed to obtain root manifest for registry.example.org/registry.example.org/registry.example.org/quay.io/openshift-release-dev/ocp-release@sha256:2db093f063ad5310fa4e5ed2d2eda4bad5215c47092b72d1cfafbcfdbf1f4dd2: unauthorized: project registry.example.org not found: project registry.example.org not found","error":"release image is invalid","stacktrace":"github.com/openshift/hypershift/hypershift-operator/controllers/hostedcluster.(*HostedClusterReconciler).reconcile\n\t/hypershift/hypershift-operator/controllers/hostedcluster/hostedcluster_controller.go:1200\ngithub.com/openshift/hypershift/hypershift-operator/controllers/hostedcluster.(*HostedClusterReconciler).Reconcile\n\t/hypershift/hypershift-operator/controllers/hostedcluster/hostedcluster_controller.go:343\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:224"}

The cause of this behavior is the fact that Hosted Control Planes does not correctly rewrite the Image Registry based on the hostname but based on some patterns. Given that in "registry.example.org/quay.io/openshift-release-dev/ocp-v4.0-art-dev" we have registry.example.com as hostname and quay.io as repository name, it sill seems to match for quay.io and implement the Image Registry rewrite. Changing the repository name to something like registry.example.org/quay/openshift-release-dev/ocp-v4.0-art-dev does workaroud the problem but still confirms the faulty behavior observed.

Given that most Image Registries are creating the repository name based on the upstream Image Registry, when using pullthrough, this issue should be resolved to prevent issues from happening during deployment and update of HostedCluster resources.

OpenShift Container Platform 4.19 and before

Always

1. As documented in the description

The update of the HostedCluster resource is stuck with the below error reported during the reconvile process.

{"level":"error","ts":"2025-09-23T16:38:55Z","msg":"reconciliation is blocked","controller":"hostedcluster","controllerGroup":"hypershift.openshift.io","controllerKind":"HostedCluster","HostedCluster":{"name":"demo","namespace":"clusters"},"namespace":"clusters","name":"demo","reconcileID":"4cc37694-2b8e-4205-a3df-e21c9256d7fb","message":"failed to lookup release image: failed to extract release metadata: failed to obtain root manifest for registry.example.org/registry.example.org/registry.example.org/quay.io/openshift-release-dev/ocp-release@sha256:2db093f063ad5310fa4e5ed2d2eda4bad5215c47092b72d1cfafbcfdbf1f4dd2: unauthorized: project registry.example.org not found: project registry.example.org not found","error":"release image is invalid","stacktrace":"github.com/openshift/hypershift/hypershift-operator/controllers/hostedcluster.(*HostedClusterReconciler).reconcile\n\t/hypershift/hypershift-operator/controllers/hostedcluster/hostedcluster_controller.go:1200\ngithub.com/openshift/hypershift/hypershift-operator/controllers/hostedcluster.(*HostedClusterReconciler).Reconcile\n\t/hypershift/hypershift-operator/controllers/hostedcluster/hostedcluster_controller.go:343\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:116\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:303\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/hypershift/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:224"}

OpenShift Container Platform should be able to deal with Image Registry URL such as registry.example.org/quay.io/openshift-release-dev/ocp-v4.0-art-dev and prevent false/incorrect rewrites from happening.

It was also discussed in https://redhat-external.slack.com/archives/C01C8502FMM/p1759218849717419

 - mirrors:
   - registry.example.org/quay/openshift-release-dev/ocp-v4.0-art-dev
   source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
 - mirrors:
   - registry.example.org/quay/openshift-release-dev/ocp-release
   source: quay.io/openshift-release-dev/ocp-release