Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-70297

Multi-arch disconnected install (payload mirrored using oc adm release mirror)- Clusterversion operator pod fails with ImagePullBackOff due to SignatureValidationFailed.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 4.21
    • Multi-Arch / IBM P and Z
    • None
    • None
    • True
    • Hide

      Disconnected Installation is the preferred installation method for most of the customers and it is not working with rc0 build. As this is one of the primary features needs to be immediately fixed or some acceptable Workaround needed here. 

      Show
      Disconnected Installation is the preferred installation method for most of the customers and it is not working with rc0 build. As this is one of the primary features needs to be immediately fixed or some acceptable Workaround needed here. 
    • None
    • Important
    • Yes
    • ppc64le, s390x
    • None
    • None
    • Rejected
    • None
    • None
    • Known Issue
    • Hide
      When mirroring an OCP release to the registry of the disconnected environment using `oc adm release mirror`, the release image's cosign signature image is not mirrored along with the image.
      Starting OCP 4.21, the ClusterImagePolicy `openshift` is deployed by default to the cluster, resulting in CRI-O verifying signatures associated with release image when pulling these images to the cluster. Pulling a release image that lacks cosign signature will result in a ImagePullBackOff failure.
      As a result, when upgrading to OCP 4.21 on a disconnected environment, the upgrade will fail if the `oc adm release mirror` was previously used in order to mirror the release.
      We highly suggest migrating to `oc-mirror v2` starting 4.21 in order to mirror releases and their signatures to your disconnected environment.
      if you need to rely on `oc adm release mirror` still, please review the workaround suggested in [mirroring to disconnected using oc adm release mirror page]
      Show
      When mirroring an OCP release to the registry of the disconnected environment using `oc adm release mirror`, the release image's cosign signature image is not mirrored along with the image. Starting OCP 4.21, the ClusterImagePolicy `openshift` is deployed by default to the cluster, resulting in CRI-O verifying signatures associated with release image when pulling these images to the cluster. Pulling a release image that lacks cosign signature will result in a ImagePullBackOff failure. As a result, when upgrading to OCP 4.21 on a disconnected environment, the upgrade will fail if the `oc adm release mirror` was previously used in order to mirror the release. We highly suggest migrating to `oc-mirror v2` starting 4.21 in order to mirror releases and their signatures to your disconnected environment. if you need to rely on `oc adm release mirror` still, please review the workaround suggested in [mirroring to disconnected using oc adm release mirror page]
    • None
    • None
    • None
    • None

      Description of problem:

      Multi-arch disconnected install - Clusterversion operator pod fails with ImagePullBackOff due to SignatureValidationFailed.  

      Version-Release number of selected component (if applicable):

      K8s Version: v1.34.1
CoreOS: 9.6.20251112-0
OCP Build Version: 4.21.0-rc.0

      How reproducible: Always

      Always

      Steps to Reproduce:

      Try to install the multi-arch cluster with disconnected mode.

      Actual results:

      pod fails to come up. 
NAME                                       READY   STATUS             RESTARTS   AGE
cluster-version-operator-b6f96cf58-z5fw2   0/1     ImagePullBackOff   0          38m

      1. Bootstrap fails to complete because the Cluster Version Operator pod in the openshift-cluster-version namespace is stuck in ImagePullBackOff state due to SignatureValidationFailed errors while pulling the image.
      2. Issue observed on multi-arch x86 and s390x. 

      Expected results:

      Pod should come up

      Additional info:
      1. Multi-arch disconnected installation was working in 4.21.0-ec.3 build.

      OCP Build Version: 4.21.0-ec.3
CoreOS Version : 9.6.20251112-0
      NAME                                        READY   STATUS    RESTARTS   AGE
cluster-version-operator-7cc989c5f4-6trwg   1/1     Running   0          125m

       

              mawerner@redhat.com Matthew Werner
              rh-ee-shallike Shreya Hallikeri
              None
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Created:
                Updated: