Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-32583

pinned images fail when using ImageDigestMirrorSets

XMLWordPrintable

    • Moderate
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When we pin an image while using ImageDigestMirrorSets, we get this failure:
      
      
      E0422 14:22:29.588035    2366 daemon.go:1380] Fatal error from auxiliary tools: failed to get auth config for image example.io/digest-example/mybusy@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a
      4591f08019: no auth found for image: "example.io/digest-example/mybusy@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019"
      
          

      Version-Release number of selected component (if applicable):

      pre-merge
          

      How reproducible:

      Always
          

      Steps to Reproduce:

          1. Create an ImageDigestMirrorSet to configure mirrors for an image
      
      apiVersion: config.openshift.io/v1
      kind: ImageDigestMirrorSet
      metadata:
        name: digest-mirror
      spec:
        imageDigestMirrors:
        - mirrors:
          - quay.io/openshifttest/busybox
          source: example.io/digest-example/mybusy
          mirrorSourcePolicy: NeverContactSource  # do not redirect to the source registry if the pull from the mirror is failed
      
      
          2. Create a pinnedimageset using the values in the ImageDigestMirrorSet
      
      apiVersion: machineconfiguration.openshift.io/v1alpha1
      kind: PinnedImageSet
      metadata:
        creationTimestamp: "2024-04-22T12:49:51Z"
        generation: 1
        labels:
          machineconfiguration.openshift.io/role: worker
        name: my-worker-pinned-images
        resourceVersion: "78482"
        uid: f06c94c4-067f-4404-b3c2-11d5aff4e0cb
      spec:
        pinnedImages:
        - name: example.io/digest-example/mybusy@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019
      
         
          

      Actual results:

      In the MCD logs we can see the following failure
      
      $ oc -n openshift-machine-config-operator logs machine-config-daemon-dmtxx
      ...
      I0422 14:26:14.117242    2438 pinned_image_set.go:274] Reconciling pinned image set: my-worker-pinned-images-2: generation: 1
      E0422 14:26:14.125965    2438 pinned_image_set.go:981] failed to get auth config for image example.io/digest-example/mybusy@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019: no auth found
       for image: "example.io/digest-example/mybusy@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019"
      W0422 14:26:14.125990    2438 pinned_image_set.go:983]  failed: worker max retries: 15
      
          

      Expected results:

      Since we can pull the image when we debug the node, we should be able to pin it:
      
      sh-5.1# crictl pull example.io/digest-example/mybusy@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019
      Image is up to date for example.io/digest-example/mybusy@sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019
      
      
      
      
          

      Additional info:

      We get a similar error when we try to pin release images while using releases built by clusterbot. For example, we can try to pin the rhel-coreos image:
      
      $ oc adm release info --image-for rhel-coreos
      registry.build03.ci.openshift.org/ci-ln-4cx6v6b/stable@sha256:85a096a567ca287ba9c0fe36642e49c34eb4dd541914f8823750e4b186fce569
      
      If we try to pin it, we get a similar error:
      
      E0422 13:03:14.979410    2951 daemon.go:1380] Fatal error from auxiliary tools: failed to get auth config for image registry.build03.ci.openshift.org/ci-ln-4cx6v6b/stable@sha256:85a096a567ca287ba9c0fe36642e49c34eb4dd541914f8823750e4b186fce569: no auth found for image: "registry.build03.ci.openshift.org/ci-ln-4cx6v6b/stable@sha256:85a096a567ca287ba9c0fe36642e49c34eb4dd541914f8823750e4b186fce569"
      
      
      But the image can actually be pulled:
      
      sh-5.1# crictl  pull registry.build03.ci.openshift.org/ci-ln-4cx6v6b/stable@sha256:85a096a567ca287ba9c0fe36642e49c34eb4dd541914f8823750e4b186fce569
      Image is up to date for registry.build03.ci.openshift.org/ci-ln-4cx6v6b/stable@sha256:85a096a567ca287ba9c0fe36642e49c34eb4dd541914f8823750e4b186fce569
          

            sbatschelet Sam Batschelet
            sregidor@redhat.com Sergio Regidor de la Rosa
            Sergio Regidor de la Rosa Sergio Regidor de la Rosa
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: