Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-35335

Failed to pull/push blob from/to image registry on external OIDC cluster

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Critical Critical
    • None
    • 4.15.z, 4.16, 4.17
    • Image Registry
    • Critical
    • No
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, the internal image registry would not correctly authenticate users on clusters configured with external OpenID Connect (OIDC) users. Consequently, this made it impossible for users to push or pull images to and from the internal image registry. With this update, the internal image registry starts by using the `SelfSubjectReview` API, dropping use of the `openshift specific user` API, which is not available on clusters configured with external OIDC users. As a result, it is now possible to successfully authenticate with the internal image registry again. (link:https://issues.redhat.com/browse/OCPBUGS-35335[*OCPBUGS-35335*])
      Show
      * Previously, the internal image registry would not correctly authenticate users on clusters configured with external OpenID Connect (OIDC) users. Consequently, this made it impossible for users to push or pull images to and from the internal image registry. With this update, the internal image registry starts by using the `SelfSubjectReview` API, dropping use of the `openshift specific user` API, which is not available on clusters configured with external OIDC users. As a result, it is now possible to successfully authenticate with the internal image registry again. (link: https://issues.redhat.com/browse/OCPBUGS-35335 [* OCPBUGS-35335 *])
    • Bug Fix
    • Done

      Description of problem:

      user.openshift.io and oauth.openshift.io APIs are not unavailable in external oidc cluster, that conducts all the common pull/push blob from/to image registry failed.

      Version-Release number of selected component (if applicable):

      4.15.15

      How reproducible:

      always

      Steps to Reproduce:

      1.Create a ROSA HCP cluster which configured external oidc users
2.Push data to image registry under a project
oc new-project wxj1
oc new-build httpd~https://github.com/openshift/httpd-ex.git 
3.

      Actual results:

      $ oc logs -f build/httpd-ex-1
Cloning "https://github.com/openshift/httpd-ex.git" ...	Commit:	1edee8f58c0889616304cf34659f074fda33678c (Update httpd.json)	Author:	Petr Hracek <phracek@redhat.com>	Date:	Wed Jun 5 13:00:09 2024 +0200time="2024-06-12T09:55:13Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"I0612 09:55:13.306937       1 defaults.go:112] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].Caching blobs under "/var/cache/blobs".Trying to pull image-registry.openshift-image-registry.svc:5000/openshift/httpd@sha256:765aa645587f34e310e49db7cdc97e82d34122adb0b604eea891e0f98050aa77...Warning: Pull failed, retrying in 5s ...Trying to pull image-registry.openshift-image-registry.svc:5000/openshift/httpd@sha256:765aa645587f34e310e49db7cdc97e82d34122adb0b604eea891e0f98050aa77...Warning: Pull failed, retrying in 5s ...Trying to pull image-registry.openshift-image-registry.svc:5000/openshift/httpd@sha256:765aa645587f34e310e49db7cdc97e82d34122adb0b604eea891e0f98050aa77...Warning: Pull failed, retrying in 5s ...error: build error: After retrying 2 times, Pull image still failed due to error: unauthorized: unable to validate token: NotFound


oc logs -f deploy/image-registry -n openshift-image-registry

time="2024-06-12T09:55:13.36003996Z" level=error msg="invalid token: the server could not find the requested resource (get users.user.openshift.io ~)" go.version="go1.20.12 X:strictfipsruntime" http.request.host="image-registry.openshift-image-registry.svc:5000" http.request.id=0c380b81-99d4-4118-8de3-407706e8767c http.request.method=GET http.request.remoteaddr="10.130.0.35:50550" http.request.uri="/openshift/token?account=serviceaccount&scope=repository%3Aopenshift%2Fhttpd%3Apull" http.request.useragent="containers/5.28.0 (github.com/containers/image)"

      Expected results:

      Should pull/push blob from/to image registry on external oidc cluster

      Additional info:

       

              fmissi Flavian Missi
              rh-ee-xiuwang XiuJuan Wang
              XiuJuan Wang XiuJuan Wang
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: