Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-28781

In HCP cluster updating pull-secret in hosted cluster CR on HUB cluster is not reflecting on HCP cluster VMs

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • 4.13
    • 4.13
    • HyperShift
    • No
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Customer created hosted control plane (HCP of type kubevirt) clusters on Hub OCP cluster
      
      Now for their workload to pull images on HCP cluster
      
      
      
      They added  auth for our registries to a secret named "scale-rm-pull-secret" in "clusters" namespace in Hub cluster.And then specified this secret "scale-rm-pull-secret" in hostedcluster CR for HCP in question in hub under namespace "clusters"
      
      
      
      They expect this  change to reflect on HCP cluster nodes and images to be pulled successfully. However they keep getting imagepullbackoff error on HCP cluster
      
      PodPibm-spectrum-scale-controller-manager-5cb84655b4-dvnxk
      
      NamespaceNSibm-spectrum-scale-operator
      
      Generated from kubelet on scale-41312-t7nml
      2 times in the last 0 minutes
      Failed to pull image "icr.io/cpopen/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8": rpc error: code = Unknown desc = (Mirrors also failed: [cp.stg.icr.io/cp/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8: Requesting bearer token: invalid status code from registry 400 (Bad Request)] [docker-na-public.artifactory.swg-devops.com/sys-spectrum-scale-team-cloud-native-docker-local/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8: unable to retrieve auth token: invalid username/password: authentication required]): icr.io/cpopen/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8: reading manifest sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8 in icr.io/cpopen/ibm-spectrum-scale-operator: manifest unknown
      
      Customer is able to pull the image manually using same credentials
      
      podman pull docker-na-public.artifactory.swg-devops.com/sys-spectrum-scale-team-cloud-native-docker-local/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8

      Version-Release number of selected component (if applicable):

       

      How reproducible:

       

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

       

      Expected results:

       

      Additional info:

      Pulled the image manually on nodes successfully after logging to registry with same credentials but pod continues to say can not pull image. ANother thing to note is that pod has imagepullpolicy as "ifnotpresent" so after manual pull on all three nodes also why it continue to throw same error
      podman pull docker-na-public.artifactory.swg-devops.com/sys-spectrum-scale-team-cloud-native-docker-local/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8
      Trying to pull docker-na-public.artifactory.swg-devops.com/sys-spectrum-scale-team-cloud-native-docker-local/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8...
      Getting image source signatures
      Copying blob 1e3d9b7d1452 skipped: already exists  
      Copying blob fe5ca62666f0 skipped: already exists  
      Copying blob e8c73c638ae9 skipped: already exists  
      Copying blob fcb6f6d2c998 skipped: already exists  
      Copying blob b02a7525f878 skipped: already exists  
      Copying blob 4aa0ea1413d3 skipped: already exists  
      Copying blob 7c881f9ab25e skipped: already exists  
      Copying blob 5627a970d25e skipped: already exists  
      Copying blob c7e34367abae skipped: already exists  
      Copying blob f92848770344 skipped: already exists  
      Copying blob a7ca0d9ba68f skipped: already exists  
      Copying config 07120ff2fe done  
      Writing manifest to image destination
      Storing signatures
      07120ff2fe00d6335ef757b33546fc9ec9e3d799a500349343f09228bcdf73c0
      sh-5.1# 
      
      PodPibm-spectrum-scale-controller-manager-5cb84655b4-dvnxk
      NamespaceNSibm-spectrum-scale-operator
      21 Sept 2023, 17:58
      Generated from kubelet on scale-41312-t7nml
      2 times in the last 0 minutes
      Failed to pull image "icr.io/cpopen/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8": rpc error: code = Unknown desc = (Mirrors also failed: [cp.stg.icr.io/cp/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8: Requesting bearer token: invalid status code from registry 400 (Bad Request)] [docker-na-public.artifactory.swg-devops.com/sys-spectrum-scale-team-cloud-native-docker-local/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8: unable to retrieve auth token: invalid username/password: authentication required]): icr.io/cpopen/ibm-spectrum-scale-operator@sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8: reading manifest sha256:f6138abb5493d7ef6405dcf0a6bb5afc697cca9f20be1a88b3214268b6382da8 in icr.io/cpopen/ibm-spectrum-scale-operator: manifest unknown

            pstefans@redhat.com Patryk Stefanski
            rhn-support-bhab Bharathi B
            Liangquan Li Liangquan Li
            Bharathi B
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: