Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-44314

Cannot access external network via https from the HCP openshift-apiserver component

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Trying to setup a disconnected HCP cluster with self-managed image registry.
      
      After the cluster installed, all the imagestream failed to import images.
      With error:
      ```
      Get "https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/": http: server gave HTTP response to HTTPS client
      ```
      
      The imagestream will talk to openshift-apiserver and get the image target there.
      
      After login to the hcp namespace, figured out that I cannot access any external network with https protocol.

      Version-Release number of selected component (if applicable):

      4.14.35    

      How reproducible:

          always

      Steps to Reproduce:

          1. Install the hypershift hosted cluster with above setup
          2. The cluster can be created successfully and all the pods on the cluster can be running with the expected images pulled
          3. Check the internal image-registry
          4. Check the openshift-apiserver pod from management cluster
          

      Actual results:

      All the imagestreams failed to sync from the remote registry.
      $ oc describe is cli -n openshift
      Name:            cli
      Namespace:        openshift
      Created:        6 days ago
      Labels:            <none>
      Annotations:        include.release.openshift.io/ibm-cloud-managed=true
                  include.release.openshift.io/self-managed-high-availability=true
                  openshift.io/image.dockerRepositoryCheck=2024-11-06T22:12:32Z
      Image Repository:    image-registry.openshift-image-registry.svc:5000/openshift/cli
      Image Lookup:        local=false
      Unique Images:        0
      Tags:            1latest
        updates automatically from registry quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:49baeac68e90026799d0b62609e04adf285be5b83bdb5dbd372de2b14442be5d  ! error: Import failed (InternalError): Internal error occurred: [122610517469.dkr.ecr.us-west-2.amazonaws.com/ocp-mirror-1@sha256:49baeac68e90026799d0b62609e04adf285be5b83bdb5dbd372de2b14442be5d: Get "https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/": http: server gave HTTP response to HTTPS client, 122610517469.dkr.ecr.us-west-2.amazonaws.com/ocp-mirror-2@sha256:49baeac68e90026799d0b62609e04adf285be5b83bdb5dbd372de2b14442be5d: Get "https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/": http: server gave HTTP response to HTTPS client, 122610517469.dkr.ecr.us-west-2.amazonaws.com/ocp-mirror-3@sha256:49baeac68e90026799d0b62609e04adf285be5b83bdb5dbd372de2b14442be5d: Get "https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/": http: server gave HTTP response to HTTPS client, 122610517469.dkr.ecr.us-west-2.amazonaws.com/ocp-mirror-4@sha256:49baeac68e90026799d0b62609e04adf285be5b83bdb5dbd372de2b14442be5d: Get "https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/": http: server gave HTTP response to HTTPS client, 122610517469.dkr.ecr.us-west-2.amazonaws.com/ocp-mirror-5@sha256:49baeac68e90026799d0b62609e04adf285be5b83bdb5dbd372de2b14442be5d: Get "https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/": http: server gave HTTP response to HTTPS client, quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:49baeac68e90026799d0b62609e04adf285be5b83bdb5dbd372de2b14442be5d: Get "https://quay.io/v2/": http: server gave HTTP response to HTTPS client]
      
      
      Access the external network from the openshift-apiserver pod:
      sh-5.1$ curl --connect-timeout 5 https://quay.io/v2
      curl: (28) Operation timed out after 5001 milliseconds with 0 out of 0 bytes received
      sh-5.1$ curl --connect-timeout 5 https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/
      curl: (28) Operation timed out after 5001 milliseconds with 0 out of 0 bytes received
      
      sh-5.1$ env | grep -i http.*proxy
      HTTPS_PROXY=http://127.0.0.1:8090
      HTTP_PROXY=http://127.0.0.1:8090
      
      

      Expected results:

      The openshift-apiserver should be able to talk to the remote https services.

      Additional info:

      It is working after set the registry to no_proxy
      
      sh-5.1$ NO_PROXY=122610517469.dkr.ecr.us-west-2.amazonaws.com curl --connect-timeout 5 https://122610517469.dkr.ecr.us-west-2.amazonaws.com/v2/
      Not Authorized
       

       

              cewong@redhat.com Cesar Wong
              bmeng_sre.openshift Bo Meng
              Jie Zhao Jie Zhao
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: