Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-8033

[IBMCloud] install disconnect cluster was blocked by an error that occurred in Post "https://iam.cloud.ibm.com/identity/token"

XMLWordPrintable

    • Critical
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Install disconnect cluster (mirror registry) failed, found error occurred in “Post "https://iam.cloud.ibm.com/identity/token"

      Version-Release number of selected component (if applicable):

      registry.build05.ci.openshift.org/ci-ln-5g4sj02/release:latest
      Tag: 4.13.0-0.ci.test-2023-02-27-033047-ci-ln-5g4sj02-latest
      pre-merge https://github.com/openshift/installer/pull/6836

      How reproducible:

      Alwats

      Steps to Reproduce:

      1.Create bastion instance maxu-ibmj-p1-int-svc
      2.Create vpc on the bastion host 
      3.Mirror the image
      ssh  -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o VerifyHostKeyDNS=yes  -i "/home/jenkins/ws/workspace/ocp-common/Flexy-install/flexy/private/config/keys/openshift-qe.pem" core@150.239.220.91 "~/oc adm release mirror --insecure=true -a mirror_registry_credentials.json --from=registry.build05.ci.openshift.org/ci-ln-5g4sj02/release:latest --to=maxu-ibmj-p1.mirror-registry.ibmcloud.qe.devcluster.openshift.com:5000/ci-ln-5g4sj02/release --to-release-image=maxu-ibmj-p1.mirror-registry.ibmcloud.qe.devcluster.openshift.com:5000/ci-ln-5g4sj02/release:latest"
      4.Install a disconnect cluster based on mirror-registry 

      Actual results:

      Install the disconnect cluster failed 
      $ oc get nodes
       NAME                          STATUS     ROLES                  AGE   VERSION 
      maxu-ibmj-p1-nq2sv-master-0   NotReady   control-plane,master   58m   v1.26.0+518f3ed maxu-ibmj-p1-nq2sv-master-1   NotReady   control-plane,master   58m   
      v1.26.0+518f3ed maxu-ibmj-p1-nq2sv-master-2   NotReady   control-plane,master   58m   v1.26.0+518f3ed
      
      $ oc get co | grep True
      cloud-controller-manager                   4.13.0-0.ci.test-2023-02-27-045511-ci-ln-5g4sj02-latest   True        False         False      103m  
      
      $ oc get pods -A | grep Running
      openshift-cloud-controller-manager-operator        cluster-cloud-controller-manager-operator-676fd7dddf-6phrn   2/2     Running   0          105m
      openshift-cloud-controller-manager                 ibm-cloud-controller-manager-76dc4fbdf5-cs8pt                1/1     Running   0          105m
      openshift-cloud-controller-manager                 ibm-cloud-controller-manager-76dc4fbdf5-gp8bs                1/1     Running   0          6m44s
      
      $ oc logs ibm-cloud-controller-manager-76dc4fbdf5-cs8pt -n openshift-cloud-controller-manager
      E0228 08:44:41.083901       1 node_controller.go:229] error syncing 'maxu-ibmj-p1-nq2sv-master-1': failed to get instance metadata for node maxu-ibmj-p1-nq2sv-master-1: An error occurred while performing the 'authenticate' step: Post "https://iam.cloud.ibm.com/identity/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers), requeuing
      E0228 08:45:11.084753       1 node_controller.go:258] Error getting instance metadata for node addresses: An error occurred while performing the 'authenticate' step: Post "https://iam.cloud.ibm.com/identity/token": dial tcp 23.212.249.210:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
      

      Expected results:

      install cluster succeed

      Additional info:

      install-config.yaml
      platform:
        ibmcloud:
          region: us-east
          networkResourceGroupName: maxu-ibmj-p1-rg
          vpcName: maxu-ibmj-p1-vpc
          controlPlaneSubnets:
          - maxu-ibmj-p1-control-plane-us-east-3
          - maxu-ibmj-p1-control-plane-us-east-2
          - maxu-ibmj-p1-control-plane-us-east-1
          computeSubnets:
          - maxu-ibmj-p1-compute-us-east-3
          - maxu-ibmj-p1-compute-us-east-2
          - maxu-ibmj-p1-compute-us-east-1
      ...
      publish: Internal
      imageContentSources:
      - mirrors:
        - maxu-ibmj-p1.mirror-registry.ibmcloud.qe.devcluster.openshift.com:5000/ci-ln-5g4sj02/release
        source: registry.build05.ci.openshift.org/ci-ln-5g4sj02/release
      - mirrors:
        - maxu-ibmj-p1.mirror-registry.ibmcloud.qe.devcluster.openshift.com:5000/ci-ln-5g4sj02/release
        source: registry.build05.ci.openshift.org/ci-ln-5g4sj02/stable
      baseDomain: ibmcloud.qe.devcluster.openshift.com
      

              jeffbnowicki Jeff Nowicki
              maxu@redhat.com May Xu
              Johnny Liu Johnny Liu
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: