Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-13873

CVO cannot reach apiserver via internal LB hostname for >50s during installation

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Undefined Undefined
    • None
    • 4.14
    • openshift-apiserver
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • No
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      CVO cannot reach apiserver for >50s during install. CVO log shows CVO pod gets startup at 04:17:22.404252 but hits context deadline exceeded.

      2023-04-30T04:17:22.404379852Z I0430 04:17:22.404252       1 start.go:23] ClusterVersionOperator 4.14.0-202304281554.p0.gf74d114.assembly.stream-f74d114
      2023-04-30T04:17:47.429315202Z W0430 04:17:47.429229       1 start.go:157] Failed to get FeatureGate from cluster: Get "https://api-int.ci-op-v41dk4wl-1ab22.qe.azure.devcluster.openshift.com:6443/apis/config.openshift.io/v1/featuregates/cluster": context deadline exceeded
      2023-04-30T04:17:47.429315202Z F0430 04:17:47.429290       1 start.go:29] error: Get "https://api-int.ci-op-v41dk4wl-1ab22.qe.azure.devcluster.openshift.com:6443/apis/config.openshift.io/v1/featuregates/cluster": context deadline exceeded 

      Then CVO gets restarted and up running eventually. 

      2023-04-30T04:18:13.360342606+00:00 stderr F I0430 04:18:13.360182       1 start.go:23] ClusterVersionOperator 4.14.0-202304281554.p0.gf74d114.assembly.stream-f74d114
      2023-04-30T04:18:13.387742435+00:00 stderr F I0430 04:18:13.387693       1 upgradeable.go:446] ConfigMap openshift-config/admin-acks added. 

      Version-Release number of selected component (if applicable):

      4.14.0-0.nightly-2023-04-29-153308

      How reproducible:

      6/184

      Steps to Reproduce:

      1. Install OCP cluster on different flavors and watch the CVO pod
      2.
      3.
      

      Actual results:

      CVO hits context deadline exceeded when reaching apiserver

      Expected results:

      CVO reaches apiserver w/o error

      Additional info:

      Must gather is available at https://drive.google.com/file/d/1f6awuVecybcBB5a0Co1rgG8vK8kluMX9/view?usp=sharing

              Unassigned Unassigned
              yanyang@redhat.com Yang Yang
              None
              None
              Rahul Gangwar Rahul Gangwar
              None
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: