Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-52940

[Custom DNS]AWS: CoreDNS pod was not running on control plane nodes

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Critical
    • None
    • None
    • Rejected
    • Installer Sprint 267, Installer Sprint 268, Installer Sprint 269, Installer Sprint 270, Installer Sprint 271
    • 5
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      
      Create a cluster with custom DNS enabled on AWS, the bootstrap failed, checking pods on control plane, the coredns was not running:
      
      [core@ip-10-0-3-86 ~]$ sudo crictl ps
      CONTAINER       	IMAGE           	CREATED         	STATE           	NAME            	ATTEMPT         	POD ID          	POD             	NAMESPACE
      <empty>
      
      install-config:
      platform:
        aws:
      	userProvisionedDNS: Enabled
      featureSet: TechPreviewNoUpgrade
      
      
          

      Version-Release number of selected component (if applicable):

      4.19.0-0.nightly-2025-03-09-063419
          

      How reproducible:

      Always
          

      Steps to Reproduce:

          1.Config install-config.yaml as described in the description
          2.Create cluster
          3.
          

      Actual results:

      No coredns pod on control plane machines, bootstrap failed.
          

      Expected results:

      Except coredns running and work well on control plane machines, bootstrap process succeeds.
          

      Additional info:

      Checking kubelet log on master
      
      Mar 11 03:06:00 ip-10-0-3-86 kubenswrapper[2485]: I0311 03:06:00.200021    2485 csi_plugin.go:887] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.yunjiang-dns12.qe.devcluster.openshift.com:6443/apis/storage.k8s.io/v1/csinodes/ip-10-0-3-86.us-east-2.compute.internal?resourceVersion=0": dial tcp: lookup api-int.yunjiang-dns12.qe.devcluster.openshift.com on 10.0.0.2:53: no such host
      
      it looks looks like api-int is not resolvable, the name servers on control plane and bootstrap are:
      
      find . -name resolv.conf -printf "%P\n" -exec cat {} \;
      
      control-plane/10.0.66.134/network/resolv.conf
      # Generated by NetworkManager
      search us-east-2.compute.internal
      nameserver 10.0.0.2
      
      control-plane/10.0.57.233/network/resolv.conf
      # Generated by NetworkManager
      search us-east-2.compute.internal
      nameserver 10.0.0.2
      
      control-plane/10.0.3.86/network/resolv.conf
      # Generated by NetworkManager
      search us-east-2.compute.internal
      nameserver 10.0.0.2
      
      
      bootstrap/network/resolv.conf
      # Generated by NetworkManager
      nameserver 127.0.0.1
      search us-east-2.compute.internal
      nameserver 10.0.0.2
      
      
      The cluster-infrastructure-02-config.yml:
      
      status:
        apiServerInternalURI: https://api-int.yunjiang-dns12.qe.devcluster.openshift.com:6443
        apiServerURL: https://api.yunjiang-dns12.qe.devcluster.openshift.com:6443
        controlPlaneTopology: HighlyAvailable
        cpuPartitioning: None
        etcdDiscoveryDomain: ""
        infrastructureName: yunjiang-dns12-pd4mm
        infrastructureTopology: HighlyAvailable
        platform: AWS
        platformStatus:
          aws:
            cloudLoadBalancerConfig:
              clusterHosted:
                apiIntLoadBalancerIPs:
                - 10.0.46.213
                - 10.0.16.77
                - 10.0.76.110
                apiLoadBalancerIPs:
                - 3.142.117.227
                - 3.135.145.102
                - 3.146.246.29
              dnsType: ClusterHosted
            region: us-east-2
          type: AWS
          

              sdasu@redhat.com Sandhya Dasu
              yunjiang-1 Yunfei Jiang
              None
              None
              Yunfei Jiang Yunfei Jiang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: