Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-16626

No cloud.network.openshift.io/egress-ipconfig in UPI GCP XPN cluster

XMLWordPrintable

    • -
    • Critical
    • No
    • SDN Sprint 239
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      No cloud.network.openshift.io/egress-ipconfig in node's annotation for UPI GCP XPN cluster, lots of failing to retrieve private IP config errors in CNCC logs

      Version-Release number of selected component (if applicable):

       

      How reproducible:

       

      Steps to Reproduce:

      1. Built UPI GCP XPN cluster using template: functionality-testing/aos-4_14/upi-on-gcp/versioned-installer-ovn 
      2. "oc describe node" on any worker node to check cloud.network.openshift.io/egress-ipconfig 
      3. check cloud-network-config-controller pod log 

      Actual results:

      There is no cloud.network.openshift.io/egress-ipconfig in node's annotation, there are lots of "error retrieving the private IP configuration" error in CNCC log

      Expected results:

      cloud.network.openshift.io/egress-ipconfig should present in node's annotation, there should be no retrieving private IP config error in CNCC
      
      $ oc -n openshift-cloud-network-config-controller log cloud-network-config-controller-67b6786d7c-txncc
      flags cannot be placed before plugin name: -n
      [jechen@jechen ~]$ oc -n openshift-cloud-network-config-controller logs cloud-network-config-controller-67b6786d7c-txncc
      W0511 13:11:46.285444       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
      I0511 13:11:46.292453       1 leaderelection.go:248] attempting to acquire leader lease openshift-cloud-network-config-controller/cloud-network-config-controller-lock...
      E0511 13:12:16.296585       1 leaderelection.go:330] error retrieving resource lock openshift-cloud-network-config-controller/cloud-network-config-controller-lock: Get "https://api-int.jechen-0511a.qe-shared-vpc.qe.gcp.devcluster.openshift.com:6443/api/v1/namespaces/openshift-cloud-network-config-controller/configmaps/cloud-network-config-controller-lock": dial tcp: lookup api-int.jechen-0511a.qe-shared-vpc.qe.gcp.devcluster.openshift.com: i/o timeout
      E0511 13:13:04.688490       1 leaderelection.go:330] error retrieving resource lock openshift-cloud-network-config-controller/cloud-network-config-controller-lock: Get "https://api-int.jechen-0511a.qe-shared-vpc.qe.gcp.devcluster.openshift.com:6443/api/v1/namespaces/openshift-cloud-network-config-controller/configmaps/cloud-network-config-controller-lock": dial tcp: lookup api-int.jechen-0511a.qe-shared-vpc.qe.gcp.devcluster.openshift.com on 172.30.0.10:53: read udp 10.130.0.28:51847->172.30.0.10:53: read: connection refused
      I0511 13:13:49.752088       1 leaderelection.go:258] successfully acquired lease openshift-cloud-network-config-controller/cloud-network-config-controller-lock
      I0511 13:13:49.754236       1 controller.go:88] Starting node controller
      I0511 13:13:49.754265       1 controller.go:91] Waiting for informer caches to sync for node workqueue
      I0511 13:13:49.754493       1 controller.go:88] Starting secret controller
      I0511 13:13:49.754503       1 controller.go:91] Waiting for informer caches to sync for secret workqueue
      I0511 13:13:49.755050       1 controller.go:88] Starting cloud-private-ip-config controller
      I0511 13:13:49.755070       1 controller.go:91] Waiting for informer caches to sync for cloud-private-ip-config workqueue
      I0511 13:13:49.770422       1 controller.go:182] Assigning key: jechen-0511a-49ntn-master-0.c.openshift-qe.internal to node workqueue
      I0511 13:13:49.770461       1 controller.go:182] Assigning key: jechen-0511a-49ntn-master-1.c.openshift-qe.internal to node workqueue
      I0511 13:13:49.770491       1 controller.go:182] Assigning key: jechen-0511a-49ntn-master-2.c.openshift-qe.internal to node workqueue
      I0511 13:13:49.855213       1 controller.go:96] Starting node workers
      I0511 13:13:49.855317       1 controller.go:96] Starting cloud-private-ip-config workers
      I0511 13:13:49.855621       1 controller.go:102] Started cloud-private-ip-config workers
      I0511 13:13:49.855652       1 controller.go:102] Started node workers
      I0511 13:13:49.855759       1 controller.go:96] Starting secret workers
      I0511 13:13:49.855815       1 controller.go:102] Started secret workers
      E0511 13:13:50.226995       1 controller.go:165] error syncing 'jechen-0511a-49ntn-master-0.c.openshift-qe.internal': error retrieving the private IP configuration for node: jechen-0511a-49ntn-master-0.c.openshift-qe.internal, err: error retrieving the network interface subnets, err: googleapi: Error 403: Required 'compute.subnetworks.get' permission for 'projects/openshift-qe-shared-vpc/regions/us-central1/subnetworks/aos-qe-master-subnet', forbidden, requeuing in node workqueue
      E0511 13:13:50.231167       1 controller.go:165] error syncing 'jechen-0511a-49ntn-master-1.c.openshift-qe.internal': error retrieving the private IP configuration for node: jechen-0511a-49ntn-master-1.c.openshift-qe.internal, err: error retrieving the network interface subnets, err: googleapi: Error 403: Required 'compute.subnetworks.get' permission for 'projects/openshift-qe-shared-vpc/regions/us-central1/subnetworks/aos-qe-master-subnet', forbidden, requeuing in node workqueue
      

      Additional info:

      UPI non-XPN and IPI XPN clusters are still good, egress-ipconfig correctly present

              jluhrsen Jamo Luhrsen
              jechen@redhat.com Jean Chen
              Jean Chen Jean Chen
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: