Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-2921

br-ex interface not configured makes ovnkube-node Pod to crashloop

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Normal
    • None
    • 4.12
    • None
    • Important
    • SDN Sprint 227, SDN Sprint 228, SDN Sprint 229, SDN Sprint 230, SDN Sprint 231, SDN Sprint 232, SDN Sprint 233, SDN Sprint 234
    • 8
    • Rejected
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      When installing an OpenShift cluster with OVN-Kubernetes in a specific OpenStack cloud used for some CI jobs we've notice the ovnkube-node Pod
      crashloops because "F1027 10:10:08.351527   31511 ovnkube.go:133] error looking up gw interface: "br-ex", error: Link not found". Installation is unable to progress. Two services have failed on the master nodes afterburn-hostname.service and ovs-configuration.service. 
      
      Additional information that might be helpful, the network of the nodes have a MTU of 1242 configured.
      
      DEVICE          TYPE           STATE      CONNECTION         
      ens3            ethernet       connected  Wired connection 1 
      genev_sys_6081  geneve         unmanaged  --                 
      lo              loopback       unmanaged  --                 
      br-int          ovs-bridge     unmanaged  --                 
      br-int          ovs-interface  unmanaged  --                 
      ovn-k8s-mp0     ovs-interface  unmanaged  --                 
      br-int          ovs-port       unmanaged  --                 
      ovn-e8fef9-0    ovs-port       unmanaged  --                 
      ovn-k8s-mp0     ovs-port       unmanaged  -- 
      
      [core@sqknth4w-71fac-gnwxk-master-1 ~]$ sudo systemctl status ovs-configuration.service
      ● ovs-configuration.service - Configures OVS with proper host networking configuration
         Loaded: loaded (/etc/systemd/system/ovs-configuration.service; enabled; vendor preset: enabled)
         Active: failed (Result: exit-code) since Thu 2022-10-27 09:52:51 UTC; 1h 22min ago
        Process: 1578 ExecStart=/usr/local/bin/configure-ovs.sh OVNKubernetes (code=exited, status=3)
       Main PID: 1578 (code=exited, status=3)
            CPU: 1.528s
      
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 configure-ovs.sh[5434]: default via 10.0.0.1 dev ens3 proto dhcp src 10.0.0.27 metric 100
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 configure-ovs.sh[5434]: 10.0.0.0/16 dev ens3 proto kernel scope link src 10.0.0.27 metric 100
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 configure-ovs.sh[5434]: 169.254.169.254 via 10.0.0.10 dev ens3 proto dhcp src 10.0.0.27 metric 100
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 configure-ovs.sh[1578]: + ip -6 route show
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 configure-ovs.sh[5435]: ::1 dev lo proto kernel metric 256 pref medium
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 configure-ovs.sh[1578]: + exit 3
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 systemd[1]: ovs-configuration.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 systemd[1]: ovs-configuration.service: Failed with result 'exit-code'.
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 systemd[1]: Failed to start Configures OVS with proper host networking configuration.
      Oct 27 09:52:51 sqknth4w-71fac-gnwxk-master-1 systemd[1]: ovs-configuration.service: Consumed 1.528s CPU time
      
      

      Version-Release number of selected component (if applicable):
      Our CI jobs are not getting detailed logs, but it seems they have started failing once we switched to use ovn-k by default OCT 10.
      https://prow.ci.openshift.org/job-history/origin-ci-test/logs/periodic-ci-shiftstack-shiftstack-ci-main-periodic-4.12-e2e-openstack-csi-cinder

      How reproducible:

       

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

       

      Expected results:

       

      Additional info:

       

       

      Attachments

        Activity

          People

            jcaamano@redhat.com Jaime Caamaño Ruiz
            mdemaced Maysa De Macedo Souza
            Ross Brattain Ross Brattain
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: