Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-49662

ovnkube-node pods crash after restarting when NAD is configured on cluster

XMLWordPrintable

    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      This bug was found while verifying another bug https://issues.redhat.com/browse/OCPBUGS-48412

      Version-Release number of selected component (if applicable):

      4.18.0

      build 4.18.0-0.nightly-2025-01-30-093109, openshift/api#2127

      How reproducible:

      Always

      Steps to Reproduce:

      1. Create a UDN namespace
      —
      apiVersion: v1
      kind: Namespace
      metadata:
        name: ns1
        labels:
          k8s.ovn.org/primary-user-defined-network:

      2. Create L3 NAD in namespace

      apiVersion: k8s.cni.cncf.io/v1
      kind: NetworkAttachmentDefinition
      metadata:
        name: l3-network-ns1
        namespace: ns1
      spec:
        config: |2
          {
                  "cniVersion": "0.3.1",
                  "name": "l3-network-ns1",
                  "type": "ovn-k8s-cni-overlay",
                  "topology":"layer3",
                  "subnets": "10.20.0.0/16/24",
                  "mtu": 1300,
                  "netAttachDefName": "ns1/l3-network-ns1",
                  "role": "primary"
          }

      3. Restart ovnkube-node pods

      Actual results:

      oc -n openshift-ovn-kubernetes get pods

      NAME                                     READY   STATUS             RESTARTS        AGE
      ovnkube-control-plane-65d7c9ddf4-blqsb   2/2     Running            0               86m
      ovnkube-control-plane-65d7c9ddf4-pc67l   2/2     Running            0               86m
      ovnkube-node-2fjt7                       7/8     CrashLoopBackOff   10 (114s ago)   28m
      ovnkube-node-64h2p                       7/8     CrashLoopBackOff   10 (94s ago)    28m
      ovnkube-node-7w2vx                       7/8     CrashLoopBackOff   10 (113s ago)   28m
      ovnkube-node-c4r2z                       7/8     CrashLoopBackOff   10 (111s ago)   28m
      ovnkube-node-djq2w                       7/8     CrashLoopBackOff   10 (73s ago)    28m
      ovnkube-node-gclwv                       7/8     CrashLoopBackOff   10 (101s ago)   28m
       
      

      Expected results:

      The pods should be restarted without any issue

      Additional info:

      Error found on ovnkube-controller of ovnkube-node pod

      0130 15:28:02.898200   18833 ovnkube.go:137] failed to run ovnkube: [failed to start network controller: failed to start default network controller: error running OVN Kubernetes Services controller: handler {0x1e34440 0x1e34120 0x1e340c0} was not added to shared informer because it has stopped already, failed to start node network controller: failed to start NAD controller: initial sync failed: failed to sync network l3-network-ns1: [node-nad-controller network controller]: failed to ensure network l3-network-ns1: failed to create network l3-network-ns1: error creating UDN gateway for network l3-network-ns1: failed to get v4 masquerade IP, network l3-network-ns1 (1): failed generating network id '1' v4-masquerade-ips gateway router ip: generated ip 169.254.169.11 from the idx 11 is out of range in the network 169.254.169.0/29]

       

              rhn-support-arghosh Arnab Ghosh
              rhn-support-asood Arti Sood
              Arti Sood Arti Sood
              Felix Enrique Llorente Pastora
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated: