Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-67284

Missing IP on bond1 Interface in OpenShift Cluster node

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Critical
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

          Observed Missing IP on bond1 Interface in OpenShift Cluster node with OCP 4.18.22, this problem was not present in previous OCP versions, whereas in the NM connection file IP shown as present, In the nm connection file ip is present as 169.253.1.53 and metal3 pod is keep assigning the ip as 169.253.0.2 i have shared the details in additional info section.

      Version-Release number of selected component (if applicable):

          

      How reproducible:

          Every time with OCP 4.18.22

      Steps to Reproduce:

          1.
          2.
          3.
          

      Actual results:

          Bond1 IP is missing 

      Expected results:

          IP should be assigned on Bond1 

      Additional info:

         Some important configurations.
      $ cat etc/NetworkManager/system-connections/bond1.nmconnection 
      [connection]
      autoconnect=true
      autoconnect-slaves=1
      id=bond1
      interface-name=bond1
      type=bond
      uuid=c3ab39d5-aada-51a1-abba-473f2352b219
      
      [bond]
      lacp_rate=fast
      miimon=140
      mode=802.3ad
      xmit_hash_policy=layer3+4
      
      [ipv4]
      address0=169.253.1.53/16
      
      metal3 pod logs:
      2025-12-08T15:41:14.141289017Z + '[' -z 169.253.0.2/16 ']'
      2025-12-08T15:41:14.141289017Z + '[' -z bond1 ']'
      2025-12-08T15:41:14.141289017Z + '[' -z bond1 ']'
      2025-12-08T15:41:14.141432605Z + /usr/sbin/ip addr add 169.253.0.2/16 dev bond1 valid_lft 10 preferred_lft 10  <--------------------
      2025-12-08T15:41:14.143242992Z Error: ipv4: Address already assigned.
      2025-12-08T15:41:14.143471558Z + true
      2025-12-08T15:41:14.143498589Z + true
      2025-12-08T15:41:14.143934903Z ++ ip -o -6 address show dev bond1 scope link
      2025-12-08T15:41:14.145474865Z + '[' '' = '' ']'
      2025-12-08T15:41:14.145474865Z + echo 1
      2025-12-08T15:41:14.145617441Z + echo 0
      2025-12-08T15:41:14.145643610Z + /usr/sbin/ip addr change 169.253.0.2/16 dev bond1 valid_lft 10 preferred_lft 10
      2025-12-08T15:41:14.147136275Z + [[ 169.253.0.2/16 =~ : ]]
      2025-12-08T15:41:14.147154409Z + sleep 5
      2025-12-08T15:41:19.150205890Z + true
      2025-12-08T15:41:19.150755084Z ++ ip -o -6 address show dev bond1 scope link
      2025-12-08T15:41:19.153048402Z + '[' '' = '' ']'
      2025-12-08T15:41:19.153048402Z + echo 1
      2025-12-08T15:41:19.153112021Z + echo 0
      2025-12-08T15:41:19.153142067Z + /usr/sbin/ip addr change 169.253.0.2/16 dev bond1 valid_lft 10 preferred_lft 10
      2025-12-08T15:41:19.155792782Z + [[ 169.253.0.2/16 =~ : ]]
      2025-12-08T15:41:19.155792782Z + sleep 5
      2025-12-08T15:41:24.157901205Z + true
      2025-12-08T15:41:24.158533474Z ++ ip -o -6 address show dev bond1 scope link
      2025-12-08T15:41:24.160570144Z + '[' '' = '' ']'
      2025-12-08T15:41:24.160570144Z + echo 1
      2025-12-08T15:41:24.160663167Z + echo 0
      2025-12-08T15:41:24.160692722Z + /usr/sbin/ip addr change 169.253.0.2/16 dev bond1 valid_lft 10 preferred_lft 10
      2025-12-08T15:41:24.162947188Z + [[ 169.253.0.2/16 =~ : ]]
      2025-12-08T15:41:24.162961094Z + sleep 5
      2025-12-08T15:41:29.165414820Z + true
      
      Provisioning configuration:
      $ oc get provisioning provisioning-configuration -o yaml
      apiVersion: metal3.io/v1alpha1
      kind: Provisioning
      metadata:
        creationTimestamp: "2025-12-08T07:18:35Z"
        finalizers:
        - provisioning.metal3.io
        generation: 2
        name: provisioning-configuration
        resourceVersion: "106171"
        uid: 810ce527-ed25-4935-8b47-6bcd92308335
      spec:
        preProvisioningOSDownloadURLs: {}
        provisioningDHCPRange: 169.253.250.0,169.253.252.254
        provisioningIP: 169.253.0.2
        provisioningInterface: bond1
        provisioningMacAddresses:
        - b8:ce:f6:08:49:de
        - b8:ce:f6:0d:1d:bc
        - b8:ce:f6:47:26:da
        provisioningNetwork: Managed              
        provisioningNetworkCIDR: 169.253.0.0/16
      
      Network Secret from the impacted node :
      
      interfaces:
      - description: Bond connection enslaving baremetal interfaces ens3f0np0 and ens3f1np1
        link-aggregation:
          mode: 802.3ad
          options:
            lacp_rate: "1"
            miimon: "140"
            xmit_hash_policy: "1"
          ports:
          - ens3f0np0
          - ens3f1np1
        mac-address: b8:ce:f6:68:06:0e
        mtu: "9000"
        name: bond0
        state: up
        type: bond
      - description: vlan connection on top of bond0 for baremetal primary interface
        ipv4:
          dhcp: true
          enabled: true
        ipv6:
          enabled: false
        name: bond0.1522
        state: up
        type: vlan
        vlan:
          base-iface: bond0
          id: 1522
      - description: Bond connection enslaving ens1f0np0 and ens1f1np1
        ipv4:
          address:
          - ip: 169.253.1.53
            prefix-length: 16
          dhcp: false
          enabled: true
        ipv6:
          enabled: false
        link-aggregation:
          mode: 802.3ad
          options:
            lacp_rate: "1"
            miimon: "140"
            xmit_hash_policy: "1"
          ports:
          - ens1f0np0
          - ens1f1np1
        name: bond1
        state: up
        type: bond
      - name: ens4f0
        state: absent
      - name: ens4f1
        state: absent
      - name: ens4f2
        state: absent
      - name: ens4f3
        state: absent 
      
      bond1 is correctly setup at boot (ip 169.253.1.53/16)Dec 08 07:40:59 control-1-ru2.f23l037.fusion.tadn.ibm.com kernel: bond1: (slave ens1f0np0): Enslaving as a backup interface with an up link
      Dec 08 07:40:59 control-1-ru2.f23l037.fusion.tadn.ibm.com NetworkManager[2276]: <info>  [1765179659.8213] device (bond1): attached bond port ens1f0np0
      Dec 08 07:41:00 control-1-ru2.f23l037.fusion.tadn.ibm.com kernel: bond1: (slave ens1f1np1): Enslaving as a backup interface with an up link
      Dec 08 07:41:00 control-1-ru2.f23l037.fusion.tadn.ibm.com NetworkManager[2276]: <info>  [1765179660.8733] device (bond1): attached bond port ens1f1np1
      Dec 08 07:41:01 control-1-ru2.f23l037.fusion.tadn.ibm.com NetworkManager[2276]: <info>  [1765179661.6485] device (bond1): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
      Dec 08 07:41:01 control-1-ru2.f23l037.fusion.tadn.ibm.com NetworkManager[2276]: <info>  [1765179661.6487] device (bond1): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
      Dec 08 07:41:01 control-1-ru2.f23l037.fusion.tadn.ibm.com NetworkManager[2276]: <info>  [1765179661.6491] device (bond1): Activation: successful, device activated.
      
      after "configure-ovs.sh" has finished during the boot process, the interface remains with the same configuration (correct and expected)Dec 08 07:42:02 control-1-ru2.f23l037.fusion.tadn.ibm.com configure-ovs.sh[4777]: 13: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
      Dec 08 07:42:02 control-1-ru2.f23l037.fusion.tadn.ibm.com configure-ovs.sh[4777]:     link/ether b8:ce:f6:0d:1d:bc brd ff:ff:ff:ff:ff:ff promiscuity 0  allmulti 0 minmtu 68 maxmtu 65535
      Dec 08 07:42:02 control-1-ru2.f23l037.fusion.tadn.ibm.com configure-ovs.sh[4777]:     bond mode 802.3ad miimon 140 updelay 0 downdelay 0 peer_notify_delay 0 use_carrier 1 arp_interval 0 arp_misse
      d_max 2 arp_validate none arp_all_targets any primary_reselect always fail_over_mac none xmit_hash_policy layer3+4 resend_igmp 1 num_grat_arp 1 all_slaves_active 0 min_links 0 lp_interval 1 packe
      ts_per_slave 1 lacp_active on lacp_rate fast ad_select stable ad_aggregator 1 ad_num_ports 2 ad_actor_key 21 ad_partner_key 21 ad_partner_mac 44:38:39:ff:00:25 ad_actor_sys_prio 65535 ad_user_por
      t_key 0 ad_actor_system 00:00:00:00:00:00 tlb_dynamic_lb 1 numtxqueues 16 numrxqueues 16 gso_max_size 65536 gso_max_segs 65535 tso_max_size 524280 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_m
      ax_size 65536 gro_ipv4_max_size 65536
      Dec 08 07:42:02 control-1-ru2.f23l037.fusion.tadn.ibm.com configure-ovs.sh[4777]:     inet 169.253.1.53/16 brd 169.253.255.255 scope global noprefixroute bond1
      Dec 08 07:42:02 control-1-ru2.f23l037.fusion.tadn.ibm.com configure-ovs.sh[4777]:        valid_lft forever preferred_lft forever

              rhn-engineering-dtantsur Dmitry Tantsur
              rhn-support-ashsharm Ashish Sharma
              None
              None
              Jad Haj Yahya Jad Haj Yahya
              None
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: