Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-35317

Unable to select DNS server with OVN Multi-NIC baremetal install

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.16
    • None
    • Moderate
    • No
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      For OCPBUGS-27821 customers are installing with two networks, one with a default route, and the other network without a default route, isolated, for use with OVN.

      We can specify which interface to use with /etc/default/nodeip-configuration and /etc/ovnk/default_bridge, but we can't force a DNS server if there are multiple DNS servers present.

      If a DNS server is configured on the non-default-gateway route and we select that DNS server in resolv-prepender.sh then we will be unable to pull the baremetal runtime cfg.

      resolv-prepender.sh[2312]: nameserver 192.168.124.1
      resolv-prepender.sh[2289]: NM resolv-prepender: Checking if baremetal runtime cfg image already exists
      NetworkManager[2162]: <debug> [1718029725.3170] dns-mgr: (device_l3cd_changed): committing DNS changes (0)
      NetworkManager[2162]: <debug> [1718029725.3170] dns-mgr: update-dns: not updating resolv.conf
      NetworkManager[2162]: <trace> [1718029725.3170] dns-mgr: config:      100 best    v4 3     : 192.168.123.1
      NetworkManager[2162]: <trace> [1718029725.3170] dns-mgr: config:      100 default v4 4     : 192.168.124.1
      NetworkManager[2162]: <trace> [1718029725.3170] dns-mgr: config:      100 default v6 4     :
      NetworkManager[2162]: <trace> [1718029725.3170] dns-mgr: config:      100 default v4 2     :
      NetworkManager[2162]: <trace> [1718029725.3170] dns-mgr: config:      100 default v6 2     :
      NetworkManager[2162]: <trace> [1718029725.3170] dns-mgr: config:      100 default v6 3     :
      NetworkManager[2162]: <trace> [1718029725.3173] dns-mgr: current configuration: [{'nameservers': <['192.168.123.1']>, 'domains': <['o16r1s-0.qe.lab.redhat.com']>, 'interface': <'enp5s0'>, 'priority': <100>, 'vpn': <false>}, {'nameservers': <['192.168.124.1']>, 'interface': <'enp6s0'>, 'priority': <100>, 'vpn': <false>}]
      resolv-prepender.sh[2356]: Error: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2da4ec5170f24df7c30d1c9182031d141347f2855fd9802b6948ab142499a64: pinging container registry quay.io: Get "https://quay.io/v2/": dial tcp: lookup quay.io on 192.168.124.1:53: server misbehaving 
      

      Version-Release number of selected component (if applicable):

      4.16.0-0.nightly-2024-06-11-053715
      

      How reproducible:

      Sometimes

      Steps to Reproduce:

      1. Create  two networks, one with default route, one without.
      2. DHCP advertise a DNS server on both networks
      3. Configure OVN  /etc/ovnk/default_bridge and if necessary /etc/default/nodeip-configuration to use the correct interface.
      4. Enable OVN "routingViaHost: true" and "ipForwarding: Global"
      5. create cluster
      

      Actual results:

      Intermittently the DNS server on the non-default route network will be selected at boot.
      quay.io will be inaccessible.

      resolv-prepender.sh[2356]: Error: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a2da4ec5170f24df7c30d1c9182031d141347f2855fd9802b6948ab142499a64: pinging container registry quay.io: Get "https://quay.io/v2/": dial tcp: lookup quay.io on 192.168.124.1:53: server misbehaving 
      

      Expected results:

      We should be able to specify a DNS server on the correct network, or have way of checking which DNS server is correct in a mutli-network setup.

      Additional info:

      Network config

      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
           link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0  allmulti 0 minmtu 0 maxmtu 0 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 524280 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
           inet 127.0.0.1/8 scope host lo
              valid_lft forever preferred_lft forever
           inet6 ::1/128 scope host
              valid_lft forever preferred_lft forever
       2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
           link/ether 52:54:00:63:2c:25 brd ff:ff:ff:ff:ff:ff promiscuity 0  allmulti 0 minmtu 68 maxmtu 9212 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536 parentbus pci parentdev 0000:04:00.0
           inet6 fe80::5054:ff:fe63:2c25/64 scope link noprefixroute
              valid_lft forever preferred_lft forever
       3: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
           link/ether 52:54:00:77:a0:a5 brd ff:ff:ff:ff:ff:ff promiscuity 0  allmulti 0 minmtu 68 maxmtu 9212 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536 parentbus pci parentdev 0000:05:00.0
           inet 192.168.123.132/24 brd 192.168.123.255 scope global dynamic noprefixroute enp5s0
              valid_lft 3533sec preferred_lft 3533sec
           inet6 fd2e:6f44:5dd8::58/128 scope global dynamic noprefixroute
              valid_lft 3535sec preferred_lft 3535sec
           inet6 fe80::5054:ff:fe77:a0a5/64 scope link noprefixroute
              valid_lft forever preferred_lft forever
       4: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000
           link/ether 52:54:00:83:8b:22 brd ff:ff:ff:ff:ff:ff promiscuity 1  allmulti 0 minmtu 68 maxmtu 9212
      nfigures OVS with proper host networking configuration.
           openvswitch_slave numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536 parentbus pci parentdev 0000:06:00.0
       6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
           link/ether 8a:c3:3b:59:5a:9d brd ff:ff:ff:ff:ff:ff promiscuity 1  allmulti 0 minmtu 68 maxmtu 65535
           openvswitch numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
       7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
           link/ether 52:54:00:83:8b:22 brd ff:ff:ff:ff:ff:ff promiscuity 1  allmulti 0 minmtu 68 maxmtu 65535
           openvswitch numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536
           inet 192.168.124.57/24 brd 192.168.124.255 scope global dynamic noprefixroute br-ex
              valid_lft 3595sec preferred_lft 3595sec
           inet6 fd2e:6f44:5de8::6e/128 scope global dynamic noprefixroute
              valid_lft 3597sec preferred_lft 3597sec
           inet6 fe80::5054:ff:fe83:8b22/64 scope link noprefixroute
              valid_lft forever preferred_lft forever
       ++ ip route show
       default via 192.168.123.1 dev enp5s0 proto dhcp src 192.168.123.132 metric 101
       192.168.123.0/24 dev enp5s0 proto kernel scope link src 192.168.123.132 metric 101
       192.168.124.0/24 dev br-ex proto kernel scope link src 192.168.124.57 metric 48
      
      ovnKubernetesConfig:
              egressIPConfig: {}
              gatewayConfig:
                ipForwarding: Global
                ipv4: {}
                ipv6: {}
                routingViaHost: true
      

              bnemec@redhat.com Benjamin Nemec
              rbrattai@redhat.com Ross Brattain
              Ross Brattain Ross Brattain
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: