Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-9965

fails to access APIServer service IP assigned on lo device

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • 4.13.0, 4.14.0
    • 4.13.0
    • MicroShift
    • None
    • Critical
    • No
    • uShift Sprint 233, uShift Sprint 234
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      In order to solve the certificate issue: https://issues.redhat.com/browse/OCPBUGS-7442, we added the first kubernetes service IP (for APIServer service) to the lo device:
      
      $ ip a s lo
      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
          link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
          inet 127.0.0.1/8 scope host lo
             valid_lft forever preferred_lft forever
          inet 10.43.0.1/32 scope global lo
             valid_lft forever preferred_lft forever
          inet6 ::1/128 scope host 
             valid_lft forever preferred_lft forever
      
      This has side effect that ovnk may pick up this "node" IP from lo device as the backend of APIServer service, which in turn results in failure accessing the actual API pod (which runs in host network and uses the actual node IP).

      Version-Release number of selected component (if applicable):

      4.12, 4.13

      How reproducible:

      not always

      Steps to Reproduce:

      1. install microshift
      2. restart node
      3.
      

      Actual results:

      [redhat@dhcp-1-235-104 ~]$ ip a s lo
      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
          link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
          inet 127.0.0.1/8 scope host lo
             valid_lft forever preferred_lft forever
          inet 10.43.0.1/32 scope global lo
             valid_lft forever preferred_lft forever
          inet6 ::1/128 scope host 
             valid_lft forever preferred_lft forever
      
      [redhat@dhcp-1-235-104 ~]$ oc -n openshift-ovn-kubernetes exec -it ovnkube-master-tqtk9 -c northd -- bash
      
      [root@dhcp-1-235-104 ~]# ovn-nbctl ls-list
      85747199-e6b4-4523-ad6a-e24c4c2c9a7b (dhcp-1-235-104.arm.eng.rdu2.redhat.com)
      85bd04db-0df3-4fe0-b9f5-a322452db706 (ext_dhcp-1-235-104.arm.eng.rdu2.redhat.com)
      0ce5719f-7ac4-4670-9161-88cbd25fa236 (join)
      
      [root@dhcp-1-235-104 ~]# ovn-nbctl ls-lb-list dhcp-1-235-104.arm.eng.rdu2.redhat.com
      UUID                                    LB                  PROTO      VIP                   IPs
      c6eaad35-a683-463a-8abd-a4f17f493e37    Service_default/    tcp        10.43.0.1:443         10.43.0.1:6443
      
      
      
      Note:
      10.43.0.1:443 is the k8s service IP for APIServer
      10.43.0.1:6443 is the APIServer pod IP that serves the requests to APIServer service

      Expected results:

       

      Additional info:

       

              pacevedo@redhat.com Pablo Acevedo Montserrat
              zshi@redhat.com Zenghui Shi
              John George John George
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: