Uploaded image for project: 'Fast Datapath Product'
  1. Fast Datapath Product
  2. FDP-1273

With pmd-maxsleep set, OVS MAC tables on client OVS DPDK not updating correctly upon F5 virtual Load Balancers HA switchover

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • openvswitch3.3
    • None
    • 8
    • False
    • Hide

      None

      Show
      None
    • False
    • rhel-9
    • None
    • rhel-net-ovs-dpdk
    • ssg_networking
    • OVS/DPDK - FDP-25.E - 1, FDP-OVS/DPDK Sprint 7
    • 2

      In RHOSP 17.1 system, customer has a setup that is using 2 x F5 Load Balancer instances (vLB's) in Active/Standby mode:
      host dpdk-0 <<< F5 vLB A
      host dpdk-1 <<< F5 vLB B

      Host dpdk-2 is used for iXIA Servers.
      Host dpdk-3 is used for iXIA Clients.

      All hosts are set up with option pmd-maxsleep.

      iXIA Clients are sending traffic (and getting replies) to iXIA Servers via the active vLB.

      The active vLB is owner of a "masquerade MAC".
      Upon switchover, the newly active vLB becomes owner of the masquerade MAC and issues a GARP to announce this change.
      Masquerade MAC is an F5 proprietary mechanism which is somewhat the MAC equivalent of a virtual IP address (VIP).

      In switchover test, it takes 5 minutes before we can see traffic getting directed to the nely active vLB.

      When pmd-maxsleep is removed, failover scenario works as expected
      ^^^^^^^

      Note that this looks very much like https://bugzilla.redhat.com/show_bug.cgi?id=2213892
      with the difference that pmd-maxsleep is enabled.

      How reproducible:
      Always

      Steps to Reproduce:
      Force vLB switchover.

      Actual results:
      What is observed on host dpdk-3 (hosting iXIA Clients):

      • using ovs-tcpdump, we see the GARP from newly active vLB is received by OVS DPDK.
      • OVS FDB table doesn't get updated.

      What is observed on hosts dpdk-0 and dpdk-1:

      • the vLB that becomes active issues the GARP
      • FDB tables on both hosts get updated with new masquerade MAC port info.

      Expected results:
      What is expected is that upon receiving the GARP, OVS FDB tables on all compute hosts get updated.

      Additional info:
      RHOSP 17.1.4
      openvswitch3.3-3.3.0-62.el9fdp.x86_64

      other_config :

      {dpdk-extra=" -n 4", dpdk-init="true", dpdk-socket-limit="4096,4096", dpdk-socket-mem="4096,4096", emc-insert-inv-prob="0", pmd-auto-lb="true", pmd-auto-lb-improvement-threshold="80", pmd-auto-lb-load-threshold="50", pmd-cpu-mask="2a8000000000002a8", pmd-rxq-assign=group, pmd-rxq-isolate="false", shared-mempool-config="9100:0,9100:1", smc-enable="true", vhost-postcopy-support="true"}

      Note that pmd-maxsleep is set/unset (on all 4 hosts) from the command line:
      ~~~
      Mar 31 18:02:54 tpa-vim-c-computedpdkcl-3 ovs-vsctl[632387]: ovs|00001|vsctl|INFO|Called as ovs-vsctl set Open_vSwitch . other_config:pmd-maxsleep=220
      Mar 31 18:47:20 tpa-vim-c-computedpdkcl-3 ovs-vsctl[914285]: ovs|00001|vsctl|INFO|Called as ovs-vsctl remove Open_vSwitch . other_config pmd-maxsleep
      ~~~

              rh-ee-mpattric Mike Pattrick
              rhn-support-fpalin Francois Palin
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: