Uploaded image for project: 'Fast Datapath Product'
  1. Fast Datapath Product
  2. FDP-2121

Incorrect TCP segmentation over tunnel with VDUSE devices

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • dpdk, openvswitch3.5
    • None
    • 5
    • False
    • False
    • rhel-9
    • None
    • rhel-net-ovs-dpdk
    • ssg_networking
    • OVS/DPDK - Sprint 11 - East
    • 1

      It was noticed while benchmarking TSO performance (and the FDP-1897 optimisation) between two OCP Virt virtual machines backed by VDUSE.
      The traffic is exchanged over a IPv4 geneve tunnel.

      /--------\    /-----------------------\    /-------------------------\    /--------\
      |  VM0   |    |         host0         |    |         host1           |    |  VM1   |
      |        |    |                       |    |                         |    |        |
      | ens9 --+----+-- vduse0 -- br-int    |    |      br-int -- vduse0 --+----+-- ens9 |
      |        |    |                       |    |                         |    |        |
      |        |    |      br-ex -- dpdk0 --+----+-- dpdk0 -- br-ex        |    |        |
      |        |    |                       |    |                         |    |        |
      \--------/    \-----------------------/    \-------------------------/    \--------/
      

      From a user pov, iperf throughput test result was erratic, and reaching 0 quickly:

      On the client side:

      Connecting to host 192.168.24.20, port 5201
      Reverse mode, remote host 192.168.24.20 is sending
      [  5] local 192.168.24.10 port 44522 connected to 192.168.24.20 port 5201
      [ ID] Interval           Transfer     Bitrate
      [  5]   0.00-10.01  sec  89.5 KBytes  73.3 Kbits/sec                  
      - - - - - - - - - - - - - - - - - - - - - - - - -
      [ ID] Interval           Transfer     Bitrate         Retr
      [  5]   0.00-10.05  sec   282 KBytes   230 Kbits/sec    5             sender
      [  5]   0.00-10.01  sec  89.5 KBytes  73.3 Kbits/sec                  receiver
      
      iperf Done.
      

      On the server side:

      -----------------------------------------------------------
      Server listening on 5201
      -----------------------------------------------------------
      Accepted connection from 192.168.24.10, port 56592
      [  5] local 192.168.24.20 port 5201 connected to 192.168.24.10 port 44522
      [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
      [  5]   0.00-1.00   sec   282 KBytes  2.31 Mbits/sec    2   1.32 KBytes       
      [  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.32 KBytes       
      [  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.32 KBytes       
      [  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.32 KBytes       
      [  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    1   1.32 KBytes       
      [  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   1.32 KBytes       
      [  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.32 KBytes       
      [  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    1   1.32 KBytes       
      [  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   1.32 KBytes       
      [  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.32 KBytes       
      [  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   1.32 KBytes       
      - - - - - - - - - - - - - - - - - - - - - - - - -
      [ ID] Interval           Transfer     Bitrate         Retr
      [  5]   0.00-10.05  sec   282 KBytes   230 Kbits/sec    5             sender
      -----------------------------------------------------------
      

      Observing the bug can be done with dumping the initial 50 first packets of the TCP iperf data session, on the receiving (this is important) physical port with ovs-tcpdump. mlx5 nics were used in this setup. It is expected that other nics will have similar issue, though it has not been confirmed.

      Debug steps are listed in a doc:
      https://docs.google.com/document/d/1V8n5J5XvzIopX1ESkTz-A9mBRDCet-TIiRWkL1OZVgU/edit?tab=t.0

              rhn-support-dmarchan David Marchand
              rhn-support-dmarchan David Marchand
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: