Uploaded image for project: 'Fast Datapath Product'
  1. Fast Datapath Product
  2. FDP-3012

Test Coverage: [NFV] Performance drop observed with small frame size tests on NetworkerNode OVS-DPDK

    • Icon: Task Task
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • ovs-dpdk
    • False
    • Hide

      None

      Show
      None
    • False
    • Hide

      ( ) The test coverage is aligned with the epic's acceptance criteria

      Show
      ( ) The test coverage is aligned with the epic's acceptance criteria
    • None

      This task is tracking the test case writing activities to cover the bug described below.

      I observed unexpectedly low throughput in a trafficgen benchmark using Networker node with OVS-DPDK with TRex as the generator. The binary search converged on a maximum sustainable rate of ~2.08% of line rate, yielding an aggregate of ~3.09 MPPS across 4 directions (bi-directional on two device pairs) with zero packet loss during a 120-second validation trial. However, attempts to push beyond this (e.g., 2.081%-2.099%) result in packet loss exceeding 0.002%.
      This ~3 MPPS aggregate seems very low for a DPDK-accelerated setup, especially with 64-byte UDP packets and isolated CPUs. 

      http://storage.scalelab.redhat.com/psahoo/PerfTaskLog/debuging/networkernode/bug/perfdrop/trafficgen--2025-08-26_06%3A20%3A01_UTC--bcc87a7a-6c85-41cb-8c85-a09d2efc9c44/run/iterations/iteration-1/sample-1/client/1/binary-search.txt.xz

      For context:
      Per-direction: ~0.77 MPPS, ~371 Mbps L2, ~519 Mbps L1.
      Aggregate L2 throughput: ~1.48 Gbps.

       

      Environment:

      Actual Results:

      • Max sustainable: ~3.09 MPPS aggregate with 0% loss.
      • Loss starts at ~2.081% rate, with symmetric drops in outbound/inbound directions.
      • No TX queue full in TRex, but potential OVS/VM bottlenecks (e.g., upcalls, PMD saturation).
      • Profiler gaps: Missing OVS files (e.g., pmd-perf-show, dpctl-dump-flows) indicate incomplete diagnostics.

      Attachments:

              ovs-qe Openvswtich Quality Engineering Bot
              rhn-support-psahoo Pradipta Sahoo
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: