Uploaded image for project: 'Fast Datapath Product'
  1. Fast Datapath Product
  2. FDP-1584

[Investigation spike] Senao SX904 PTL card: the system occur out of memory when running ovs dpdk nic partition case

    • Icon: Task Task
    • Resolution: Duplicate
    • Icon: Normal Normal
    • None
    • None
    • openvswitch3.5
    • None
    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • Hide

      Given the issue described in FDP-1530,

      When an engineer investigates and analyzes the logs,

      Then a comment should be added to this Jira ticket summarizing the root cause of these errors and potential solutions for the problem.

      Definition of Done

      ( ) Spend up to 2 hours investigating and analyzing the logs provided
      ( ) Write a comment in this ticket summarizing the root cause of the issue and suggesting fix or follow-up actions

      Show
      Given the issue described in FDP-1530 , When an engineer investigates and analyzes the logs, Then a comment should be added to this Jira ticket summarizing the root cause of these errors and potential solutions for the problem. Definition of Done ( ) Spend up to 2 hours investigating and analyzing the logs provided ( ) Write a comment in this ticket summarizing the root cause of the issue and suggesting fix or follow-up actions
    • rhel-9
    • None

      This ticket is created automatically to track an investigation spike related to the issue: FDP-1530.

      The goal is to identify the root cause or clarify unknowns and evaluate possible solutions or workarounds for the problem described below.

       Problem Description: Clearly explain the issue.

      Senao SX904 PTL card: the system occur out of memory when running ovs dpdk nic partition case
       

       Impact Assessment: Describe the severity and impact (e.g., network down,availability of a workaround, etc.).

       

       Software Versions: Specify the exact versions in use (e.g.,openvswitch3.1-3.1.0-147.el8fdp).

      openvswitch3.5-3.5.0-5.el9fdp
      dpdk-24.11.1-2.el9
      kernel-5.14.0-570.24.1.el9_6
       

        Issue Type: Indicate whether this is a new issue or a regression (if a regression, state the last known working version).

       

       Reproducibility: Confirm if the issue can be reproduced consistently. If not, describe how often it occurs.

      Cannot always reproduce
       

       Reproduction Steps: Provide detailed steps or scripts to replicate the issue.

      server: wsfd-advnetlab241-intel-ptl.anl.eng.bos2.dc.redhat.com E823 <---> client: wsfd-advnetlab241.anl.eng.bos2.dc.redhat.com E810
      Run ovs dpdk nic partition job:
      both server and client, create vf for two pf, and add vf to the ovs dpdk bridge, and start guest with vhostuser. and run ping test inside guest.
      The job contain following cases
      spoofchk_test
      tunnel_gre_test
      tunnel_geneve_test
      tunnel_vxlan_test
      tunnel_vlan_test
      qinq_test_dpdk
      vlan_test_dpdk1
      vlan_test_dpdk2
      vlan_test_dpdk3
      qos_test_dpdk
      novlan_test_dpdk
      mtu_test_dpdk

       

       Expected Behavior: Describe what should happen under normal circumstances.

      All cases work well
       

       Observed Behavior: Explain what actually happens.

      job:
      https://beaker.engineering.redhat.com/jobs/11341843
      According to the test log https://beaker-archive-prod.beak-001.prod.iad2.dc.redhat.com/beaker-logs/2025/07/113418/11341843/19076083/197600972/taskout.log, When run following command with mtu_test_dpdk case, the system is out of memory.
      ovs-vsctl set interface dpdkvhostuserclient0 mtu_request=2000

      However, due to running subsequent jobs, the system did not save the OVS log and SOS report at that time.
      I will upload some screenshots of the machine afterwards.
      I ran this job again later and did not reproduce the issue.
      https://beaker.engineering.redhat.com/jobs/11343865

       Troubleshooting Actions: Outline the steps taken to diagnose or resolve the issue so far.

       

       Logs: If you collected logs please provide them (e.g. sos report, /var/log/openvswitch/* , testpmd console)

              ovsdpdk-triage ovsdpdk triage
              nstbot NST Bot
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: