Description of problem:

Version-Release number of selected component (if applicable):
Both rhel9.2/9.4 and rhel 8.6
openvswitch3.2-test-3.2.0-63.el9fdp.noarch

How reproducible:

Run ovs dpdk vhostuser 1queue 4pmd case.
Steps to Reproduce:

750-37 10g ixgbe <-directly connect-> 740-55 10g i40e driver
on 750-37 side:
1. Build ovs dpdk topo with vf port
echo 1 > /sys/devices/pci0000:16/0000:16:02.0/0000:17:00.0/sriov_numvfs
echo 1 > /sys/devices/pci0000:16/0000:16:02.0/0000:17:00.1/sriov_numvfs
ip link set ens2f0 vf 0 trust on
ip link set ens2f1 vf 0 trust on
ip link set ens2f0 vf 0 spoofchk off
ip link set ens2f1 vf 0 spoofchk off

[root@dell-per750-37 ~]# ip link show
6: ens2f0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 9200 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 90:e2:ba:29:bf:cc brd ff:ff:ff:ff:ff:ff
vf 0 link/ether 02:09:c0:a3:e3:8c brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust on, query_rss off
altname enp23s0f0
7: ens2f1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 9200 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 90:e2:ba:29:bf:cd brd ff:ff:ff:ff:ff:ff
vf 0 link/ether 02:09:c0:de:5b:3e brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust on, query_rss off
altname enp23s0f1

driverctl -v set-override 0000:17:10.0 vfio-pci
driverctl -v set-override 0000:17:10.1 vfio-pci
systemctl start openvswitch
ovs-vsctl set Open_vSwitch . 'other_config={}'
ovs-vsctl set Open_vSwitch . other_config:dpdk-socket-mem=8192
ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=40000000004000000000
ovs-vsctl set Open_vSwitch . other_config:dpdk-lcore-mask=0x1
ovs-vsctl set Open_vSwitch . other_config:vhost-iommu-support=true
ovs-vsctl set Open_vSwitch . other_config:userspace-tso-enable=false
ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true
ovs-vsctl -no-wait set Open_vSwitch . 'other_config:dpdk-extra=-log-level pmd.net.bnxt.driver:info'
ovs-vsctl --if-exists del-br ovsbr0
ovs-vsctl --may-exist add-br ovsbr0 – set bridge ovsbr0 datapath_type=netdev
ovs-vsctl add-port ovsbr0 dpdk0 – set Interface dpdk0 type=dpdk options:dpdk-devargs=0000:17:10.0 options:n_rxq=1 options:n_rxq_desc=2048 options:n_txq_desc=2048 mtu_request=9200
ovs-vsctl add-port ovsbr0 dpdk1 – set Interface dpdk1 type=dpdk options:dpdk-devargs=0000:17:10.1 options:n_rxq=1 options:n_rxq_desc=2048 options:n_txq_desc=2048 mtu_request=9200
systemctl restart openvswitch
ovs-ofctl del-flows ovsbr0
ovs-ofctl add-flow ovsbr0 'in_port=dpdk0 actions=output:dpdk1'
ovs-ofctl add-flow ovsbr0 'in_port=dpdk1 actions=output:dpdk0'

On 740-55 side:
send traffic on T-rex server
[root@dell-per740-55 trafficgen]# ./binary-search.py --traffic-generator=trex-txrx --frame-size=64 --num-flows=1024 --max-loss-pct=0 --search-runtime=10 --validation-runtime=60 --rate-tolerance=10 --runtime-tolerance=10 --rate=25 --rate-unit=% --duplicate-packet-failure=retry-to-fail --negative-packet-loss=retry-to-fail --warmup-trial --warmup-trial-runtime=10 --rate=100 --rate-unit=% --one-shot=0 --use-src-ip-flows=1 --use-dst-ip-flows=1 --use-src-mac-flows=1 --use-dst-mac-flows=1 --send-teaching-measurement --send-teaching-warmup --teaching-warmup-packet-type=generic --teaching-warmup-packet-rate=1000

Actual results:
It cannot got the throughtput.
[root@dell-per750-37 perf]# ovs-tcpdump -i dpdk0 -vven
s 1.1.5.0 (Broadcast) tell 1.1.5.0, length 46
04:19:45.554297 00:00:00:03:fc:02 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 2.2.5.254 is-at 00:00:00:03:fc:02, length 46
04:19:45.554299 00:00:00:03:fc:02 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 2.2.5.254 (Broadcast) tell 2.2.5.254, length 46
04:19:45.554827 00:00:00:03:fd:02 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 2.2.5.255 is-at 00:00:00:03:fd:02, length 46
04:19:45.555750 00:00:00:03:fe:02 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 2.2.6.0 is-at 00:00:00:03:fe:02, length 46

[root@dell-per750-37 perf]# ovs-tcpdump -i dpdk1 -vven
s 2.2.4.192 (Broadcast) tell 2.2.4.192, length 46
04:20:40.210896 00:00:00:02:c2:01 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 1.1.3.195 is-at 00:00:00:02:c2:01, length 46
04:20:40.210897 00:00:00:02:c2:01 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 1.1.3.195 (Broadcast) tell 1.1.3.195, length 46
04:20:40.211569 00:00:00:02:bf:02 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 2.2.4.193 is-at 00:00:00:02:bf:02, length 46
04:20:40.211571 00:00:00:02:bf:02 > Broadcast, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 2.2.4.193 (Broadcast) tell 2.2.4.193, length 46

[root@dell-per750-37 perf]# ovs-ofctl dump-flows ovsbr0
cookie=0x0, duration=1758.442s, table=0, n_packets=5366, n_bytes=321960, in_port=dpdk0 actions=output:dpdk1
cookie=0x0, duration=1758.439s, table=0, n_packets=5455, n_bytes=327300, in_port=dpdk1 actions=output:dpdk0

rhel9.4 job: https://beaker.engineering.redhat.com/jobs/9242966
rhel9.2 job:https://beaker.engineering.redhat.com/jobs/9247569
rhel 8.6 job:https://beaker.engineering.redhat.com/jobs/9252062