-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
False
-
-
False
-
rhel-sst-network-fastdatapath
-
-
-
ssg_networking
Description of problem:
The guest run ping test failed after do netperf test with 8193byte on rhel9.2
Version-Release number of selected component (if applicable):
[root@dell-per750-37 ~]# uname -r
5.14.0-284.10.1.el9_2.x86_64
[root@dell-per750-37 ~]# rpm -qa|grep openvswitch
openvswitch-selinux-extra-policy-1.0-31.el9fdp.noarch
openvswitch2.17-2.17.0-87.el9fdp.x86_64
How reproducible:
Steps to Reproduce:
SERVER: dell750-37
CLIENT: dell740-55
ixgbe dell750-37 <connect> i40e dell740-55
On server side:
1. Build following ovs topo:
bind the pf to vfi-pci and add the ovs bridge. and add dpdkvhostuserclient0 to ovs bridge too.
[root@dell-per750-37 ~]# ovs-vsctl show
27e5be3e-c9e3-4941-8321-aeda5f851dd7
Bridge ovsbr0
datapath_type: netdev
Port dpdk0
Interface dpdk0
type: dpdk
options:
Port ovsbr0
Interface ovsbr0
type: internal
Port dpdkvhostuserclient0
Interface dpdkvhostuserclient0
type: dpdkvhostuserclient
options:
ovs_version: "2.17.7"
2. configure mtu to 9000
ovs-vsctl set Interface dpdkvhostuserclient0 mtu_request=9000
ovs-vsctl set Interface dpdk0 mtu_request=9000
2. Start guest with following xml
[root@dell-per750-37 ~]# virsh dumpxml master
<domain type='kvm' id='4'>
<name>master</name>
<uuid>37425e76-af6a-44a6-aba0-73434afe34c0</uuid>
<memory unit='KiB'>5242880</memory>
<currentMemory unit='KiB'>5242880</currentMemory>
<memoryBacking>
<hugepages>
<page size='1048576' unit='KiB'/>
</hugepages>
<access mode='shared'/>
</memoryBacking>
<vcpu placement='static'>3</vcpu>
<cputune>
<vcpupin vcpu='2' cpuset='0'/>
<emulatorpin cpuset='2'/>
</cputune>
<numatune>
<memory mode='strict' nodeset='0'/>
</numatune>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='x86_64' machine='pc-q35-rhel9.2.0'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pmu state='off'/>
<vmport state='off'/>
<ioapic driver='qemu'/>
</features>
<cpu mode='host-passthrough' check='none' migratable='on'>
<feature policy='require' name='tsc-deadline'/>
<numa>
<cell id='0' cpus='0-2' memory='5242880' unit='KiB' memAccess='shared'/>
</numa>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/home/master.qcow2' index='1'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</disk>
<controller type='usb' index='0' model='none'>
<alias name='usb'/>
</controller>
<controller type='pci' index='0' model='pcie-root'>
<alias name='pcie.0'/>
</controller>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x10'/>
<alias name='pci.1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x11'/>
<alias name='pci.2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0x8'/>
<alias name='pci.3'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0x9'/>
<alias name='pci.4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0xa'/>
<alias name='pci.5'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</controller>
<controller type='pci' index='6' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0xb'/>
<alias name='pci.6'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</controller>
<controller type='sata' index='0'>
<alias name='ide'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
</controller>
<interface type='vhostuser'>
<mac address='52:54:00:11:8f:e8'/>
<source type='unix' path='/tmp/dpdkvhostuserclient0' mode='server'/>
<target dev='dpdkvhostuserclient0'/>
<model type='virtio'/>
<driver name='vhost' iommu='on' ats='on'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</interface>
<interface type='bridge'>
<mac address='52:54:00:bb:63:7b'/>
<source bridge='virbr0'/>
<target dev='vnet3'/>
<model type='virtio'/>
<alias name='net1'/>
<address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</interface>
<serial type='pty'>
<source path='/dev/pts/1'/>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
<alias name='serial0'/>
</serial>
<serial type='pty'>
<source path='/dev/pts/2'/>
<target type='isa-serial' port='1'>
<model name='isa-serial'/>
</target>
<alias name='serial1'/>
</serial>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
<input type='mouse' bus='ps2'>
<alias name='input0'/>
</input>
<input type='keyboard' bus='ps2'>
<alias name='input1'/>
</input>
<graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
</graphics>
<audio id='1' type='none'/>
<video>
<model type='cirrus' vram='16384' heads='1' primary='yes'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
</video>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</memballoon>
<iommu model='intel'>
<driver intremap='on' caching_mode='on' iotlb='on'/>
<alias name='iommu0'/>
</iommu>
</devices>
<seclabel type='dynamic' model='selinux' relabel='yes'>
<label>system_u:system_r:svirt_t:s0:c421,c512</label>
<imagelabel>system_u:object_r:svirt_image_t:s0:c421,c512</imagelabel>
</seclabel>
<seclabel type='dynamic' model='dac' relabel='yes'>
<label>+107:+1001</label>
<imagelabel>+107:+1001</imagelabel>
</seclabel>
</domain>
3. Configure the port inside guest
ip address: 20.0.0.1
mtu: 9000
[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:bb:63:7b brd ff:ff:ff:ff:ff:ff
inet 192.168.122.138/24 brd 192.168.122.255 scope global dynamic noprefixroute enp2s0
valid_lft 2682sec preferred_lft 2682sec
inet6 fe80::bc2:f054:b47a:3336/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:11:8f:e8 brd ff:ff:ff:ff:ff:ff
inet 20.0.0.1/24 scope global enp3s0
valid_lft forever preferred_lft forever
inet6 fe80::43f0:cab:404f:8586/64 scope link noprefixroute
valid_lft forever preferred_lft forever
On client side:
configure ip address and mtu 9000 on client:
[root@dell-per740-55 ~]# ip a
6: ens3f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
link/ether f8:f2:1e:bb:cf:80 brd ff:ff:ff:ff:ff:ff
altname enp94s0f0
inet 20.0.0.3/24 scope global ens3f0
valid_lft forever preferred_lft forever
Inside server's guest ping client's port address:
[root@localhost ~]# ping 20.0.0.3
PING 20.0.0.3 (20.0.0.3) 56(84) bytes of data.
64 bytes from 20.0.0.3: icmp_seq=2 ttl=64 time=0.060 ms
64 bytes from 20.0.0.3: icmp_seq=3 ttl=64 time=0.135 ms
64 bytes from 20.0.0.3: icmp_seq=4 ttl=64 time=0.062 ms
Start netserver on client side:
[root@dell-per740-55 ~]# ps aux|grep netserver
root 2144 0.0 0.0 3840 1352 ? Ss 03:25 0:00 netserver
root 2291 0.0 0.0 6408 2300 pts/1 S+ 05:05 0:00 grep --color=auto netserver
Do netperf test inside guest of server,
[root@localhost ~]# netperf -t UDP_STREAM -H 20.0.0.3 – -m 1025
MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 20.0.0.3 () port 0 AF_INET
Socket Message Elapsed Messages
Size Size Time Okay Errors Throughput
bytes bytes secs # # 10^6bits/sec
212992 1025 10.00 5205532 0 4268.52
212992 10.00 5110575 4190.66
[root@localhost ~]# netperf -t UDP_STREAM -H 20.0.0.3 – -m 8193
MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 20.0.0.3 () port 0 AF_INET
Socket Message Elapsed Messages
Size Size Time Okay Errors Throughput
bytes bytes secs # # 10^6bits/sec
212992 8193 14.20 476933 0 2200.71
212992 14.20 0 0.00
Run ping again:
[root@localhost ~]# ping 20.0.0.3
PING 20.0.0.3 (20.0.0.3) 56(84) bytes of data.
ping: sendmsg: No buffer space available
ping: sendmsg: No buffer space available
Actual results:
netperf test with 8193byte failed.
And after do the netperf test with 8193byte, the guest cannot ping the client ip address anymore.
Expected results:
After do the netperf test with 8193byte, the guest should ping the client ip address successfully.
When restart the guest, the guest still ping the client ip failed.
[root@localhost ~]# ping 20.0.0.3
PING 20.0.0.3 (20.0.0.3) 56(84) bytes of data.
From 20.0.0.1 icmp_seq=1 Destination Host Unreachable
When restart the openvswitch service, the guest ping the client ip successfully.
Additional info:
https://beaker.engineering.redhat.com/jobs/7892906
For another the i40e<->i40e test env, it also has this issue.