Summary: ================ Pod to pod [east-west] HW Offload Pass. Pod to external [north-south] HW offload is not working. Pod to nodePort service [north-south] Depending on test HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957. Depending on test HW offload is not working. Pod to clusterIP service Backed by regular pod (same node) [east-west] HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by regular pod (diff node) [east-west] HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by hostnetwork pod (same node) [north-south] HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by hostnetwork pod (diff node) [north-south] HW offload is not working. Hostnetwork pod to ClusterIP service Backed by regular pod (diff node) [north-south] HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by regular pod (same node) [north-south] HW Offload Pass. Backed by hostnetwork pod (diff node) [north-south] HW offload is not working. Backed by hostnetwork pod (same node) [north-south] HW offload is not working. Hostnetwork pod to NodePort service [north-south] Depending on test HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957. Depending on test HW offload is not working. External network traffic to nodePort/External IP service [north-south] HW offload is not working. External network traffic to pods (multiple external gateway) [north-south] Unsure how to test this. Bare metal hosts running this kernel version 8.6. ==================================================== Linux worker-advnetlab48 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux Linux worker-advnetlab49 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux Environment Variables: ======================== Default/Override Values: Launch Control: FT_HOSTONLY false FT_CLIENTONLY false FT_NAMESPACE default FT_REQ_SERVER_NODE worker-advnetlab48 FT_REQ_REMOTE_CLIENT_NODE worker-advnetlab49 FT_SRIOV_NODE_LABEL feature.node.kubernetes.io/network-sriov.capable FT_EXPORT_SVC false Label Management: FT_SERVER_NODE_LABEL ft.ServerPod FT_CLIENT_NODE_LABEL ft.ClientPod Test Control: TEST_CASE (0 means all) 1 VERBOSE false FT_VARS true FT_NOTES false FT_DEBUG false CURL true CURL_CMD curl -m 5 IPERF true IPERF_CMD iperf3 IPERF_TIME 30 FT_CLIENT_CPU_MASK OVN_TRACE false OVN_TRACE_CMD ./ovnkube-trace -loglevel=5 -tcp FT_SVC_QUALIFIER FT_MC_NAMESPACE submariner-operator FT_MC_CO_SERVER_LABEL submariner.io/gateway=true OVN Trace Control: OVN_K_NAMESPACE ovn-kubernetes SSL_ENABLE -noSSL From YAML Files: NET_ATTACH_DEF_NAME ftnetattach SRIOV_RESOURCE_NAME openshift.io/mlxnics TEST_IMAGE quay.io/wizhao/ft-base-image:0.8-x86_64 CLIENT_POD_NAME_PREFIX ft-client-pod http Server: HTTP_SERVER_POD_NAME ft-http-server-pod-v4 HTTP_SERVER_HOST_POD_NAME ft-http-server-host-v4 HTTP_CLUSTERIP_POD_SVC_NAME ft-http-service-clusterip-pod-v4 HTTP_CLUSTERIP_POD_SVC_PORT 8080 HTTP_CLUSTERIP_HOST_SVC_NAME ft-http-service-clusterip-host-v4 HTTP_CLUSTERIP_HOST_SVC_PORT 8079 HTTP_NODEPORT_SVC_NAME ft-http-service-nodeport-pod-v4 HTTP_NODEPORT_POD_SVC_PORT 30080 HTTP_NODEPORT_HOST_SVC_NAME ft-http-service-nodeport-host-v4 HTTP_NODEPORT_HOST_SVC_PORT 30079 iperf Server: IPERF_SERVER_POD_NAME ft-iperf-server-pod-v4 IPERF_SERVER_HOST_POD_NAME ft-iperf-server-host-v4 IPERF_CLUSTERIP_POD_SVC_NAME ft-iperf-service-clusterip-pod-v4 IPERF_CLUSTERIP_POD_SVC_PORT 5201 IPERF_CLUSTERIP_HOST_SVC_NAME ft-iperf-service-clusterip-host-v4 IPERF_CLUSTERIP_HOST_SVC_PORT 5202 IPERF_NODEPORT_POD_SVC_NAME ft-iperf-service-nodeport-pod-v4 IPERF_NODEPORT_POD_SVC_PORT 30201 IPERF_NODEPORT_HOST_SVC_NAME ft-iperf-service-nodeport-host-v4 IPERF_NODEPORT_HOST_SVC_PORT 30202 SERVER_PATH /etc/httpserver/ POD_SERVER_STRING Server - Pod Backend Reached HOST_SERVER_STRING Server - Host Backend Reached EXTERNAL_SERVER_STRING The document has moved KUBEAPI_SERVER_STRING serverAddressByClientCIDRs External Access: EXTERNAL_IP 8.8.8.8 EXTERNAL_URL google.com Queried Values: Pod Backed: HTTP_SERVER_POD_IP 10.131.0.93 IPERF_SERVER_POD_IP 10.131.0.94 SERVER_POD_NODE worker-advnetlab48 LOCAL_CLIENT_NODE worker-advnetlab48 LOCAL_CLIENT_POD ft-client-pod-sriov-nt6c6 REMOTE_CLIENT_NODE_LIST worker-advnetlab49 REMOTE_CLIENT_POD_LIST ft-client-pod-sriov-fw94j HTTP_CLUSTERIP_POD_SVC_IPV4_LIST 172.30.14.114 HTTP_CLUSTERIP_POD_SVC_PORT 8080 HTTP_NODEPORT_POD_SVC_IPV4_LIST 172.30.20.0 HTTP_NODEPORT_POD_SVC_PORT 30080 IPERF_CLUSTERIP_POD_SVC_IPV4_LIST 172.30.53.175 IPERF_CLUSTERIP_POD_SVC_PORT 5201 IPERF_NODEPORT_POD_SVC_IPV4_LIST 172.30.0.142 IPERF_NODEPORT_POD_SVC_PORT 30201 Host backed: HTTP_SERVER_HOST_IP 192.168.111.33 IPERF_SERVER_HOST_IP 192.168.111.33 SERVER_HOST_NODE worker-advnetlab48 LOCAL_CLIENT_HOST_NODE worker-advnetlab48 LOCAL_CLIENT_HOST_POD ft-client-pod-host-tx2cz REMOTE_CLIENT_HOST_NODE_LIST worker-advnetlab49 REMOTE_CLIENT_HOST_POD_LIST ft-client-pod-host-d8gtj HTTP_CLUSTERIP_HOST_SVC_IPV4_LIST 172.30.202.221 HTTP_CLUSTERIP_HOST_SVC_PORT 8079 HTTP_NODEPORT_HOST_SVC_IPV4_LIST 172.30.157.253 HTTP_NODEPORT_HOST_SVC_PORT 30079 IPERF_CLUSTERIP_HOST_SVC_IPV4_LIST 172.30.164.109 IPERF_CLUSTERIP_HOST_SVC_PORT 5202 IPERF_NODEPORT_HOST_SVC_IPV4_LIST 172.30.166.232 IPERF_NODEPORT_HOST_SVC_PORT 30202 Kubernetes API: HTTP_CLUSTERIP_KUBEAPI_SVC_IPV4 172.30.0.1 HTTP_CLUSTERIP_KUBEAPI_SVC_PORT 443 HTTP_CLUSTERIP_KUBEAPI_EP_IP 192.168.111.20 HTTP_CLUSTERIP_KUBEAPI_EP_PORT 6443 HTTP_CLUSTERIP_KUBEAPI_SVC_NAME kubernetes.default.svc [root@wsfd-advnetlab15 ovn-kuber-traffic-flow-tests]# oc get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ft-client-pod-host-8p942 1/1 Running 0 4h20m 192.168.111.20 master-0 ft-client-pod-host-d4gkv 1/1 Running 0 4h20m 192.168.111.22 master-2 ft-client-pod-host-d8gtj 1/1 Running 0 4h20m 192.168.111.34 worker-advnetlab49 ft-client-pod-host-fvsvr 1/1 Running 0 4h20m 192.168.111.21 master-1 ft-client-pod-host-tx2cz 1/1 Running 0 4h20m 192.168.111.33 worker-advnetlab48 ft-client-pod-sriov-fw94j 1/1 Running 0 4h20m 10.128.2.27 worker-advnetlab49 ft-client-pod-sriov-nt6c6 1/1 Running 0 4h20m 10.131.0.95 worker-advnetlab48 ft-http-server-host-v4 1/1 Running 0 4h20m 192.168.111.33 worker-advnetlab48 ft-http-server-pod-v4 1/1 Running 0 4h20m 10.131.0.93 worker-advnetlab48 ft-iperf-server-host-v4 1/1 Running 0 4h20m 192.168.111.33 worker-advnetlab48 ft-iperf-server-pod-v4 1/1 Running 0 4h20m 10.131.0.94 worker-advnetlab48 worker-advnetlab48-debug 1/1 Running 0 4h10m 192.168.111.33 worker-advnetlab48 worker-advnetlab49-debug 1/1 Running 0 4h5m 192.168.111.34 worker-advnetlab49 ========================= Pod to pod [east-west] ========================= TEST_CASE=1 FT_NOTES=false FT_VARS=true IPERF=true IPERF_TIME=50 ./test.sh FLOW 01: Pod to Pod traffic --------------------------- *** 1-a: Pod to Pod (Same Node) *** admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://10.131.0.93:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- iperf3 -c 10.131.0.94 -p 5201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 89.2 GBytes 25.5 Gbits/sec 14376 sender [ 5] 0.00-30.00 sec 89.2 GBytes 25.5 Gbits/sec receiver SUCCESS worker-advnetlab48 0134f26cf7816c3: rx_packets:0/s tx_packets:0/s vport_rx_packets:3001357/s vport_tx_packets:62750/s 43227e1ca5d2bbb: rx_packets:0/s tx_packets:0/s vport_rx_packets:62766/s vport_tx_packets:3002108/s ens1f0: rx_packets:90/s tx_packets:155/s rx_packets_phy:90/s tx_packets_phy:155/s tcpdump -i 0134f26cf7816c3 => No traffic after a few seconds. tcpdump -i 43227e1ca5d2bbb => No traffic after a few seconds. CONCLUSION: HW offload is working as expected. *** 1-b: Pod to Pod (Different Node) *** admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://10.131.0.93:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- iperf3 -c 10.131.0.94 -p 5201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 77.7 GBytes 22.2 Gbits/sec 665 sender [ 5] 0.00-30.00 sec 77.7 GBytes 22.2 Gbits/sec receiver SUCCESS worker-advnetlab48 43227e1ca5d2bbb: rx_packets:3/s tx_packets:0/s vport_rx_packets:12799/s vport_tx_packets:527589/s ens1f0: rx_packets:38/s tx_packets:43/s rx_packets_phy:521124/s tx_packets_phy:12681/s tcpdump -i 43227e1ca5d2bbb => No traffic after a few seconds. worker-advnetlab49 ens1f0: rx_packets:40/s tx_packets:22/s rx_packets_phy:44150/s tx_packets_phy:1874926/s f676fc595a64ac5: rx_packets:0/s tx_packets:0/s vport_rx_packets:2114216/s vport_tx_packets:49971/s tcpdump -i f676fc595a64ac5 => No traffic after a few seconds. CONCLUSION: HW offload is working as expected. =============================== Pod to external [north-south] =============================== FLOW 13: Cluster -> External Network ------------------------------------ *** 13-a: Pod -> External Network *** admin:worker-advnetlab49 -> External:External kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://google.com/" SUCCESS *** 13-b: Host -> External Network *** admin:worker-advnetlab49 -> External:External kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://google.com/" SUCCESS Additional Test: kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl "https://raw.githubusercontent.com/owid/covid-19-data/master/public/data/internal/megafile--all-reduced.json" worker-advnetlab49 ens1f0: rx_packets:49/s tx_packets:109/s rx_packets_phy:49/s tx_packets_phy:109/s f676fc595a64ac5: rx_packets:260/s tx_packets:3291/s vport_rx_packets:260/s vport_tx_packets:3291/s CONCLUSION: HW Offload working is inconclusive. It seems that some packets are received hardware offloaded. Additional Test Scenario: iperf3 -s -p 9999 on worker-advnetlab15 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- iperf3 -c 10.19.128.33 -p 9999 -t 30 Connecting to host 10.19.128.33, port 9999 [ 5] local 10.131.0.95 port 54338 connected to 10.19.128.33 port 9999 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 14.7 GBytes 4.20 Gbits/sec 1687 sender [ 5] 0.00-30.04 sec 14.7 GBytes 4.20 Gbits/sec receiver worker-advnetlab48 0134f26cf7816c3: rx_packets:465274/s tx_packets:14282/s vport_rx_packets:465274/s vport_tx_packets:14282/s ens1f0: rx_packets:14362/s tx_packets:466386/s rx_packets_phy:14363/s tx_packets_phy:466391/s tcpdump -i 0134f26cf7816c3 1:52:56.034180 IP worker-advnetlab48.55796 > wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct: Flags [.], seq 615977336:616042040, ack 1, win 213, options [nop,nop,TS val 3832451252 ecr 3141327515], length 64704 21:52:56.034238 IP worker-advnetlab48.55796 > wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct: Flags [.], seq 616042040:616106744, ack 1, win 213, options [nop,nop,TS val 3832451252 ecr 3141327515], length 64704 21:52:56.034282 IP worker-advnetlab48.55796 > wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct: Flags [.], seq 616106744:616171448, ack 1, win 213, options [nop,nop,TS val 3832451252 ecr 3141327515], length 64704 21:52:56.034314 IP worker-advnetlab48.55796 > wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct: Flags [.], seq 616171448:616228064, ack 1, win 213, options [nop,nop,TS val 3832451252 ecr 3141327515], length 56616 21:52:56.035038 IP wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct > 10.131.0.95.55796: Flags [.], ack 615525794, win 24189, options [nop,nop,TS val 3141327515 ecr 3832451252], length 0 21:52:56.035042 IP wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct > 10.131.0.95.55796: Flags [.], ack 615601282, win 24511, options [nop,nop,TS val 3141327515 ecr 3832451252], length 0 21:52:56.035043 IP wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct > 10.131.0.95.55796: Flags [.], ack 615665986, win 24463, options [nop,nop,TS val 3141327515 ecr 3832451252], length 0 21:52:56.035045 IP wsfd-advnetlab15.anl.lab.eng.bos.redhat.com.distinct > 10.131.0.95.55796: Flags [.], ack 615684858, win 24576, options [nop,nop,TS val 3141327515 ecr 3832451252], length 0 CONCLUSION: HW Offload working is not working. ====================================== Pod to nodePort service [north-south] ====================================== FLOW 05: Pod -> NodePort Service traffic (Pod Backend) ------------------------------------------------------ *** 5-a: Pod -> NodePort Service traffic (Pod Backend - Same Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://172.30.20.0:8080/etc/httpserver/" SUCCESS curl HostIP:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://192.168.111.33:30080/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://ft-http-service-nodeport-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- iperf3 -c 192.168.111.33 -p 30201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 79.1 GBytes 22.6 Gbits/sec 14682 sender [ 5] 0.00-30.00 sec 79.1 GBytes 22.6 Gbits/sec receiver SUCCESS worker-advnetlab48 0134f26cf7816c3: rx_packets:0/s tx_packets:55582/s vport_rx_packets:2648971/s vport_tx_packets:55578/s 43227e1ca5d2bbb: rx_packets:55581/s tx_packets:0/s vport_rx_packets:55577/s vport_tx_packets:2648877/s ens1f0: rx_packets:32/s tx_packets:42/s rx_packets_phy:32/s tx_packets_phy:42/s tcpdump -i 43227e1ca5d2bbb 18:44:44.341298 IP worker-advnetlab48.30201 > 10.131.0.95.47386: Flags [.], ack 1842518337, win 24225, options [nop,nop,TS val 2321646683 ecr 1761421566], length 0 tcpdump -i 0134f26cf7816c3 18:48:48.440286 IP worker-advnetlab48.30201 > 10.131.0.95.49786: Flags [.], ack 660653334, win 22821, options [nop,nop,TS val 2321890782 ecr 1761665665], length 0 CONCLUSION: HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 *** 5-b: Pod -> NodePort Service traffic (Pod Backend - Different Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://172.30.20.0:8080/etc/httpserver/" SUCCESS curl HostIP:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://192.168.111.33:30080/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://ft-http-service-nodeport-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- iperf3 -c 192.168.111.33 -p 30201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 534 MBytes 149 Mbits/sec 5453 sender [ 5] 0.00-30.02 sec 533 MBytes 149 Mbits/sec receiver SUCCESS worker-advnetlab48 43227e1ca5d2bbb: rx_packets:0/s tx_packets:18215/s vport_rx_packets:947/s vport_tx_packets:18215/s ens1f0: rx_packets:18262/s tx_packets:119/s rx_packets_phy:18248/s tx_packets_phy:1068/s tcpdump -i 43227e1ca5d2bbb 19:03:09.276478 IP 100.64.0.5.49986 > 10.131.0.94.targus-getdata1: Flags [.], seq 621176734:621178082, ack 1, win 213, options [nop,nop,TS val 2806411344 ecr 840524460], length 1348 19:03:09.276494 IP 100.64.0.5.49986 > 10.131.0.94.targus-getdata1: Flags [.], seq 621178082:621179430, ack 1, win 213, options [nop,nop,TS val 2806411344 ecr 840524460], length 1348 worker-advnetlab49 ens1f0: rx_packets:1031/s tx_packets:19033/s rx_packets_phy:1031/s tx_packets_phy:19033/s f676fc595a64ac5: rx_packets:1090/s tx_packets:73/s vport_rx_packets:1090/s vport_tx_packets:73/s tcpdump -i f676fc595a64ac5 19:03:09.146744 IP worker-advnetlab48.30201 > 10.128.2.27.49986: Flags [.], ack 618029154, win 24576, options [nop,nop,TS val 840524354 ecr 2806411214], length 0 19:03:09.146749 IP worker-advnetlab48.30201 > 10.128.2.27.49986: Flags [.], ack 618072290, win 24576, options [nop,nop,TS val 840524354 ecr 2806411218], length 0 19:03:09.146824 IP worker-advnetlab49.49986 > worker-advnetlab48.30201: Flags [.], seq 618530572:618584492, ack 1, win 213, options [nop,nop,TS val 2806411239 ecr 840524354], length 53920 19:03:09.146844 IP worker-advnetlab49.49986 > worker-advnetlab48.30201: Flags [.], seq 618584492:618595276, ack 1, win 213, options [nop,nop,TS val 2806411239 ecr 840524354], length 10784 CONCLUSION: HW offload is not working. FLOW 06: Pod -> NodePort Service traffic (Host Backend) ------------------------------------------------------- *** 6-a: Pod -> NodePort Service traffic (Host Backend - Same Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://172.30.157.253:8079/etc/httpserver/" SUCCESS curl HostIP:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://192.168.111.33:30079/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://ft-http-service-nodeport-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- iperf3 -c 192.168.111.33 -p 30202 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 68.6 GBytes 19.6 Gbits/sec 11538 sender [ 5] 0.00-30.04 sec 68.6 GBytes 19.6 Gbits/sec receiver SUCCESS worker-advnetlab48 0134f26cf7816c3: rx_packets:0/s tx_packets:49854/s vport_rx_packets:2374348/s vport_tx_packets:49853/s ens1f0: rx_packets:2368674/s tx_packets:52/s rx_packets_phy:53/s tx_packets_phy:52/s tcpdump -i 0134f26cf7816c3 19:40:38.456451 IP worker-advnetlab48.30202 > 10.131.0.95.40170: Flags [.], ack 3207324950, win 24576, options [nop,nop,TS val 2529365098 ecr 1764775681], length 0 CONCLUSION: HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 *** 6-b: Pod -> NodePort Service traffic (Host Backend - Different Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://172.30.157.253:8079/etc/httpserver/" SUCCESS curl HostIP:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://192.168.111.33:30079/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://ft-http-service-nodeport-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- iperf3 -c 192.168.111.33 -p 30202 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 1.29 GBytes 368 Mbits/sec 1891 sender [ 5] 0.00-30.34 sec 1.28 GBytes 364 Mbits/sec receiver SUCCESS worker-advnetlab48 ens1f0: rx_packets:45620/s tx_packets:21520/s rx_packets_phy:45813/s tx_packets_phy:21527/s worker-advnetlab49 ens1f0: rx_packets:20093/s tx_packets:42379/s rx_packets_phy:20093/s tx_packets_phy:42379/s f676fc595a64ac5: rx_packets:161/s tx_packets:109/s vport_rx_packets:169/s vport_tx_packets:109/s tcpdump -i f676fc595a64ac5 19:45:27.877167 IP worker-advnetlab49.60738 > worker-advnetlab48.30202: Flags [.], seq 484109936:484130156, ack 1, win 213, options [nop,nop,TS val 2808949969 ecr 4150184827], length 20220 19:45:27.877231 IP worker-advnetlab49.60738 > worker-advnetlab48.30202: Flags [.], seq 484130156:484169248, ack 1, win 213, options [nop,nop,TS val 2808949969 ecr 4150184828], length 39092 19:45:27.877791 IP worker-advnetlab48.30202 > 10.128.2.27.60738: Flags [.], ack 482788934, win 24576, options [nop,nop,TS val 4150184828 ecr 2808949928], length 0 19:45:27.877795 IP worker-advnetlab48.30202 > 10.128.2.27.60738: Flags [.], ack 482791630, win 24576, options [nop,nop,TS val 4150184828 ecr 2808949928], length 0 CONCLUSION: HW offload is not working. ====================================== Pod to clusterIP service ====================================== Backed by regular pod (same node) [east-west] FLOW 03: Pod -> Cluster IP Service traffic (Pod Backend) -------------------------------------------------------- *** 3-a: Pod -> Cluster IP Service traffic (Pod Backend - Same Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://172.30.14.114:8080/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://10.131.0.93:8080/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://ft-http-service-clusterip-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- iperf3 -c 172.30.53.175 -p 5201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 77.3 GBytes 22.1 Gbits/sec 10823 sender [ 5] 0.00-30.00 sec 77.3 GBytes 22.1 Gbits/sec receiver SUCCESS worker-advnetlab48 0134f26cf7816c3: rx_packets:0/s tx_packets:51360/s vport_rx_packets:2416898/s vport_tx_packets:51361/s 43227e1ca5d2bbb: rx_packets:51336/s tx_packets:0/s vport_rx_packets:51336/s vport_tx_packets:2415625/s ens1f0: rx_packets:51/s tx_packets:48/s rx_packets_phy:51/s tx_packets_phy:48/s tcpdump -i 43227e1ca5d2bbb 20:00:15.328018 IP 172.30.53.175.targus-getdata1 > 10.131.0.95.32970: Flags [.], ack 258258049, win 24576, options [nop,nop,TS val 2326177670 ecr 2741415750], length 0 CONCLUSION: HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by regular pod (diff node) [east-west] *** 3-b: Pod -> Cluster IP Service traffic (Pod Backend - Different Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://172.30.14.114:8080/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://10.131.0.93:8080/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://ft-http-service-clusterip-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- iperf3 -c 172.30.53.175 -p 5201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 68.9 GBytes 19.7 Gbits/sec 6104 sender [ 5] 0.00-30.00 sec 68.9 GBytes 19.7 Gbits/sec receiver SUCCESS worker-advnetlab48 43227e1ca5d2bbb: rx_packets:0/s tx_packets:0/s vport_rx_packets:43815/s vport_tx_packets:1786029/s ens1f0: rx_packets:48/s tx_packets:24/s rx_packets_phy:1786677/s tx_packets_phy:43861/s tcpdump -i 43227e1ca5d2bbb => No traffic after a few seconds. worker-advnetlab49 ens1f0: rx_packets:48865/s tx_packets:38/s rx_packets_phy:48864/s tx_packets_phy:2056888/s f676fc595a64ac5: rx_packets:0/s tx_packets:42524/s vport_rx_packets:1725024/s vport_tx_packets:42525/s tcpdump -i f676fc595a64ac5 20:09:40.943640 IP 172.30.53.175.targus-getdata1 > 10.128.2.27.44706: Flags [.], ack 1892272930, win 24576, options [nop,nop,TS val 686149516 ecr 2044076875], length 0 CONCLUSION: HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by hostnetwork pod (same node) [north-south] *** 4-a: Pod -> Cluster IP Service traffic (Host Backend - Same Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://172.30.202.221:8079/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://192.168.111.33:8079/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- curl -m 5 "http://ft-http-service-clusterip-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-nt6c6 -- iperf3 -c 172.30.164.109 -p 5202 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 70.5 GBytes 20.2 Gbits/sec 15510 sender [ 5] 0.00-30.04 sec 70.5 GBytes 20.2 Gbits/sec receiver SUCCESS worker-advnetlab48 0134f26cf7816c3: rx_packets:0/s tx_packets:48146/s vport_rx_packets:2291129/s vport_tx_packets:48146/s ens1f0: rx_packets:2287794/s tx_packets:122/s rx_packets_phy:55/s tx_packets_phy:122/s tcpdump -i 0134f26cf7816c3 20:22:27.050941 IP 172.30.164.109.targus-getdata2 > 10.131.0.95.51794: Flags [.], ack 1539869562, win 24576, options [nop,nop,TS val 2531873693 ecr 1799733654], length 0 CONCLUSION: HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by hostnetwork pod (diff node) [north-south] *** 4-b: Pod -> Cluster IP Service traffic (Host Backend - Different Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://172.30.202.221:8079/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://192.168.111.33:8079/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- curl -m 5 "http://ft-http-service-clusterip-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-sriov-fw94j -- iperf3 -c 172.30.164.109 -p 5202 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 16.2 GBytes 4.63 Gbits/sec 2725 sender [ 5] 0.00-30.04 sec 16.2 GBytes 4.63 Gbits/sec receiver SUCCESS worker-advnetlab48 ens1f0: rx_packets:530755/s tx_packets:15068/s rx_packets_phy:530903/s tx_packets_phy:15075/s worker-advnetlab49 ens1f0: rx_packets:16194/s tx_packets:573907/s rx_packets_phy:16192/s tx_packets_phy:573898/s f676fc595a64ac5: rx_packets:527317/s tx_packets:14840/s vport_rx_packets:527411/s vport_tx_packets:14840/s tcpdump -i f676fc595a64ac5 20:27:13.714012 IP 172.30.164.109.targus-getdata2 > 10.128.2.27.50816: Flags [.], ack 2227301086, win 24576, options [nop,nop,TS val 4293548891 ecr 2757159312], length 0 20:27:13.714013 IP 172.30.164.109.targus-getdata2 > 10.128.2.27.50816: Flags [.], ack 2227430494, win 24576, options [nop,nop,TS val 4293548891 ecr 2757159312], length 0 20:27:13.714047 IP worker-advnetlab49.50816 > worker-advnetlab48.targus-getdata2: Flags [.], seq 2227430456:2227495160, ack 1, win 213, options [nop,nop,TS val 2757159313 ecr 4293548891], length 64704 20:27:13.714090 IP worker-advnetlab49.50816 > worker-advnetlab48.targus-getdata2: Flags [.], seq 2227495160:2227559864, ack 1, win 213, options [nop,nop,TS val 2757159313 ecr 4293548891], length 64704 CONCLUSION: HW offload is not working. ====================================== Hostnetwork pod to ClusterIP service ====================================== Backed by regular pod (same node) [north-south] *** 9-a: Host Pod -> Cluster IP Service traffic (Pod Backend - Same Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://172.30.14.114:8080/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://10.131.0.93:8080/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://ft-http-service-clusterip-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- iperf3 -c 172.30.53.175 -p 5201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 87.0 GBytes 24.9 Gbits/sec 6864 sender [ 5] 0.00-30.00 sec 87.0 GBytes 24.9 Gbits/sec receiver SUCCESS worker-advnetlab48 43227e1ca5d2bbb: rx_packets:59608/s tx_packets:2800642/s vport_rx_packets:59608/s vport_tx_packets:2800635/s ens1f0: rx_packets:46/s tx_packets:122/s rx_packets_phy:46/s tx_packets_phy:122/s tcpdump -i 43227e1ca5d2bbb 21:06:56.879467 IP 100.64.0.5.47104 > 10.131.0.94.targus-getdata1: Flags [.], seq 1926032682:1926097386, ack 1, win 213, options [nop,nop,TS val 2032119304 ecr 847952086], length 64704 21:06:56.879492 IP 100.64.0.5.47104 > 10.131.0.94.targus-getdata1: Flags [.], seq 1926097386:1926162090, ack 1, win 213, options [nop,nop,TS val 2032119304 ecr 847952086], length 64704 21:06:56.879488 IP 172.30.53.175.targus-getdata1 > 169.254.169.2.47104: Flags [.], ack 1923444485, win 24576, options [nop,nop,TS val 847952086 ecr 2032119304], length 0 21:06:56.879496 IP 172.30.53.175.targus-getdata1 > 169.254.169.2.47104: Flags [.], ack 1923509189, win 24568, options [nop,nop,TS val 847952086 ecr 2032119304], length 0 21:06:56.879508 IP 100.64.0.5.47104 > 10.131.0.94.targus-getdata1: Flags [.], seq 1926162090:1926163438, ack 1, win 213, options [nop,nop,TS val 2032119304 ecr 847952086], length 1348 21:06:56.879541 IP 100.64.0.5.47104 > 10.131.0.94.targus-getdata1: Flags [.], seq 1926163438:1926228142, ack 1, win 213, options [nop,nop,TS val 2032119305 ecr 847952086], length 64704 21:06:56.879562 IP 172.30.53.175.targus-getdata1 > 169.254.169.2.47104: Flags [.], ack 1923575241, win 24217, options [nop,nop,TS val 847952087 ecr 2032119304], length 0 21:06:56.879564 IP 100.64.0.5.47104 > 10.131.0.94.targus-getdata1: Flags [.], seq 1926228142:1926233534, ack 1, win 213, options [nop,nop,TS val 2032119305 ecr 847952086], length 5392 CONCLUSION: HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 Backed by regular pod (diff node) [north-south] *** 9-b: Host Pod -> Cluster IP Service traffic (Pod Backend - Different Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://172.30.14.114:8080/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://10.131.0.93:8080/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://ft-http-service-clusterip-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- iperf3 -c 172.30.53.175 -p 5201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 35.4 GBytes 10.1 Gbits/sec 233 sender [ 5] 0.00-30.00 sec 35.4 GBytes 10.1 Gbits/sec receiver SUCCESS worker-advnetlab48 43227e1ca5d2bbb: rx_packets:0/s tx_packets:0/s vport_rx_packets:22026/s vport_tx_packets:994396/s ens1f0: rx_packets:65/s tx_packets:122/s rx_packets_phy:994020/s tx_packets_phy:22137/s tcpdump -i 43227e1ca5d2bbb => No traffic after a few seconds. worker-advnetlab49 ens1f0: rx_packets:26038/s tx_packets:1147387/s rx_packets_phy:26039/s tx_packets_phy:1147404/s CONCLUSION: HW offload is working as expected. Backed by hostnetwork pod (same node) [north-south] *** 10-a: Host Pod -> Cluster IP Service traffic (Host Backend - Same Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://172.30.202.221:8079/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://192.168.111.33:8079/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://ft-http-service-clusterip-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- iperf3 -c 172.30.164.109 -p 5202 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 3.34 GBytes 956 Mbits/sec 16875 sender [ 5] 0.00-30.04 sec 3.34 GBytes 955 Mbits/sec receiver SUCCESS worker-advnetlab48 ens1f0: rx_packets:80/s tx_packets:133/s rx_packets_phy:80/s tx_packets_phy:133/s CONCLUSION: HW offload is not working. Backed by hostnetwork pod (diff node) [north-south] *** 10-b: Host Pod -> Cluster IP Service traffic (Host Backend - Different Node) *** curl SvcClusterIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://172.30.202.221:8079/etc/httpserver/" SUCCESS curl SvcEndPointIP:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://192.168.111.33:8079/etc/httpserver/" SUCCESS curl SvcName:SvcPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://ft-http-service-clusterip-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- iperf3 -c 172.30.164.109 -p 5202 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 60.8 GBytes 17.4 Gbits/sec 9743 sender [ 5] 0.00-30.04 sec 60.8 GBytes 17.4 Gbits/sec receiver SUCCESS worker-advnetlab48 ens1f0: rx_packets:1780543/s tx_packets:54282/s rx_packets_phy:1780965/s tx_packets_phy:54288/s worker-advnetlab49 ens1f0: rx_packets:52571/s tx_packets:1694144/s rx_packets_phy:52575/s tx_packets_phy:1694321/s CONCLUSION: HW offload is not working. ====================================== Hostnetwork pod to NodePort service [north-south] ====================================== *** 11-a: Host Pod -> NodePort Service traffic (Pod Backend - Same Node) *** curl SvcClusterIP:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://172.30.20.0:8080/etc/httpserver/" SUCCESS curl hostIP:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://192.168.111.33:30080/etc/httpserver/" SUCCESS curl SvcName:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://ft-http-service-nodeport-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- iperf3 -c 192.168.111.33 -p 30201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 85.8 GBytes 24.6 Gbits/sec 9223 sender [ 5] 0.00-30.00 sec 85.8 GBytes 24.6 Gbits/sec receiver SUCCESS worker-advnetlab48 43227e1ca5d2bbb: rx_packets:63002/s tx_packets:2887412/s vport_rx_packets:63001/s vport_tx_packets:2887371/s ens1f0: rx_packets:53/s tx_packets:31/s rx_packets_phy:53/s tx_packets_phy:31/s tcpdump -i 43227e1ca5d2bbb 21:28:07.976689 IP 172.30.0.142.targus-getdata1 > 169.254.169.2.37806: Flags [.], ack 217710277, win 24576, options [nop,nop,TS val 849223184 ecr 2217174606], length 0 21:28:07.976696 IP 172.30.0.142.targus-getdata1 > 169.254.169.2.37806: Flags [.], ack 217774981, win 24576, options [nop,nop,TS val 849223184 ecr 2217174606], length 0 21:28:07.976700 IP 172.30.0.142.targus-getdata1 > 169.254.169.2.37806: Flags [.], ack 217839685, win 24576, options [nop,nop,TS val 849223184 ecr 2217174606], length 0 21:28:07.976710 IP 100.64.0.5.37806 > 10.131.0.94.targus-getdata1: Flags [.], seq 457868714:457933418, ack 1, win 342, options [nop,nop,TS val 2217174606 ecr 849223184], length 64704 21:28:07.976732 IP 100.64.0.5.37806 > 10.131.0.94.targus-getdata1: Flags [.], seq 457933418:457998122, ack 1, win 342, options [nop,nop,TS val 2217174606 ecr 849223184], length 64704 CONCLUSION: HW offload is partially working. Affected by bug: https://bugzilla.redhat.com/show_bug.cgi?id=2079957 *** 11-b: Host Pod -> NodePort Service traffic (Pod Backend - Different Node) *** curl SvcClusterIP:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://172.30.20.0:8080/etc/httpserver/" SUCCESS curl hostIP:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://192.168.111.33:30080/etc/httpserver/" SUCCESS curl SvcName:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://ft-http-service-nodeport-pod-v4:8080/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- iperf3 -c 192.168.111.33 -p 30201 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 546 MBytes 153 Mbits/sec 5442 sender [ 5] 0.00-30.02 sec 544 MBytes 152 Mbits/sec receiver SUCCESS worker-advnetlab48 43227e1ca5d2bbb: rx_packets:888/s tx_packets:18563/s vport_rx_packets:888/s vport_tx_packets:18561/s ens1f0: rx_packets:18439/s tx_packets:905/s rx_packets_phy:18439/s tx_packets_phy:906/s tcpdump -i 43227e1ca5d2bbb 21:31:58.260591 IP 100.64.0.5.50900 > 10.131.0.94.targus-getdata1: Flags [.], seq 254529398:254530746, ack 1, win 229, options [nop,nop,TS val 3197897802 ecr 849453447], length 1348 21:31:58.260594 IP 100.64.0.5.50900 > 10.131.0.94.targus-getdata1: Flags [.], seq 254083210:254084558, ack 1, win 229, options [nop,nop,TS val 3197897802 ecr 849453447], length 1348 21:31:58.260597 IP 100.64.0.5.50900 > 10.131.0.94.targus-getdata1: Flags [.], seq 254530746:254532094, ack 1, win 229, options [nop,nop,TS val 3197897802 ecr 849453447], length 1348 21:31:58.260600 IP 100.64.0.5.50900 > 10.131.0.94.targus-getdata1: Flags [.], seq 254084558:254085906, ack 1, win 229, options [nop,nop,TS val 3197897802 ecr 849453447], length 1348 21:31:58.260602 IP 100.64.0.5.50900 > 10.131.0.94.targus-getdata1: Flags [.], seq 254085906:254087254, ack 1, win 229, options [nop,nop,TS val 3197897802 ecr 849453447], length 1348 worker-advnetlab49 ens1f0: rx_packets:1116/s tx_packets:21868/s rx_packets_phy:1116/s tx_packets_phy:21868/s CONCLUSION: HW offload is not working. *** 12-a: Host Pod -> NodePort Service traffic (Host Backend - Same Node) *** curl SvcClusterIP:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://172.30.157.253:8079/etc/httpserver/" SUCCESS curl EndPointIP:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://192.168.111.33:30079/etc/httpserver/" SUCCESS curl SvcName:NODEPORT admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- curl -m 5 "http://ft-http-service-nodeport-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab48 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-tx2cz -- iperf3 -c 192.168.111.33 -p 30202 -t 30 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 77.8 KBytes 21.2 Kbits/sec 0 sender [ 5] 0.00-30.04 sec 0.00 Bytes 0.00 bits/sec receiver SUCCESS worker-advnetlab48 ens1f0: rx_packets:61/s tx_packets:54/s rx_packets_phy:61/s tx_packets_phy:54/s CONCLUSION: HW offload is not working. *** 12-b: Host Pod -> NodePort Service traffic (Host Backend - Different Node) *** curl SvcClusterIP:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://172.30.157.253:8079/etc/httpserver/" SUCCESS curl EndPointIP:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://192.168.111.33:30079/etc/httpserver/" SUCCESS curl SvcName:NODEPORT admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- curl -m 5 "http://ft-http-service-nodeport-host-v4:8079/etc/httpserver/" SUCCESS admin:worker-advnetlab49 -> admin:worker-advnetlab48 kubectl exec -it -n default ft-client-pod-host-d8gtj -- iperf3 -c 192.168.111.33 -p 30202 -t 30 iperf3: error - unable to write to stream socket: Connection reset by peer FAILED External network traffic to nodePort/External IP service [north-south] FLOW 15: External Network -> NodePort Service traffic ----------------------------------------------------- *** 15-a: External Network -> NodePort Service traffic (Pod Backend) *** curl EndPointIP:NODEPORT admin:External -> admin:worker-advnetlab48 curl -m 5 "http://192.168.111.33:30080/etc/httpserver/" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 164 100 164 0 0 152 0 0:00:01 0:00:01 --:--:-- 152 SUCCESS *** 15-b: External Network -> NodePort Service traffic (Host Backend) *** curl EndPointIP:NODEPORT admin:External -> admin:worker-advnetlab48 curl -m 5 "http://192.168.111.33:30079/etc/httpserver/" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 170 100 170 0 0 382 0 --:--:-- --:--:-- --:--:-- 382 SUCCESS Additional test: [root@wsfd-advnetlab15 ovn-kuber-traffic-flow-tests]# iperf3 -c 192.168.111.33 -p 30201 -t 30 Connecting to host 192.168.111.33, port 30201 [ 5] local 192.168.111.1 port 50136 connected to 192.168.111.33 port 30201 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 555 MBytes 155 Mbits/sec 6527 sender [ 5] 0.00-30.22 sec 554 MBytes 154 Mbits/sec receiver worker-advnetlab48 43227e1ca5d2bbb: rx_packets:0/s tx_packets:18784/s vport_rx_packets:1124/s vport_tx_packets:18784/s ens1f0: rx_packets:18811/s tx_packets:122/s rx_packets_phy:18811/s tx_packets_phy:1250/s tcpdump -i 43227e1ca5d2bbb 22:03:24.992145 IP 100.64.0.5.50140 > 10.131.0.94.targus-getdata1: Flags [.], seq 100316850:100318198, ack 1, win 229, options [nop,nop,TS val 1345741330 ecr 851340185], length 1348 22:03:24.992148 IP 100.64.0.5.50140 > 10.131.0.94.targus-getdata1: Flags [.], seq 100318198:100319546, ack 1, win 229, options [nop,nop,TS val 1345741330 ecr 851340185], length 1348 22:03:24.992153 IP 100.64.0.5.50140 > 10.131.0.94.targus-getdata1: Flags [.], seq 100319546:100320894, ack 1, win 229, options [nop,nop,TS val 1345741330 ecr 851340185], length 1348 22:03:24.992157 IP 100.64.0.5.50140 > 10.131.0.94.targus-getdata1: Flags [.], seq 100320894:100322242, ack 1, win 229, options [nop,nop,TS val 1345741330 ecr 851340185], length 1348 22:03:24.992162 IP 100.64.0.5.50140 > 10.131.0.94.targus-getdata1: Flags [.], seq 100322242:100323590, ack 1, win 229, options [nop,nop,TS val 1345741330 ecr 851340185], length 1348 CONCLUSION: HW offload is not working. External network traffic to pods (multiple external gateway) [north-south] Not Tested.