-
Bug
-
Resolution: Cannot Reproduce
-
Undefined
-
None
-
4.14.z
-
None
-
None
-
False
-
-
Description of problem:
On MNO spoke with nohz=off, latency test of DPDK application reported lower latency than with nohz=on
Version-Release number of selected component (if applicable):
4.14.22
How reproducible:
1. Deploy MNO
2. Run ping from Pod and get RTT
Pod Definition - guaranteed QoS pods with annotation:
apiVersion: v1
kind: Pod
metadata:
name: test-pod-1
namespace: test1
annotations:
cpu-quota.crio.io: "disable"
cpu-load-balancing.crio.io: "disable"
irq-load-balancing.crio.io: "disable"
cpu-c-states.crio.io: "disable"
cpu-freq-governor.crio.io: "performance"
k8s.v1.cni.cncf.io/networks: '[
{
"name": "cnf-core-up",
"namespace": "openshift-multus"
},
{
"name": "sriov-net-1",
"namespace": "openshift-sriov-network-operator"
}
]'
spec:
containers:
- name: sample-pod1
command: ["/bin/bash", "-c", "sleep 2000000000000"]
image: registry.redhat.io/openshift4/dpdk-base-rhel8
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: "8"
hugepages-1Gi: 1Gi
memory: 12Gi
requests:
cpu: "8"
hugepages-1Gi: 1Gi
memory: 12Gi
securityContext:
privileged: true
volumeMounts:
- mountPath: /hugepages
name: hugepage
runtimeClassName: performance-appworker-a-pp
volumes:
- emptyDir:
medium: HugePages
name: hugepage
pod definition - guaranteed QoS pods without annotation
apiVersion: v1
kind: Pod
metadata:
name: test-pod-1
namespace: test1
annotations:
k8s.v1.cni.cncf.io/networks: '[
{
"name": "cnf-core-up",
"namespace": "openshift-multus"
},
{
"name": "sriov-net-1",
"namespace": "openshift-sriov-network-operator"
}
]'
spec:
containers:
- name: sample-pod1
command: ["/bin/bash", "-c", "sleep 2000000000000"]
image: registry.redhat.io/openshift4/dpdk-base-rhel8
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: "8"
hugepages-1Gi: 1Gi
memory: 12Gi
requests:
cpu: "8"
hugepages-1Gi: 1Gi
memory: 12Gi
securityContext:
privileged: true
volumeMounts:
- mountPath: /hugepages
name: hugepage
runtimeClassName: performance-appworker-a-pp
volumes:
- emptyDir:
medium: HugePages
name: hugepage
qosClass value in pods under test
qosClass: Guaranteed
Results - Testing with Guaranteed QoS , annotation and runtimeClassName (nohz=off show better latency than nohz=on)
Nohz=off sh-5.1# cat /proc/cmdline BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-2c6f0c83664a10a1ea3cca4d971502f456cb13b280813e294816e03b151a0ae9/vmlinuz-5.14.0-284.77.1.el9_2.x86_64 ignition.platform.id=metal ostree=/ostree/boot.0/rhcos/2c6f0c83664a10a1ea3cca4d971502f456cb13b280813e294816e03b151a0ae9/0 ip=ens10f0:dhcp root=UUID=b8cce51a-a7ea-4647-908c-e0cac4bd8a54 rw rootflags=prjquota boot=UUID=572375d1-7fff-47f7-819e-68a62addbeaf skew_tick=1 tsc=reliable rcupdate.rcu_normal_after_boot=1 rcu_nocbs=3-27,30-55,59-83,86-111 tuned.non_isolcpus=00300000,07000000,30000007 systemd.cpu_affinity=0,1,2,84,85,56,57,58,28,29 intel_iommu=on iommu=pt isolcpus=managed_irq,3-27,30-55,59-83,86-111 default_hugepagesz=1G hugepagesz=1G hugepages=24 hugepagesz=2M hugepages=14000 nohz=off intel_pstate=passive systemd.unified_cgroup_hierarchy=0 systemd.legacy_systemd_cgroup_controller=1sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.282 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.181 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.163 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.114 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.188 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.112 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.043 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.157 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 170ms rtt min/avg/max/mdev = 0.043/0.155/0.282/0.065 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.075 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.130 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.193 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.199 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.175 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.266 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.050 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.111 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 201ms rtt min/avg/max/mdev = 0.050/0.149/0.266/0.068 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.133 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.202 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.055 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.141 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.139 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.055 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.133 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.044 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 159ms rtt min/avg/max/mdev = 0.044/0.112/0.202/0.053 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.157 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.145 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.197 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.113 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.113 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.121 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.138 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.196 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 198ms rtt min/avg/max/mdev = 0.113/0.147/0.197/0.034 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.224 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.240 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.185 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.123 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.144 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.117 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.145 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.130 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 184ms rtt min/avg/max/mdev = 0.117/0.163/0.240/0.046 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.149 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.184 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.142 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.197 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.045 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.110 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.041 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.038 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 183ms rtt min/avg/max/mdev = 0.038/0.113/0.197/0.061 ms sh-4.4# sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.223 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.158 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.208 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.153 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.127 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.101 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.052 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.196 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 162ms rtt min/avg/max/mdev = 0.052/0.152/0.223/0.054 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.064 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.127 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.169 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.225 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.217 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.153 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.147 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.254 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 190ms rtt min/avg/max/mdev = 0.064/0.169/0.254/0.058 ms Nohz=on sh-5.1# cat /proc/cmdline BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-2c6f0c83664a10a1ea3cca4d971502f456cb13b280813e294816e03b151a0ae9/vmlinuz-5.14.0-284.77.1.el9_2.x86_64 ignition.platform.id=metal ostree=/ostree/boot.0/rhcos/2c6f0c83664a10a1ea3cca4d971502f456cb13b280813e294816e03b151a0ae9/0 ip=ens10f0:dhcp root=UUID=b8cce51a-a7ea-4647-908c-e0cac4bd8a54 rw rootflags=prjquota boot=UUID=572375d1-7fff-47f7-819e-68a62addbeaf skew_tick=1 tsc=reliable rcupdate.rcu_normal_after_boot=1 nohz=on rcu_nocbs=3-27,30-55,59-83,86-111 tuned.non_isolcpus=00300000,07000000,30000007 systemd.cpu_affinity=0,1,2,84,85,56,57,58,28,29 intel_iommu=on iommu=pt isolcpus=managed_irq,3-27,30-55,59-83,86-111 default_hugepagesz=1G hugepagesz=1G hugepages=24 hugepagesz=2M hugepages=14000 intel_pstate=passive systemd.unified_cgroup_hierarchy=0 systemd.legacy_systemd_cgroup_controller=1 sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.475 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.194 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.185 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.179 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.207 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.187 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.188 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.186 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 210ms rtt min/avg/max/mdev = 0.179/0.225/0.475/0.095 ms sh-4.4# sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.205 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.198 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.185 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.207 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.144 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.196 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.202 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.170 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 179ms rtt min/avg/max/mdev = 0.144/0.188/0.207/0.023 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.224 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.186 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.203 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.203 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.204 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.192 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.196 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.179 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 188ms rtt min/avg/max/mdev = 0.179/0.198/0.224/0.017 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.189 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.190 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.188 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.185 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.196 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.165 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.183 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.205 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 154ms rtt min/avg/max/mdev = 0.165/0.187/0.205/0.018 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.220 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.196 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.191 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.204 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.191 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.182 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.200 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.198 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 210ms rtt min/avg/max/mdev = 0.182/0.197/0.220/0.020 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.223 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.203 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.174 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.193 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.184 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.195 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.222 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.194 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 164ms rtt min/avg/max/mdev = 0.174/0.198/0.223/0.021 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.212 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.191 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.200 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.179 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.201 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.197 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.260 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.162 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 215ms rtt min/avg/max/mdev = 0.162/0.200/0.260/0.028 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.199 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.206 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.193 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.187 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.184 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.182 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.201 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.187 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 179ms rtt min/avg/max/mdev = 0.182/0.192/0.206/0.014 ms
Results - Testing without annotations nohz=off vs nohz=on (nohz=off show better latency than nohz=on)
Nohz=off sh-5.1# cat /proc/cmdline BOOT_IMAGE=(hd0,gpt3)/ostree/rhcos-2c6f0c83664a10a1ea3cca4d971502f456cb13b280813e294816e03b151a0ae9/vmlinuz-5.14.0-284.77.1.el9_2.x86_64 ignition.platform.id=metal ostree=/ostree/boot.0/rhcos/2c6f0c83664a10a1ea3cca4d971502f456cb13b280813e294816e03b151a0ae9/0 ip=ens10f0:dhcp root=UUID=209773ae-b0a8-4437-ac68-f7deeab59a63 rw rootflags=prjquota boot=UUID=38af53d3-3f52-4e37-b588-347c00a957f0 skew_tick=1 tsc=reliable rcupdate.rcu_normal_after_boot=1 rcu_nocbs=3-27,30-55,59-83,86-111 tuned.non_isolcpus=00300000,07000000,30000007 systemd.cpu_affinity=0,1,2,84,85,56,57,58,28,29 intel_iommu=on iommu=pt isolcpus=managed_irq,3-27,30-55,59-83,86-111 default_hugepagesz=1G hugepagesz=1G hugepages=24 hugepagesz=2M hugepages=14000 nohz=off intel_pstate=passive systemd.unified_cgroup_hierarchy=0 systemd.legacy_systemd_cgroup_controller=1 sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.406 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.272 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.246 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.257 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.199 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.226 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.257 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.237 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 184ms rtt min/avg/max/mdev = 0.199/0.262/0.406/0.060 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.315 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.174 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.309 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.353 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.207 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.254 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.195 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.226 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 213ms rtt min/avg/max/mdev = 0.174/0.254/0.353/0.061 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.284 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.251 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.281 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.314 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.241 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.189 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.277 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.322 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 199ms rtt min/avg/max/mdev = 0.189/0.269/0.322/0.045 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.338 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.290 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.132 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.237 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.248 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.240 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.244 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.225 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 189ms rtt min/avg/max/mdev = 0.132/0.244/0.338/0.055 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.323 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.229 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.252 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.121 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.245 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.270 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.357 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.297 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 174ms rtt min/avg/max/mdev = 0.121/0.261/0.357/0.069 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.343 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.259 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.249 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.174 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.183 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.148 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.185 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.259 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 163ms rtt min/avg/max/mdev = 0.148/0.225/0.343/0.059 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.169 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.214 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.228 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.096 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.105 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.289 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.136 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.215 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 176ms rtt min/avg/max/mdev = 0.096/0.181/0.289/0.063 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.238 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.126 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.111 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.234 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.145 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.258 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.232 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.215 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 188ms rtt min/avg/max/mdev = 0.111/0.194/0.258/0.057 ms Nohz=onsh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.473 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.324 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.213 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.211 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.281 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.147 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.249 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.226 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 200ms rtt min/avg/max/mdev = 0.147/0.265/0.473/0.093 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.263 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.200 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.255 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.209 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.219 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.297 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.326 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.305 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 212ms rtt min/avg/max/mdev = 0.200/0.259/0.326/0.045 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.229 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.275 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.292 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.247 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.301 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.289 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.225 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.245 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 157ms rtt min/avg/max/mdev = 0.225/0.262/0.301/0.035 ms sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.188 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.283 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.201 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.340 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.279 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.325 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.269 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.246 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 197ms rtt min/avg/max/mdev = 0.188/0.266/0.340/0.052 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.303 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.281 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.281 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.266 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.220 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.286 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.209 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.222 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 207ms rtt min/avg/max/mdev = 0.209/0.258/0.303/0.037 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.290 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.232 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.304 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.185 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.300 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.221 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.256 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.284 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 215ms rtt min/avg/max/mdev = 0.185/0.259/0.304/0.040 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.286 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.277 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.266 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.283 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.254 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.277 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.252 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.305 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 182ms rtt min/avg/max/mdev = 0.252/0.275/0.305/0.016 ms sh-4.4# sh-4.4# ping -I net2 -c 8 192.168.76.8 PING 192.168.76.8 (192.168.76.8) from 192.168.76.9 net2: 56(84) bytes of data. 64 bytes from 192.168.76.8: icmp_seq=1 ttl=64 time=0.221 ms 64 bytes from 192.168.76.8: icmp_seq=2 ttl=64 time=0.263 ms 64 bytes from 192.168.76.8: icmp_seq=3 ttl=64 time=0.245 ms 64 bytes from 192.168.76.8: icmp_seq=4 ttl=64 time=0.213 ms 64 bytes from 192.168.76.8: icmp_seq=5 ttl=64 time=0.221 ms 64 bytes from 192.168.76.8: icmp_seq=6 ttl=64 time=0.273 ms 64 bytes from 192.168.76.8: icmp_seq=7 ttl=64 time=0.216 ms 64 bytes from 192.168.76.8: icmp_seq=8 ttl=64 time=0.226 ms--- 192.168.76.8 ping statistics --- 8 packets transmitted, 8 received, 0% packet loss, time 177ms rtt min/avg/max/mdev = 0.213/0.234/0.273/0.028 ms sh-4.4#
Expected results: There should be no impact on latency on setting nohz=on or nohz=off but apparently there is according to results above.