-
Bug
-
Resolution: Duplicate
-
Undefined
-
None
-
4.16
-
None
-
Quality / Stability / Reliability
-
False
-
-
None
-
Important
-
No
-
None
-
Proposed
-
None
-
In Progress
-
Release Note Not Required
-
None
-
None
-
None
-
None
-
None
Description of problem:
On SNO spoke with telco DU profile applied, oslat reported 20us latency spike on a 1h run
Version-Release number of selected component (if applicable):
OCP 4.16.0-0.nightly-2024-02-26-013420
cluster-logging.v5.9.0
local-storage-operator.v4.16.0-202403050942
packageserver
ptp-operator.v4.16.0-202403050942
sriov-fec.v2.8.0
sriov-network-operator.v4.16.0-202403050942
How reproducible:
Multiple
Steps to Reproduce:
1. Deploy DU node
2. Run OSLAT test
Actual results:
One sample at 20us
Expected results:
All samples below 20us
Additional info:
############# dumping env ###########
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_SERVICE_PORT=443
RUNTIME_SECONDS=1h
HOSTNAME=oslat0
NSS_SDB_USE_CACHE=no
DISTTAG=f39container
PWD=/root
TRACE_THRESHOLD=
container=oci
INITIAL_DELAY_SEC=30
HOME=/root
KUBERNETES_PORT_443_TCP=tcp://[fd02::1]:443
FGC=f39
tool=oslat
manual=n
PRIO=1
TERM=xterm
SHLVL=1
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_ADDR=fd02::1
KUBERNETES_SERVICE_HOST=fd02::1
KUBERNETES_PORT=tcp://[fd02::1]:443
KUBERNETES_PORT_443_TCP_PORT=443
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
delay=60
_=/usr/bin/env
#####################################
########## container info ###########
/proc/cmdline:
BOOT_IMAGE=(hd2,gpt3)/ostree/rhcos-c93172ea2fd2a9678115e61863f53f6c3ddd9480a800d82a05487e67f320655d/vmlinuz-5.14.0-424.el9.x86_64+rt ignition.platform.id=metal ostree=/ostree/boot.0/rhcos/c93172ea2fd2a9678115e61863f53f6c3ddd9480a800d82a05487e67f320655d/0 root=UUID=ecba3874-f410-4a52-a9b3-df1685718abb rw rootflags=prjquota boot=UUID=a96d9c7c-8f98-4f89-8014-9a96b6789954 crashkernel=512M intel_iommu=on iommu=pt skew_tick=1 tsc=reliable rcupdate.rcu_normal_after_boot=1 nohz=on rcu_nocbs=2-19,22-39 tuned.non_isolcpus=00300003 systemd.cpu_affinity=0,1,20,21 intel_iommu=on iommu=pt isolcpus=managed_irq,2-19,22-39 nohz_full=2-19,22-39 tsc=reliable nosoftlockup nmi_watchdog=0 mce=off skew_tick=1 rcutree.kthread_prio=11 default_hugepagesz=1G hugepagesz=1G hugepages=32 rcupdate.rcu_normal_after_boot=0 vfio_pci.enable_sriov=1 vfio_pci.disable_idle_d3=1 efi=runtime module_blacklist=irdma intel_pstate=disable tsc=reliable systemd.unified_cgroup_hierarchy=0 systemd.legacy_systemd_cgroup_controller=1
#####################################
oslat0 5.14.0-424.el9.x86_64+rt
realtime-tests-2.5-3.fc39.x86_64
allowed cpu list: 2-9,22-29
removing cpu22 from the cpu list because it is a sibling of cpu2 which will be the cpu-main-thread
new cpu list: 3,4,5,6,7,8,9,23,24,25,26,27,28,29
cmd to run: oslat -D 1h --rtprio 1 --cpu-list 3,4,5,6,7,8,9,23,24,25,26,27,28,29 --cpu-main-thread 2
sleep 60 before test
oslat V 2.50
Total runtime: 3600 seconds
Thread priority: SCHED_FIFO:1
CPU list: 3,4,5,6,7,8,9,23,24,25,26,27,28,29
CPU for main thread: 2
Workload: no
Workload mem: 0 (KiB)
Preheat cores: 14
Pre-heat for 1 seconds...
Test starts...
Test completed.
Core: 3 4 5 6 7 8 9 23 24 25 26 27 28 29
Counter Freq: 1400 1400 1400 1400 1400 1400 1400 1400 1400 1400 1400 1400 1400 1400 (Mhz)
001 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
002 (us): 110617313046 110777571000 110648587455 110135403042 109779651579 109689688143 110396610660 110617005138 110777382581 110648197969 110135155127 109779651109 109689686845 110396887336
003 (us): 7 9 4 3 3 3 4 4 6 6 4 3 3 4
004 (us): 9 8 7 7 7 6 9 12 11 11 9 7 7 6
005 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
006 (us): 0 0 0 0 0 1 0 0 1 0 0 0 1 1
007 (us): 2 2 2 2 2 1 2 2 1 2 2 2 1 1
008 (us): 1 0 0 0 0 0 0 0 0 0 0 0 0 0
009 (us): 0 0 1 0 0 0 0 1 0 0 0 0 0 0
010 (us): 0 0 0 0 0 0 0 1 0 0 0 0 0 0
011 (us): 1 2 0 0 0 0 1 0 0 0 0 0 0 1
012 (us): 0 0 1 0 1 0 1 1 1 0 0 0 0 0
013 (us): 0 0 1 0 1 0 0 0 0 0 1 0 0 0
014 (us): 0 1 0 0 0 1 0 0 0 0 0 0 1 1
015 (us): 1 0 0 1 0 0 1 0 0 1 0 0 0 0
016 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
017 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
018 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
019 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
020 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
021 (us): 0 0 0 0 0 0 0 1 0 0 0 0 0 0
022 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
023 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
024 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
025 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
026 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
027 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
028 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
029 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
030 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
031 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0
032 (us): 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (including overflows)
Minimum: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 (us)
Average: 2.000 2.000 2.000 2.000 2.000 2.000 2.000 2.000 2.000 2.000 2.000 2.000 2.000 2.000 (us)
Maximum: 14 13 12 14 12 13 14 20 11 14 12 6 13 13 (us)
Max-Min: 13 12 11 13 11 12 13 19 10 13 11 5 12 12 (us)
Duration: 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 3599.991 (sec)
- is caused by
-
-
- Closed
-