-
Bug
-
Resolution: Obsolete
-
None
-
None
-
5
-
False
-
-
---
-
---
-
-
CNV I/U Operators Sprint 242, CNV Virtualization Sprint 244
-
Urgent
-
No
I'm running a scale regression setup on :
=========================================
OpenShift 4.13.2
OpenShift Virtualization 4.13.1
OpenShift Container Storage - 4.12.4-rhodf
this is a large-scale setup with 132 nodes running 6000 RHEL VMs on an external RHCS.
after powering up 3000 out of 6000 VMs I opened the UI and see thousands of the following warnings regarding virt-handlers memory consumption:
============================================================================================================================================
KubeVirtComponentExceedsRequestedCPU
25 Jun 2023, 11:58
Pod virt-handler-hmztx cpu usage exceeds the CPU requested
View details
KubeVirtComponentExceedsRequestedCPU
25 Jun 2023, 11:58
Pod virt-handler-b7jfm cpu usage exceeds the CPU requested
View details
KubeVirtComponentExceedsRequestedMemory
25 Jun 2023, 11:58
Container virt-handler in pod virt-handler-66x4l memory usage exceeds the memory requested
View details
============================================================================================================================================
from oc adm top:
============================================================================================================================================
virt-handler-x46kr 12m 314Mi
virt-handler-pld9d 21m 315Mi
virt-handler-sfqnh 17m 316Mi
virt-handler-dlh4w 26m 317Mi
virt-handler-lbfj7 17m 317Mi
virt-handler-tcx9l 24m 319Mi
virt-handler-fggsg 18m 321Mi
virt-handler-7gzm8 17m 325Mi
virt-handler-lk9bp 12m 325Mi
virt-handler-gcwfh 18m 329Mi
============================================================================================================================================
if you look at the screenshot I attached you will see that because of it the status of OpenShift Virtualization is at a "Degraded" state
in addition, the thousands of warnings cause the console to significantly slow down and freeze.
I collected the logs but I found it odd that I could not see the above events via cli, note this is a 44G folder when extracted:
============================================================================================================================================
http://perf148h.perf.lab.eng.bos.redhat.com/share/BZ_logs/virthandler_mem_cpu_too_low.tar.gz
- is related to
-
CNV-37900 Deprecate KubeVirtComponentExceedsRequestedCPU and KubeVirtComponentExceedsRequestedMemory alerts
- Closed