-
Bug
-
Resolution: Duplicate
-
Undefined
-
None
-
4.14.z
-
None
-
Important
-
No
-
False
-
-
Description of problem:
Performing the hardware reboot on each MNO nodes the tuned enters the following state: NAMESPACE NAME TUNED APPLIED DEGRADED AGE openshift-cluster-node-tuning-operator master0.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 10h openshift-cluster-node-tuning-operator master1.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 12h openshift-cluster-node-tuning-operator master2.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 11h openshift-cluster-node-tuning-operator worker0.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-worker True True 11h openshift-cluster-node-tuning-operator worker1.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-worker True False 11h
Version-Release number of selected component (if applicable):
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.14.14 True False 11h Cluster version is 4.14.14
How reproducible:
80%
Steps to Reproduce:
1.Apply the PAO and Tuned patch to the cluster highlited in the following support case: https://access.redhat.com/support/cases/#/case/03708995/discussion?commentId=a0a6R00000WCpsBQAT 2. Perform reboot of each individual node in the cluster for 24h 3.
Actual results:
NAMESPACE NAME TUNED APPLIED DEGRADED AGE openshift-cluster-node-tuning-operator master0.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 10h openshift-cluster-node-tuning-operator master1.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 12h openshift-cluster-node-tuning-operator master2.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 11h openshift-cluster-node-tuning-operator worker0.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-worker True True 11h openshift-cluster-node-tuning-operator worker1.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-worker True False 11h apiVersion: tuned.openshift.io/v1 kind: Profile metadata: creationTimestamp: "2024-03-19T19:20:50Z" generation: 4 name: worker0.winterfell-mno-2.lab.neat.nsn-rdnet.net namespace: openshift-cluster-node-tuning-operator ownerReferences: - apiVersion: tuned.openshift.io/v1 blockOwnerDeletion: true controller: true kind: Tuned name: default uid: 5ab6d677-ef3e-4eaf-98c5-8aa3038c9f5f resourceVersion: "138405" uid: dedf4e04-a88c-4961-98a6-0b971653f5f2 spec: config: debug: false providerName: baremetalhost tunedConfig: {} tunedProfile: rdpmc-patch-worker status: conditions: - lastTransitionTime: "2024-03-19T19:58:08Z" message: TuneD profile applied. reason: AsExpected status: "True" type: Applied - lastTransitionTime: "2024-03-19T21:34:23Z" message: 'TuneD daemon issued one or more error message(s) during profile application. TuneD stderr: ERROR tuned.plugins.plugin_scheduler: Failed to set affinity of PID 1110 to ''[29]'': [Errno 22] Invalid argument' reason: TunedError status: "True" type: Degraded tunedProfile: rdpmc-patch-worker $ omc logs -n openshift-cluster-node-tuning-operator tuned-4p4qm | grep ERROR 2024-03-19T21:34:20.242693269Z 2024-03-19 21:34:20,242 ERROR tuned.plugins.plugin_scheduler: Failed to set SMP affinity of IRQ 476 to 'ffffffff,ffffffff': [Errno 2] No such file or directory: '/proc/irq/476/smp_affinity' 2024-03-19T21:34:20.242855820Z 2024-03-19 21:34:20,242 ERROR tuned.plugins.plugin_scheduler: Failed to set SMP affinity of IRQ 493 to 'ffffffff,ffffffff': [Errno 2] No such file or directory: '/proc/irq/493/smp_affinity' 2024-03-19T21:34:20.243051936Z 2024-03-19 21:34:20,243 ERROR tuned.plugins.plugin_scheduler: Failed to set SMP affinity of IRQ 510 to 'ffffffff,ffffffff': [Errno 2] No such file or directory: '/proc/irq/510/smp_affinity' 2024-03-19T21:34:20.243449807Z 2024-03-19 21:34:20,243 ERROR tuned.plugins.plugin_scheduler: Failed to set SMP affinity of IRQ 517 to 'ffffffff,ffffffff': [Errno 2] No such file or directory: '/proc/irq/517/smp_affinity' 2024-03-19T21:34:20.243654335Z 2024-03-19 21:34:20,243 ERROR tuned.plugins.plugin_scheduler: Failed to set SMP affinity of IRQ 519 to 'ffffffff,ffffffff': [Errno 2] No such file or directory: '/proc/irq/519/smp_affinity' 2024-03-19T21:34:20.244274428Z 2024-03-19 21:34:20,243 ERROR tuned.plugins.plugin_scheduler: Failed to set SMP affinity of IRQ 595 to 'ffffffff,ffffffff': [Errno 2] No such file or directory: '/proc/irq/595/smp_affinity' 2024-03-19T21:34:20.403920714Z 2024-03-19 21:34:20,403 ERROR tuned.plugins.plugin_scheduler: Failed to set affinity of PID 1110 to '[29]': [Errno 22] Invalid argument 2024-03-19T21:34:25.816277984Z 2024-03-19 21:34:25,816 ERROR tuned.plugins.plugin_scheduler: Failed to set affinity of PID 1110 to '[29]': [Errno 22] Invalid argument
Expected results:
NAMESPACE NAME TUNED APPLIED DEGRADED AGE openshift-cluster-node-tuning-operator master0.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 11h openshift-cluster-node-tuning-operator master1.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 13h openshift-cluster-node-tuning-operator master2.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-master True False 13h openshift-cluster-node-tuning-operator worker0.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-worker True False 13h openshift-cluster-node-tuning-operator worker1.winterfell-mno-2.lab.neat.nsn-rdnet.net rdpmc-patch-worker True False 13h
Additional info:
- duplicates
-
OCPBUGS-31844 tuned: tuned breaks dynamic IRQ affinity
- Closed
- is incorporated by
-
OCPBUGS-26400 tuned: tuned breaks dynamic IRQ affinity
- Closed