-
Bug
-
Resolution: Done-Errata
-
Critical
-
4.16.0
-
+
-
No
-
OCPNODE Sprint 251 (Green)
-
1
-
Proposed
-
False
-
-
Release Note Not Required
-
In Progress
-
Description of problem:
Container process of burstable pods have their cpu affinity not matching with that containers cpuset.cpus.
Version-Release number of selected component (if applicable):
4.16.0
How reproducible:
everytime
Steps to Reproduce:
1. Setup a ocp cluster 2. Label one of the node as worker-cnf node. 3. Create mcp pool for worker-cnf 4. Apply the below Performance profile, where apiVersion: performance.openshift.io/v2 kind: PerformanceProfile metadata: name: performance spec: cpu: isolated: "0-2" reserved: "3" workloadHints: realTime: true net: userLevelNetworking: true nodeSelector: node-role.kubernetes.io/worker-cnf: "" numa: topologyPolicy: single-numa-node realTimeKernel: enabled: false 4. Take any burstable pod (For example ovn-kubenode pod) and select any container in the pod . Under the containers cgroup . Check the cgroup.procs to see the container process pid 5. Execute taskset -pc <pid> to get the container process 6. Check cpuset controller interface file cpuset.cpus 7. Cpu affinity of container process should match with cpuset.cpus
Actual results: Currently container process's affinity matches with reserved cpus and doesn't have access to all the online cpus> Example: below is the output of crictl ps to get the container id of ovn-controller container ce64bd4fbea18 136fb7d44ff690480ae9cc8796c31cfc7c0d9c95c092be11180be478c46b803b 52 minutes ago Running ovn-controller 2 f8802eae98c2d ovnkube-node-kz9tk The container process pid of the above container is : 2932: sh-5.1# crictl inspect ce64bd4fbea18 | grep pid "pid": 2932, "pids": { Get the cgroup path of the container: sh-5.1# crictl inspect ce64bd4fbea18 | grep cgroup "destination": "/sys/fs/cgroup", "type": "cgroup", "source": "cgroup", "cgroupsPath": "kubepods-burstable-pod95dc49f5_b84d_4587_a958_1bc7848fd95b.slice:crio:ce64bd4fbea1873be599b12efa704db236a1bc9626a75123c0d21effff4835ae", Check the cpuaffinity of the pid 2932 sh-5.1# taskset -pc 2932 pid 2932's current affinity list: 3 Change to container cgroup directory: cd /sys/fs/cgroup/cpuset/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95dc49f5_b84d_4587_a958_1bc7848fd95b.slice/crio-ce64bd4fbea1873be599b12efa704db236a1bc9626a75123c0d21effff4835ae.scope/ sh-5.1# cat cgroup.procs 2932 sh-5.1# cat cpuset.cpus 0-3
Expected results:
Burstable pod's Container Process affinity should not have reserved cpus and it's cpu affinity should match with cpuset.cpus
Additional info:
- is related to
-
OCPBUGS-27834 Burstable pods have reserved cpu affinity when performance profile is applied
- Closed
- links to
-
RHEA-2024:0041 OpenShift Container Platform 4.16.z bug fix update