Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-76273

[RHCOS10] platform pods like tuned are pinned to reserved cpus instead of all cpus after applying Performance Profile

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 4.21
    • Node / CRI-O
    • None
    • True
    • Hide

      None

      Show
      None
    • None
    • Critical
    • Yes
    • x86_64
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      On RHEL10.1 RHCOS after applying Performance profile all the platform pods like tuned, ovn-kubenet are pinned to reserved cpus instead of all the cpus of the node.    

      Version-Release number of selected component (if applicable):

          cri-o-1.34.3-2.rhaos4.21.gitd80d17d.el9.x86_64

      How reproducible:

          everytime

      Steps to Reproduce:

      1. Apply the below performance profile:
      apiVersion: performance.openshift.io/v2
      kind: PerformanceProfile
      metadata:
        name: performance
      spec:
        cpu:
          isolated: 8-23
          reserved: 0-7
        hugepages:
          defaultHugepagesSize: 1G
          pages:
          - count: 1
            node: 0
            size: 1G
          - count: 128
            node: 1
            size: 2M
        kernelPageSize: 4k
        machineConfigPoolSelector:
          machineconfiguration.openshift.io/role: worker-cnf
        net:
          userLevelNetworking: true
        nodeSelector:
          node-role.kubernetes.io/worker-cnf: ""
        numa:
          topologyPolicy: single-numa-node
        realTimeKernel:
          enabled: false
        workloadHints:
          highPowerConsumption: true
          perPodPowerManagement: false
          realTime: true
      
      
      2. oc debug node/<worker-cnf> (node on which performance profile is applied)
      3. Find any burstable container process and run `taskset -pc <pid>`
      4. Observe the cpu affinity contains all cpus on the node
      
      Versions: 
      Red Hat Enterprise Linux release 10.1 (Coughlan)
      kernel-modules-extra-matched-6.12.0-124.27.1.el10_1.x86_64
      kernel-core-6.12.0-124.27.1.el10_1.x86_64
      kernel-modules-core-6.12.0-124.27.1.el10_1.x86_64
      kernel-modules-6.12.0-124.27.1.el10_1.x86_64
      kernel-modules-extra-6.12.0-124.27.1.el10_1.x86_64
      kernel-6.12.0-124.27.1.el10_1.x86_64
      registry.ci.openshift.org/rhcos-devel/ocp-4.21-10.1:4.21.0-rc.2-x86_64
      

      Actual results:

      sh-5.2# crictl ps | grep tuned
      f09e737fa1838       50fd4628fe135e8e679960606e983bbcf613e4cf3dd371ae2d81e23a078de8df                                                         2 hours ago         Running             tuned                                6                   1c6b08fa54519       tuned-rjhgm                                                                   openshift-cluster-node-tuning-operator
      sh-5.2# crictl inspect f09e737fa1838 | grep pid
          "pid": 5098,
                "pids": {
      sh-5.2# taskset -pc 5098
      pid 5098's current affinity list: 0-7
      
      sh-5.2# crictl inspect 0ca2e9011a35f | grep pid
          "pid": 5548,
                "pids": {
      sh-5.2# taskset -pc 5548
      pid 5548's current affinity list: 0-7
      
      

       

      Expected results:

          burstable pods should have all the cpus 0-23

      Additional info:

          Refer similar bug filed in RHEL9: https://issues.redhat.com/browse/OCPBUGS-15102

              rh-ee-atokubi Ayato Tokubi
              mniranja Niranjan Mallapadi Raghavendra Rao
              None
              None
              Min Li Min Li
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: