Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-31905

On SNO with DU profile(RT kernel) tuned profile is always degraded due to net.core.busy_read, net.core.busy_poll and kernel.numa_balancing sysctl not existing in RT kernel

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Undefined Undefined
    • None
    • 4.14.z
    • Node Tuning Operator
    • None
    • Important
    • No
    • CNF Compute Sprint 252
    • 1
    • False
    • Hide

      None

      Show
      None
    • Hide
      *Cause*: These sysctl parameters (kernel.numa_balancing, net.core.busy_poll and net.core.busy_read) seem not to be available with RT kernel.
      *Consequence*: On OCP with RT kernel tuned profile is always degraded
      *Fix*: Added the openshift-node-performance-rt profile that would be included if an RT kernel is detected, thereby dropping the unsupported kernel parameters before they are applied.
      *Result*: Bug doesn’t present anymore.
      Show
      *Cause*: These sysctl parameters (kernel.numa_balancing, net.core.busy_poll and net.core.busy_read) seem not to be available with RT kernel. *Consequence*: On OCP with RT kernel tuned profile is always degraded *Fix*: Added the openshift-node-performance-rt profile that would be included if an RT kernel is detected, thereby dropping the unsupported kernel parameters before they are applied. *Result*: Bug doesn’t present anymore.
    • Bug Fix
    • In Progress

      This is a clone of issue OCPBUGS-23167. The following is the description of the original issue:

      Description of problem:

      On SNO with DU profile(RT kernel) tuned profile is always degraded due to net.core.busy_read, net.core.busy_poll and kernel.numa_balancing sysctl not existing in RT kernel

      Version-Release number of selected component (if applicable):

      4.14.1

      How reproducible:

      100%

      Steps to Reproduce:

      1. Deploy SNO with DU profile(RT kernel)
      2. Check tuned profile
      

      Actual results:

      oc -n openshift-cluster-node-tuning-operator get profile -o yaml
      apiVersion: v1
      items:
      - apiVersion: tuned.openshift.io/v1
        kind: Profile
        metadata:
          creationTimestamp: "2023-11-09T18:26:34Z"
          generation: 2
          name: sno.kni-qe-1.lab.eng.rdu2.redhat.com
          namespace: openshift-cluster-node-tuning-operator
          ownerReferences:
          - apiVersion: tuned.openshift.io/v1
            blockOwnerDeletion: true
            controller: true
            kind: Tuned
            name: default
            uid: 4e7c05a2-537e-4212-9009-e2724938dec9
          resourceVersion: "287891"
          uid: 5f4d5819-8f84-4b3b-9340-3d38c41501ff
        spec:
          config:
            debug: false
            tunedConfig: {}
            tunedProfile: performance-patch
        status:
          conditions:
          - lastTransitionTime: "2023-11-09T18:26:39Z"
            message: TuneD profile applied.
            reason: AsExpected
            status: "True"
            type: Applied
          - lastTransitionTime: "2023-11-09T18:26:39Z"
            message: 'TuneD daemon issued one or more error message(s) during profile application.
              TuneD stderr: net.core.rps_default_mask'
            reason: TunedError
            status: "True"
            type: Degraded
          tunedProfile: performance-patch
      kind: List
      metadata:
        resourceVersion: ""
      

      Expected results:

      Not degraded

      Additional info:

      Looking at the tuned log the following errors show up which are probably causing the profile to get into degraded state:
      
      2023-11-09 18:30:49,287 ERROR    tuned.plugins.plugin_sysctl: Failed to read sysctl parameter 'net.core.busy_read', the parameter does not exist
      2023-11-09 18:30:49,287 ERROR    tuned.plugins.plugin_sysctl: sysctl option net.core.busy_read will not be set, failed to read the original value.
      2023-11-09 18:30:49,287 ERROR    tuned.plugins.plugin_sysctl: Failed to read sysctl parameter 'net.core.busy_poll', the parameter does not exist
      2023-11-09 18:30:49,287 ERROR    tuned.plugins.plugin_sysctl: sysctl option net.core.busy_poll will not be set, failed to read the original value.
      2023-11-09 18:30:49,287 ERROR    tuned.plugins.plugin_sysctl: Failed to read sysctl parameter 'kernel.numa_balancing', the parameter does not exist
      2023-11-09 18:30:49,287 ERROR    tuned.plugins.plugin_sysctl: sysctl option kernel.numa_balancing will not be set, failed to read the original value.
      
      These sysctl parameters seem not to be available with RT kernel.

              rh-ee-rbaturov Ronny Baturov
              openshift-crt-jira-prow OpenShift Prow Bot
              Shereen Haj Shereen Haj
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: