Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-4362

Hard eviction thresholds is different with k8s default when PAO is enabled

    XMLWordPrintable

Details

    • False
    • Hide

      None

      Show
      None
    • Previously, kubelet hard eviction thresholds were different from k8s defaults when a Performance profile was applied to a node. With this release, the defaults have been updated to match the expected kubernetes defaults.
    • Bug Fix
    • 12/19: per Martin, there's a simple workaround for this - just need to close on if that needs to be release noted or if the fix will land for 4.12

    Description

      This is a clone of issue OCPBUGS-2209. The following is the description of the original issue:
      —
      Description of problem:

      Hard eviction thresholds is different with k8s default when PAO is enabled. 

According to https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#hard-eviction-thresholds, the default Hard eviction thresholds in k8s are: 
memory.available<100Mi nodefs.available<10% imagefs.available<15% nodefs.inodesFree<5% (Linux nodes), but when PAO is enabled, the default setting is changed to memory.available<100Mi: https://github.com/openshift/cluster-node-tuning-operator/blob/master/pkg/performanceprofile/controller/performanceprofile/components/kubeletconfig/kubeletconfig.go#L59-L64

There is no doc mentioned this difference, considering 'nodefs.available<10%' will impact the calculation of node allocatable ephemeral-storage, the customer is seeing different allocatable ephemeral-storage with/without PAO enabled which brought confusions.

Considering there was no special reason to overwrite the default settings except that the kubeconfig needs a non-nil value as a default, and there is doc to describe that PAO would change the default behavior, this should be addressed.

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Always

      Steps to Reproduce:

      1. Prepare a OCP cluster
2. Install the PAO operator and create a performanceprofile
3.

      Actual results:

      The kubelet config contains a settings of  "evictionHard": { "memory.available": "100Mi"  }

      Expected results:

      The kubelet config should not change the k8s default evictionHard settings unless there is a reason

      Additional info:

       

      Attachments

        Issue Links

          clones

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-2209 Hard eviction thresholds is different with k8s default when PAO is enabled

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          is blocked by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-2209 Hard eviction thresholds is different with k8s default when PAO is enabled

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          links to

          GitHub openshift/cluster-node-tuning-operator#520: [release-4.12] OCPBUGS-4362: Fix default hard eviction threshold when PCC is applied

          Activity

            People

              msivak@redhat.com Martin Sivak
              openshift-crt-jira-prow OpenShift Prow Bot
              Mallapadi Niranjan Mallapadi Niranjan
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: