Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-49453

SystemMemoryExceedsReservation triggered when autoSizingReserved enabled

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Undefined Undefined
    • None
    • 4.16.z
    • Node / Kubelet
    • Important
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      SystemMemoryExceedsReservation triggered when autoSizingReserved enabled    

      Version-Release number of selected component (if applicable):

          4.16.30

      How reproducible:

          100%

      Steps to Reproduce:

          1. Enable autoSizingReserved
          2. Add a workload to the cluster
          3. Observe alerts
          

      Actual results:

      SystemMemoryExceedsReservation
      
      System memory usage of 1.74G on <HOST> exceeds 95% of the reservation. Reserved memory ensures system processes can function even when the node is fully allocated and protects against workload out of memory events impacting the proper functioning of the node. The default reservation is expected to be sufficient for most configurations and should be increased (https://docs.openshift.com/container-platform/latest/nodes/nodes/nodes-nodes-managing.html) when running nodes with high numbers of pods (either due to rate of change or at steady state).
      View details

      Expected results:

          No alert

      Additional info:

      $ oc debug {node} -- chroot /host cat /etc/node-sizing.env'
      SYSTEM_RESERVED_MEMORY=1Gi
      SYSTEM_RESERVED_CPU=500m
      SYSTEM_RESERVED_ES=1Gi
      
      $ oc debug {node} -- chroot /host /usr/local/sbin/dynamic-system-reserved-calc.sh true 1Gi 500m 1Gi
      
      $ oc debug {node} -- chroot /host cat /etc/node-sizing.env'
      SYSTEM_RESERVED_MEMORY=17Gi
      SYSTEM_RESERVED_CPU=0.35
      SYSTEM_RESERVED_ES=1Gi
      
      <REBOOT>
      
      $ oc debug {node} -- chroot /host cat /etc/node-sizing.env'
      SYSTEM_RESERVED_MEMORY=1Gi
      SYSTEM_RESERVED_CPU=500m
      SYSTEM_RESERVED_ES=1Gi
      
      
      
      Setting values manually removes the alert
        kubeletConfig:
          systemReserved: 
            cpu: 2000m
            memory: 17Gi
            ephemeral-storage: 1Gi

       

              svanka@redhat.com Sai Ramesh Vanka
              rhn-support-sbennert Sarah Bennert
              Cameron Meadors Cameron Meadors
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: