Uploaded image for project: 'Machine Config Operator'
  1. Machine Config Operator
  2. MCO-1587

Add runbook for ExtremelyHighIndividualControlPlaneMemory alert

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • 0

      MCO will send an alert when a node  for 45 minutes, when a control plane node has extremely high memory usage

      The alerts describes the following 

      "summary: >-
                    Extreme memory utilization per node within control plane nodes is extremely high, and could impact responsiveness and stability.
                  description: >-
                    The memory utilization per instance within control plane nodes influence the stability, and responsiveness of the cluster.
                    This can lead to cluster instability and slow responses from kube-apiserver or failing requests especially on etcd.
                    Moreover, OOM kill is expected which negatively influences the pod scheduling.
                    If this happens on container level, the descheduler will not be able to detect it, as it works on the pod level.
                    To fix this, increase memory of the affected node of control plane nodes."

      It is possible that admin may not be able to interpret exact action to be taken after looking at the alert and the cluster state. Adding runbook (https://github.com/openshift/runbooks) can help admin in better troubleshooting and taking appropriate action.

       

      Acceptance Criteria:

      • Runbook doc is created for ExtremelyHighIndividualControlPlaneMemory alert
      • Created runbook link is accessible to cluster admin with ExtremelyHighIndividualControlPlaneMemory alert

       

              rhn-support-cruhm Courtney Ruhm
              rhn-support-cruhm Courtney Ruhm
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: