Uploaded image for project: 'OpenShift Container Platform (OCP) Strategy'
  1. OpenShift Container Platform (OCP) Strategy
  2. OCPSTRAT-2346

Reduce High Cardinality in container_network_* Metrics for Host-Networked Pods After Switching to OVN

XMLWordPrintable

    • Product / Portfolio Work
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Feature Overview (aka. Goal Summary)  

      After migrating from OpenShift SDN to OVN-Kubernetes, users have reported a significant increase in the number of container_network_ metric series generated by {}hostNetwork-enabled pods. This change results in {}high metric cardinality{*}, particularly on large clusters where numerous ephemeral network interfaces are reported per pod.

      With OVN, interfaces for all pods on the node (even those not using host networking) are exposed in Prometheus metrics, unlike SDN where this was limited to a smaller, more stable set of interfaces. As a result, Prometheus TSDB becomes overloaded, consuming excessive CPU and memory and risking instability of monitoring infrastructure.

      Goals (aka. expected user outcomes)

      Prevent metric cardinality explosion on clusters using OVN

              gausingh@redhat.com Gaurav Singh
              gausingh@redhat.com Gaurav Singh
              None
              Peter Hunt
              None
              None
              None
              Derrick Ornelas Derrick Ornelas
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: