Uploaded image for project: 'OpenShift Container Platform (OCP) Strategy'
  1. OpenShift Container Platform (OCP) Strategy
  2. OCPSTRAT-2346

Reduce High Cardinality in container_network_* Metrics for Host-Networked Pods After Switching to OVN : TP 4.22

XMLWordPrintable

    • Product / Portfolio Work
    • None
    • True
    • Hide

      Technology Preview: CRI Stats Integration in CRI-O (KEP-2371 Alignment) 

      https://issues.redhat.com/browse/OCPSTRAT-2556 

      Show
      Technology Preview: CRI Stats Integration in CRI-O (KEP-2371 Alignment)  https://issues.redhat.com/browse/OCPSTRAT-2556  
    • False
    • None
    • None
    • Tech Preview
    • None
    • None
    • None
    • None
    • None

      Feature Overview (aka. Goal Summary)  

      After migrating from OpenShift SDN to OVN-Kubernetes, users have reported a significant increase in the number of container_network_ metric series generated by {}hostNetwork-enabled pods. This change results in {}high metric cardinality{*}, particularly on large clusters where numerous ephemeral network interfaces are reported per pod.

      With OVN, interfaces for all pods on the node (even those not using host networking) are exposed in Prometheus metrics, unlike SDN where this was limited to a smaller, more stable set of interfaces. As a result, Prometheus TSDB becomes overloaded, consuming excessive CPU and memory and risking instability of monitoring infrastructure.

      Goals (aka. expected user outcomes)

      Prevent metric cardinality explosion on clusters using OVN

              gausingh@redhat.com Gaurav Singh
              gausingh@redhat.com Gaurav Singh
              None
              Damien Grisonnet
              Mrunal Patel Mrunal Patel
              Aruna Naik Aruna Naik
              Matthew Werner Matthew Werner
              Derrick Ornelas Derrick Ornelas
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: