Uploaded image for project: 'Observability Documentation'
  1. Observability Documentation
  2. OBSDOCS-183

Allow admins to configure label keys allowed for resource labels' metric

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • None
    • Monitoring
    • 5
    • True
    • Hide

      Blocked because the dev work has not been started.

      Show
      Blocked because the dev work has not been started.
    • False

      Resource labels' metrics report which labels are attached to a particular resource. For instance:

      kube_service_labels{container="kube-rbac-proxy-main", endpoint="https-main", job="kube-state-metrics", namespace="openshift-monitoring", service="alertmanager-main"} 1
      

      By default kube-state-metrics exposes only the "name" and "namespace" labels to avoid performance penalty (resources with many labels increase the amount of memory used by the head index).

      Over time, we've received many requests to allow more labels being exposed and when the request was justified, we tweaked the kube-state-metrics configuration [1][2][3][4]. In the long run, it's not sustainable for the monitoring team to vet which labels should be exposed + it creates friction for customers that rely on labels for their operations.

      CMO needs to offer more flexibility on which resource labels should be exposed (and collected). The default list of "resources to label keys" should be unchanged but a cluster admin should be able to add more (for instance, expose all label keys matching "kubernetes.io/*" for kube_service_labels).

      [1] https://github.com/openshift/cluster-monitoring-operator/pull/1439
      [2] https://github.com/openshift/cluster-monitoring-operator/pull/1421
      [3] https://github.com/openshift/cluster-monitoring-operator/pull/1424
      [4] https://github.com/openshift/cluster-monitoring-operator/pull/1337

              eromanov@redhat.com Eliska Romanova
              rhn-support-bburt Brian Burt
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: