Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-5873

etcd_object_counts is deprecated and replaced with apiserver_storage_objects, causing "etcd Object Count" dashboard to only show OpenShift resources

XMLWordPrintable

    • Important
    • None
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Bug Fix
    • Done

      Description of problem:

      Reviewing https://console-openshift-console.apps.example.com/monitoring/dashboards/grafana-dashboard-api-performance?apiserver=kube-apiserver shows only OpenShift related resources in "etcd Object Count".
      
      This is based on the fact that `etcd_object_counts` was deprecated in `kubernetes` version 1.21 and replaced with `apiserver_storage_objects` (see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md#other-cleanup-or-flake-3).
      
      When running the query `topk(25, max(etcd_object_counts) by (resource))` from the above mentioned dashboard, only OpenShift resources are shown. But when running the query with `topk(25, max(apiserver_storage_objects) by (resource))` query it again shows all the expected resources. 
      
      This means the dashboard on "/monitoring/dashboards/grafana-dashboard-api-performance?apiserver=kube-apiserver" showing "etcd Object Count" should be fixed and rather use `apiserver_storage_objects`.

      Version-Release number of selected component (if applicable):

      OpenShift Container Platform 4.13.0-ec.1

      How reproducible:

      - Always

      Steps to Reproduce:

      1. Install OpenShift Container Platform 4.10
      2. Access https://console-openshift-console.apps.example.com/monitoring/dashboards/grafana-dashboard-api-performance?apiserver=kube-apiserver and view "etcd Object Count" 

      Actual results:

      Only resources from `openshift-apiserver` are being reported. Core `kubernetes` resources and others are missing.
      

      Expected results:

      All resources within `etcd` to be reported and shown to understand how much object of each resource is available in `etcd`
      

      Additional info:

      This was raised a while ago in https://bugzilla.redhat.com/show_bug.cgi?id=2110365 and is still a problem in early OpenShift Container Platform 4.13 releases.

       

        1. dashboard-1.png
          dashboard-1.png
          77 kB
        2. dashboard-2.png
          dashboard-2.png
          73 kB
        3. dashboard-3.png
          dashboard-3.png
          108 kB

              dgrisonn@redhat.com Damien Grisonnet
              rhn-support-sreber Simon Reber
              Ke Wang Ke Wang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: