Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-5873

etcd_object_counts is deprecated and replaced with apiserver_storage_objects, causing "etcd Object Count" dashboard to only show OpenShift resources

    XMLWordPrintable

Details

    • Important
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Bug Fix
    • Done

    Description

      Description of problem:

      Reviewing https://console-openshift-console.apps.example.com/monitoring/dashboards/grafana-dashboard-api-performance?apiserver=kube-apiserver shows only OpenShift related resources in "etcd Object Count".
      
      This is based on the fact that `etcd_object_counts` was deprecated in `kubernetes` version 1.21 and replaced with `apiserver_storage_objects` (see https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md#other-cleanup-or-flake-3).
      
      When running the query `topk(25, max(etcd_object_counts) by (resource))` from the above mentioned dashboard, only OpenShift resources are shown. But when running the query with `topk(25, max(apiserver_storage_objects) by (resource))` query it again shows all the expected resources. 
      
      This means the dashboard on "/monitoring/dashboards/grafana-dashboard-api-performance?apiserver=kube-apiserver" showing "etcd Object Count" should be fixed and rather use `apiserver_storage_objects`.

      Version-Release number of selected component (if applicable):

      OpenShift Container Platform 4.13.0-ec.1

      How reproducible:

      - Always

      Steps to Reproduce:

      1. Install OpenShift Container Platform 4.10
      2. Access https://console-openshift-console.apps.example.com/monitoring/dashboards/grafana-dashboard-api-performance?apiserver=kube-apiserver and view "etcd Object Count" 

      Actual results:

      Only resources from `openshift-apiserver` are being reported. Core `kubernetes` resources and others are missing.
      

      Expected results:

      All resources within `etcd` to be reported and shown to understand how much object of each resource is available in `etcd`
      

      Additional info:

      This was raised a while ago in https://bugzilla.redhat.com/show_bug.cgi?id=2110365 and is still a problem in early OpenShift Container Platform 4.13 releases.

       

      Attachments

        1. dashboard-1.png
          dashboard-1.png
          77 kB
        2. dashboard-2.png
          dashboard-2.png
          73 kB
        3. dashboard-3.png
          dashboard-3.png
          108 kB

        Activity

          People

            dgrisonn@redhat.com Damien Grisonnet
            rhn-support-sreber Simon Reber
            Ke Wang Ke Wang
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: