Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: Logging 5.2
Affects Version/s: Logging 5.2
Component/s: Log Storage
Labels:
- no-doc

Blocked:
False
Ready:
False
Docs QE Status:
NEW
QE Status:
NEW

Sprint:
Logging (LogExp) - Sprint 206

SFDC Cases Links:
SFDC Cases Counter:

Description

Description of problem:

The value of card `Total Namespace Count` in Logging/Elasticsearch dashboard equals to $es_data_node_count * es_index_namespaces_total, which I think it's not correct.

For example, in a cluster there are 3 ES pods, the value of metric `es_index_namespaces_total` is 32, but the value of `Total Namespace Count` is 96:

$ curl -k -H "Authorization: Bearer `oc sa get-token prometheus-k8s -n openshift-monitoring`"   -H "Content-type: application/json" https://prometheus-k8s-openshift-monitoring.apps.xxxx.com/api/v1/query?query=es_index_namespaces_total |jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1068    0  1068    0     0   1399      0 --:--:-- --:--:-- --:--:--  1397
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "es_index_namespaces_total",
          "cluster": "elasticsearch",
          "container": "proxy",
          "endpoint": "elasticsearch",
          "instance": "10.128.2.23:60001",
          "job": "elasticsearch-metrics",
          "namespace": "openshift-logging",
          "pod": "elasticsearch-cdm-gfepvhgn-2-f567669f7-6wtwt",
          "service": "elasticsearch-metrics"
        },
        "value": [
          1630375284.642,
          "32"
        ]
      },
      {
        "metric": {
          "__name__": "es_index_namespaces_total",
          "cluster": "elasticsearch",
          "container": "proxy",
          "endpoint": "elasticsearch",
          "instance": "10.129.2.33:60001",
          "job": "elasticsearch-metrics",
          "namespace": "openshift-logging",
          "pod": "elasticsearch-cdm-gfepvhgn-3-554dd4dd88-rp92g",
          "service": "elasticsearch-metrics"
        },
        "value": [
          1630375284.642,
          "32"
        ]
      },
      {
        "metric": {
          "__name__": "es_index_namespaces_total",
          "cluster": "elasticsearch",
          "container": "proxy",
          "endpoint": "elasticsearch",
          "instance": "10.131.0.71:60001",
          "job": "elasticsearch-metrics",
          "namespace": "openshift-logging",
          "pod": "elasticsearch-cdm-gfepvhgn-1-c5966bf74-kphn2",
          "service": "elasticsearch-metrics"
        },
        "value": [
          1630375284.642,
          "32"
        ]
      }
    ]
  }
}