-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
4.17, 4.16.z
-
Important
-
None
-
Rejected
-
False
-
Description of problem:
In HyperShift guest env, Admin can't query metrics of components which are under namespace "clusters-<hostedcluster name>", although those components in HyperShift guest env indeed have exposed their metrics.
Version-Release number of selected component (if applicable):
Latest 4.17 and 4.16 nightly payloads.
How reproducible:
Always
Steps to Reproduce:
To demonstrate this bug issue, below uses oauth component's metrics "openshift_auth_basic_password_count_result" for example. But note, the issue seems to exist for HyperShift guest cluster's all components' under namespace "clusters-<hostedcluster name>".
1. Run some oc login -u xxx -p xxx commands against HyperShift guest cluster. Login to HyperShift guest cluster's management console with cluster-admin user. Navigate to "Observe" -> "Metrics", input query "openshift_auth_basic_password_count_result" and run queries.
Actual results:
1. The console shows "No datapoints found", see screenshot https://drive.google.com/file/d/1Pr4hLILhu4ZiLoPtUuVmmbEkYZddvKDo/view?usp=drive_link . But the endpoint pods of the component indeed have exposed metrics, see:
# Hosted cluster's oauth-openshift backend pods are in namespace clusters-<hostedcluster name>:
$ oc get -n clusters-hypershift-ci-298077 pod | grep oauth-openshift
oauth-openshift-78b5f7687d-bk6wg 3/3 Running 0 149m
oauth-openshift-78b5f7687d-kxrbf 3/3 Running 0 149m
oauth-openshift-78b5f7687d-zk857 3/3 Running 0 149m
$ POD=oauth-openshift-78b5f7687d-kxrbf
# curl the pod's /metrics, the metrics have indeed been exposed:
$ oc exec -n clusters-hypershift-ci-298077 -c oauth-server $POD -- bash -c "unset ALL_PROXY HTTPS_PROXY; curl -sS -k -H 'Authorization: bearer $TOKEN' https://127.0.0.1:6443/metrics" | grep openshift_auth_basic_password_count_result
# HELP openshift_auth_basic_password_count_result [ALPHA] Counts basic password authentication attempts by result
# TYPE openshift_auth_basic_password_count_result counter
openshift_auth_basic_password_count_result{result="error"} 1
openshift_auth_basic_password_count_result{result="failure"} 1
openshift_auth_basic_password_count_result{result="success"} 5
Expected results:
In HyperShift guest env, Admin should be able to query metrics of components which are under namespace "clusters-<hostedcluster name>"
Additional info:
If it is expected to be unable to query metrics of components which are under namespace "clusters-<hostedcluster name>" in In HyperShift guest env, this issue should be well documented otherwise customers are confused to see the management console's "Observe" -> "Metrics" does not work for those metrics.