-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
4.18
-
None
-
Yes
-
Rejected
-
False
-
Description of problem:
[kube-controller-manager] `kube_persistentvolume_plugin_type_counts` metrics does not get the correct plugin_name
Version-Release number of selected component (if applicable):
4.18.0-0.nightly-2024-11-15-113437
How reproducible:
Always
Steps to Reproduce:
1. Install an Openshift cluster on AWS.
2. Create one pvc use the gp3-csi storageclass and pod consume the pvc.
3. Wait for the volume provisioned succeed and check the cluster:kube_persistentvolume_plugin_type_counts:sum metric data as expected.
Actual results:
In step 3: the cluster:kube_persistentvolume_plugin_type_counts:sum metric data does not get the correct plugin_name
$ token=`oc create token prometheus-k8s -n openshift-monitoring`
$ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -ks -H "Authorization: Bearer $token" 'https://thanos-querier.openshift-monitoring.svc:9091/api/v1/query?' --data-urlencode 'query=cluster:kube_persistentvolume_plugin_type_counts:sum'|jq .
{
"status": "success",
"data": {
"resultType": "vector",
"result": [
{
"metric": {
"__name__": "cluster:kube_persistentvolume_plugin_type_counts:sum",
"plugin_name": "N/A",
"prometheus": "openshift-monitoring/k8s",
"volume_mode": "Filesystem"
},
"value": [
1732155453.690,
"1"
]
}
],
"analysis": {}
}
}
Expected results:
In step 3: the cluster:kube_persistentvolume_plugin_type_counts:sum metric data plugin_name should be "plugin_name": "kubernetes.io/csi:ebs.csi.aws.com" consistent with 4.17.z.
# 4.17 metic data
$ oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -ks -H "Authorization: Bearer $token" 'https://thanos-querier.openshift-monitoring.svc:9091/api/v1/query?' --data-urlencode 'query=cluster:kube_persistentvolume_plugin_type_counts:sum'|jq .
{
"status": "success",
"data": {
"resultType": "vector",
"result": [
{
"metric": {
"__name__": "cluster:kube_persistentvolume_plugin_type_counts:sum",
"plugin_name": "kubernetes.io/csi:ebs.csi.aws.com",
"prometheus": "openshift-monitoring/k8s",
"volume_mode": "Filesystem"
},
"value": [
1732156441.173,
"3"
]
}
],
"analysis": {}
}
}
Additional info:
The metric `cluster:kube_persistentvolume_plugin_type_counts:sum` is from https://github.com/openshift/cluster-monitoring-operator/blob/master/assets/cluster-monitoring-operator/prometheus-rule.yaml#L224-L225 actually from the source metric `pv_collector_total_pv_count` which collected by kcm is already missed the plugin_name(N/A).