Details
-
Bug
-
Resolution: Duplicate
-
Normal
-
None
-
4.12.0
-
Moderate
-
False
-
Description
Description of problem:
as we filed 4.11 bug for kube-state-metrics: https://bugzilla.redhat.com/show_bug.cgi?id=2069543
the same issue exists in 4.12 and triggered APIRemovedInNextEUSReleaseInUse/APIRemovedInNextReleaseInUse alerts
# oc -n openshift-monitoring logs -c kube-state-metrics $(oc -n openshift-monitoring get pod --no-headers | grep kube-state-metrics | awk '{print $1}') | grep HorizontalPodAutoscaler W1019 07:18:50.931381 1 warnings.go:70] autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.23+, unavailable in v1.26+; use autoscaling/v2 HorizontalPodAutoscaler W1019 07:18:50.936268 1 warnings.go:70] autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.23+, unavailable in v1.26+; use autoscaling/v2 HorizontalPodAutoscaler W1019 07:27:45.938541 1 warnings.go:70] autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.23+, unavailable in v1.26+; use autoscaling/v2 HorizontalPodAutoscaler ... # token=`oc create token prometheus-k8s -n openshift-monitoring` # oc -n openshift-monitoring exec -c prometheus prometheus-k8s-0 -- curl -k -H "Authorization: Bearer $token" 'https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/query?' --data-urlencode 'query=ALERTS{alertname=~"APIRemovedInNextEUSReleaseInUse|APIRemovedInNextReleaseInUse",alertstate="firing"}' | jq { "status": "success", "data": { "resultType": "vector", "result": [ { "metric": { "__name__": "ALERTS", "alertname": "APIRemovedInNextEUSReleaseInUse", "alertstate": "firing", "group": "autoscaling", "namespace": "openshift-kube-apiserver", "removed_release": "1.26", "resource": "horizontalpodautoscalers", "severity": "info", "version": "v2beta2" }, "value": [ 1666167509.532, "1" ] }, { "metric": { "__name__": "ALERTS", "alertname": "APIRemovedInNextReleaseInUse", "alertstate": "firing", "group": "autoscaling", "namespace": "openshift-kube-apiserver", "removed_release": "1.26", "resource": "horizontalpodautoscalers", "severity": "info", "version": "v2beta2" }, "value": [ 1666167509.532, "1" ] } ] } }
and HorizontalPodAutoscaler version is autoscaling/v2
# oc explain horizontalpodautoscalers KIND: HorizontalPodAutoscaler VERSION: autoscaling/v2
Version-Release number of selected component (if applicable):
4.12.0-0.nightly-2022-10-18-192348 kube-state-metrics 2.6.0
How reproducible:
always
Steps to Reproduce:
1. check the alert for the cluster 2. 3.
Actual results:
APIRemovedInNextEUSReleaseInUse/APIRemovedInNextReleaseInUse alerts triggered for horizontalpodautoscalers/v2beta2
Expected results:
no alert
Additional info:
APIRemovedInNextEUSReleaseInUse/APIRemovedInNextReleaseInUse alerts defined in
# oc -n openshift-kube-apiserver get prometheusrules api-usage -oyaml ... spec: groups: - name: pre-release-lifecycle rules: - alert: APIRemovedInNextReleaseInUse annotations: description: Deprecated API that will be removed in the next version is being used. Removing the workload that is using the {{ $labels.group }}.{{ $labels.version }}/{{ $labels.resource }} API might be necessary for a successful upgrade to the next cluster version with Kubernetes {{ $labels.removed_release }}. Refer to `oc get apirequestcounts {{ $labels.resource }}.{{ $labels.version }}.{{ $labels.group }} -o yaml` to identify the workload. summary: Deprecated API that will be removed in the next version is being used. expr: group by (group,version,resource,removed_release) (apiserver_requested_deprecated_apis{removed_release="1.26"}) * on (group,version,resource) group_left () sum by (group,version,resource) ( rate(apiserver_request_total{system_client!="kube-controller-manager",system_client!="cluster-policy-controller"}[4h]) ) > 0 for: 1h labels: namespace: openshift-kube-apiserver severity: info - alert: APIRemovedInNextEUSReleaseInUse annotations: description: Deprecated API that will be removed in the next EUS version is being used. Removing the workload that is using the {{ $labels.group }}.{{ $labels.version }}/{{ $labels.resource }} API might be necessary for a successful upgrade to the next EUS cluster version with Kubernetes {{ $labels.removed_release }}. Refer to `oc get apirequestcounts {{ $labels.resource }}.{{ $labels.version }}.{{ $labels.group }} -o yaml` to identify the workload. summary: Deprecated API that will be removed in the next EUS version is being used. expr: group by (group,version,resource,removed_release) (apiserver_requested_deprecated_apis{removed_release=~"1[.]2[67]"}) * on (group,version,resource) group_left () sum by (group,version,resource) ( rate(apiserver_request_total{system_client!="kube-controller-manager",system_client!="cluster-policy-controller"}[4h]) ) > 0 for: 1h labels: namespace: openshift-kube-apiserver severity: info
Attachments
Issue Links
- links to