-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
4.18
-
None
-
None
-
False
-
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
The HPA now contains feature that allow for scaling on `containerResource`
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
labels:
name: hpa-hello
namespace: test-metrics
spec:
minReplicas: 1
maxReplicas: 2
metrics:
- containerResource:
container: hello-1
name: cpu
target:
averageUtilization: 90
type: Utilization
type: ContainerResource
- containerResource:
container: hello-2
name: cpu
target:
averageUtilization: 90
type: Utilization
type: ContainerResource
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: hello
When the above configuration is used, PrometheusDuplicateTimestamps alerts are generated.
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
namespace: test-metrics
labels:
app: hello
version: v1
spec:
strategy:
type: Recreate
replicas: 1
selector:
matchLabels:
app: hello
template:
metadata:
labels:
app: hello
version: v1
spec:
automountServiceAccountToken: false
containers:
- name: hello-1
image: registry.access.redhat.com/ubi9/ubi-minimal:latest
command: ["/bin/bash", "-c", "echo 'Hello from container 1'; sleep infinity"]
resources:
requests:
cpu: 50m
memory: 128Mi
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
- name: hello-2
image: registry.access.redhat.com/ubi9/ubi-minimal:latest
command: ["/bin/bash", "-c", "echo 'Hello from container 2'; sleep infinity"]
resources:
requests:
cpu: 50m
memory: 128Mi
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
currently the metrics used by the HPA - `kube_horizontalpodautoscaler_spec_target_metric` does not have a `containername` label
oc exec -n openshift-monitoring prometheus-k8s-0 -- curl -s -k --cert /etc/prometheus/secrets/metrics-client-certs/tls.crt --key /etc/prometheus/secrets/metrics-client-certs/tls.key https://kube-state-metrics.openshift-monitoring.svc:8443/metrics | uniq -D kube_horizontalpodautoscaler_spec_target_metric{namespace="test-metrics",horizontalpodautoscaler="hpa-hello",metric_name="cpu",metric_target_type="utilization"} 90 kube_horizontalpodautoscaler_spec_target_metric{namespace="test-metrics",horizontalpodautoscaler="hpa-hello",metric_name="cpu",metric_target_type="utilization"} 90
The following upstream issue has been logged - https://github.com/kubernetes/kube-state-metrics/issues/2403