-
Bug
-
Resolution: Unresolved
-
Normal
-
4.21.0
-
None
-
Quality / Stability / Reliability
-
False
-
-
None
-
Moderate
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
case "Alerts shouldn't exceed the series limit of total series sent via telemetry from each cluster" failed on e2e-aws-ovn-techpreview job recently, example
:
[sig-instrumentation][Late] Alerts shouldn't exceed the series limit of
total series sent via telemetry from each cluster
[Suite:openshift/conformance/parallel]
{ fail [github.com/openshift/origin/test/extended/prometheus/prometheus.go:564]: Unexpected error:
<errors.aggregate | len:1, cap:1>:
promQL query returned unexpected results:
avg_over_time(cluster:telemetry_selected_series:count[1h22m50s]) >= 780
[
{
"metric": {
"prometheus": "openshift-monitoring/k8s"
},
"value": [
1758534739.622,
"784.5"
]
}
]
[
<*errors.errorString | 0xc00735a8b0>{
s: "promQL query returned unexpected results:\navg_over_time(cluster:telemetry_selected_series:count[1h22m50s]) >= 780\n[\n {\n \"metric\": {\n \"prometheus\": \"openshift-monitoring/k8s\"\n },\n \"value\": [\n 1758534739.622,\n \"784.5\"\n ]\n }\n]",
},
]
:
[sig-instrumentation][Late] Alerts shouldn't exceed the series limit of
total series sent via telemetry from each cluster
[Suite:openshift/conformance/parallel]
{ fail [github.com/openshift/origin/test/extended/prometheus/prometheus.go:564]: Unexpected error:
<errors.aggregate | len:1, cap:1>:
promQL query returned unexpected results:
avg_over_time(cluster:telemetry_selected_series:count[1h37m56s]) >= 780
[
{
"metric": {
"prometheus": "openshift-monitoring/k8s"
},
"value": [
1758776352.201,
"788.4744897959183"
]
}
]
[
<*errors.errorString | 0xc0074b8030>{
s: "promQL query returned unexpected results:\navg_over_time(cluster:telemetry_selected_series:count[1h37m56s]) >= 780\n[\n {\n \"metric\": {\n \"prometheus\": \"openshift-monitoring/k8s\"\n },\n \"value\": [\n 1758776352.201,\n \"788.4744897959183\"\n ]\n }\n]",
},
]
:
[sig-instrumentation][Late] Alerts shouldn't exceed the series limit of
total series sent via telemetry from each cluster
[Suite:openshift/conformance/parallel]
{ fail [github.com/openshift/origin/test/extended/prometheus/prometheus.go:564]: Unexpected error:
<errors.aggregate | len:1, cap:1>:
promQL query returned unexpected results:
avg_over_time(cluster:telemetry_selected_series:count[1h39m48s]) >= 780
[
{
"metric": {
"prometheus": "openshift-monitoring/k8s"
},
"value": [
1758728450.108,
"788.0301507537688"
]
}
]
[
<*errors.errorString | 0xc0060bb0d0>{
s: "promQL query returned unexpected results:\navg_over_time(cluster:telemetry_selected_series:count[1h39m48s]) >= 780\n[\n {\n \"metric\": {\n \"prometheus\": \"openshift-monitoring/k8s\"\n },\n \"value\": [\n 1758728450.108,\n \"788.0301507537688\"\n ]\n }\n]",
},
]
occurred}
maybe change the limit to 790 is fine
https://github.com/openshift/origin/blob/main/test/extended/prometheus/prometheus.go#L544
Version-Release number of selected component (if applicable):
4.21
How reproducible:
frequently
Steps to Reproduce:
1. see the steps
Actual results:
case "Alerts shouldn't exceed the series limit of total series sent via telemetry from each cluster" failed on e2e-aws-ovn-techpreview job
Expected results:
pass without error
Additional info:
