-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
4.14.0
-
None
-
Quality / Stability / Reliability
-
False
-
-
None
-
Low
-
No
-
None
-
None
-
None
-
MON Sprint 243, MON Sprint 245, MON Sprint 248
-
3
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
4.14.0-0.nightly-2023-08-11-055332 cluster, enable UWM, then will see error log "Operation cannot be fulfilled on thanosrulers.monitoring.coreos.com \"user-workload\"" in UWM prometheus-operator, the error does not affect the function
$ oc logs deployment/prometheus-operator -c prometheus-operator -n openshift-user-workload-monitoring level=info ts=2023-08-14T02:47:16.060727889Z caller=main.go:239 msg="Starting Prometheus Operator" version="(version=0.67.1, branch=rhaos-4.14-rhel-8, revision=f717460)" level=info ts=2023-08-14T02:47:16.060802209Z caller=main.go:240 build_context="(go=go1.20.5 X:strictfipsruntime, platform=linux/arm64, user=root, date=20230809-05:28:40, tags=strictfipsruntime)" level=warn ts=2023-08-14T02:47:16.081655528Z caller=main.go:107 msg="scrapeconfigs CRD not supported" reason="unmet prerequisites: scrapeconfigs resource (apiGroup: \"monitoring.coreos.com/v1alpha1\") not installed" level=info ts=2023-08-14T02:47:16.083679328Z caller=operator.go:356 component=prometheusoperator msg="Kubernetes API capabilities" endpointslices=true level=warn ts=2023-08-14T02:47:16.084642488Z caller=main.go:107 msg="prometheusagents CRD not supported" reason="unmet prerequisites: prometheusagents resource (apiGroup: \"monitoring.coreos.com/v1alpha1\") not installed" level=info ts=2023-08-14T02:47:16.086469088Z caller=main.go:119 msg="Starting insecure server on 127.0.0.1:8080" level=info ts=2023-08-14T02:47:16.087361888Z caller=operator.go:313 component=thanosoperator msg="connection established" cluster-version=v1.27.4+deb2c60 level=info ts=2023-08-14T02:47:16.087387448Z caller=operator.go:322 component=thanosoperator msg="CRD API endpoints ready" level=info ts=2023-08-14T02:47:16.087435408Z caller=operator.go:481 component=prometheusoperator msg="connection established" cluster-version=v1.27.4+deb2c60 level=info ts=2023-08-14T02:47:16.087438928Z caller=operator.go:476 component=alertmanageroperator msg="connection established" cluster-version=v1.27.4+deb2c60 level=info ts=2023-08-14T02:47:16.087448608Z caller=operator.go:490 component=prometheusoperator msg="CRD API endpoints ready" level=info ts=2023-08-14T02:47:16.087456528Z caller=operator.go:485 component=alertmanageroperator msg="CRD API endpoints ready" level=info ts=2023-08-14T02:47:16.187822123Z caller=operator.go:274 component=thanosoperator msg="successfully synced all caches" level=info ts=2023-08-14T02:47:16.188243283Z caller=operator.go:413 component=prometheusoperator msg="successfully synced all caches" level=info ts=2023-08-14T02:47:16.201367603Z caller=operator.go:307 component=alertmanageroperator msg="successfully synced all caches" level=info ts=2023-08-14T02:47:18.442283267Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" level=info ts=2023-08-14T02:47:18.528044223Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" level=info ts=2023-08-14T02:47:18.646051018Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" level=info ts=2023-08-14T02:47:18.851978809Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" level=info ts=2023-08-14T02:47:18.913692487Z caller=operator.go:552 component=thanosoperator key=openshift-user-workload-monitoring/user-workload msg="sync thanos-ruler" level=info ts=2023-08-14T02:47:18.952845965Z caller=operator.go:659 component=thanosoperator msg="StatefulSet not found" key=openshift-user-workload-monitoring/thanos-ruler-user-workload level=info ts=2023-08-14T02:47:18.965643124Z caller=operator.go:552 component=thanosoperator key=openshift-user-workload-monitoring/user-workload msg="sync thanos-ruler" level=info ts=2023-08-14T02:47:18.966071804Z caller=operator.go:659 component=thanosoperator msg="StatefulSet not found" key=openshift-user-workload-monitoring/thanos-ruler-user-workload level=info ts=2023-08-14T02:47:19.016365442Z caller=operator.go:552 component=thanosoperator key=openshift-user-workload-monitoring/user-workload msg="sync thanos-ruler" level=error ts=2023-08-14T02:47:19.116527918Z caller=klog.go:116 component=k8s_client_runtime func=ErrorDepth msg="status \"openshift-user-workload-monitoring/user-workload\" failed: failed to update status subresource: Operation cannot be fulfilled on thanosrulers.monitoring.coreos.com \"user-workload\": the object has been modified; please apply your changes to the latest version and try again" level=info ts=2023-08-14T02:47:28.931009098Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" level=info ts=2023-08-14T02:47:28.953216857Z caller=operator.go:552 component=thanosoperator key=openshift-user-workload-monitoring/user-workload msg="sync thanos-ruler" level=info ts=2023-08-14T02:47:29.006615294Z caller=operator.go:552 component=thanosoperator key=openshift-user-workload-monitoring/user-workload msg="sync thanos-ruler" level=info ts=2023-08-14T02:47:29.017325574Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" level=info ts=2023-08-14T02:47:38.465331049Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" level=info ts=2023-08-14T02:47:38.556070685Z caller=operator.go:1189 component=prometheusoperator key=openshift-user-workload-monitoring/user-workload msg="sync prometheus" $ oc get thanosrulers --all-namespaces NAMESPACE NAME VERSION REPLICAS READY RECONCILED AVAILABLE AGE openshift-user-workload-monitoring user-workload 2 2 True True 4h12m
Version-Release number of selected component (if applicable):
4.14.0-0.nightly-2023-08-11-055332
How reproducible:
always