2025-10-31T16:17:50Z	INFO	setup	running with arguments	{"namespace": "openshift-cluster-observability-operator", "metrics-bind-address": ":8080", "images": "health-analyzer=registry.redhat.io/cluster-observability-operator/cluster-health-analyzer-rhel9@sha256:7a96387d5c3eb672346958133e462513eca8fcbaf3c5278411eb02d7987c1b06,korrel8r=registry.redhat.io/cluster-observability-operator/korrel8r-rhel9@sha256:2919dd79eaef284bb94586f93d2d925d872f21cacf7d6bb5a0485ed5813e7c57,perses=registry.redhat.io/cluster-observability-operator/perses-rhel9@sha256:1a374584c0b1f984383d4879ba862c66fd4635a07e59ec7d5c08fc85f59de6c3,ui-dashboards=registry.redhat.io/cluster-observability-operator/dashboards-console-plugin-rhel9@sha256:1006738cfc0da3ea2274e013e3da1fbb4ecbc0c9d4ff8d6c0c326b13be4586e3,ui-distributed-tracing-pf4=registry.redhat.io/cluster-observability-operator/distributed-tracing-console-plugin-pf4-rhel9@sha256:cf16d885f60f7b3fdfc4491e37badba65fed4c012c4cf124af54acfb9f84fe40,ui-distributed-tracing-pf5=registry.redhat.io/cluster-observability-operator/distributed-tracing-console-plugin-pf5-rhel9@sha256:933eabaeda4d8014cc225eb529d1df6fdb7fcc68e1c516f7fb470c9677d49821,ui-distributed-tracing=registry.redhat.io/cluster-observability-operator/distributed-tracing-console-plugin-rhel9@sha256:de2cfe24fdc6a61df39ef0db49963ba9f02ddfb60e4d3149c09900cb1a8b95fa,ui-logging-pf4=registry.redhat.io/cluster-observability-operator/logging-console-plugin-pf4-rhel9@sha256:64c2d962de0906dda9dd8b4bce25d0b862f8c7f3369e9b591ea8b9f611e255d2,ui-logging=registry.redhat.io/cluster-observability-operator/logging-console-plugin-rhel9@sha256:176762d743ce219e4bf7865c9cd69ff8ac79ee3443b678ea2de5fec964730962,ui-monitoring-pf5=registry.redhat.io/cluster-observability-operator/monitoring-console-plugin-pf5-rhel9@sha256:a4d5e06ac5afa1b08be1d8a47f7ad7663af74b2d33665ae826d473a2adf233a3,ui-monitoring=registry.redhat.io/cluster-observability-operator/monitoring-console-plugin-rhel9@sha256:2fa20795e23e3d7b11a9eef29a614f42721d4442dbe9f9ad6fc81e3f39fd31b1,ui-troubleshooting-panel=registry.redhat.io/cluster-observability-operator/troubleshooting-panel-console-plugin-rhel9@sha256:3ab1bccd19839eaddab1552efb8c03a97553b8b928178e907468b8b1f33faba7", "openshift.enabled": true}
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"serving-cert::/etc/tls/private/tls.crt::/etc/tls/private/tls.key", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded serving cert ["serving-cert::/etc/tls/private/tls.crt::/etc/tls/private/tls.key"]: "observability-operator.openshift-cluster-observability-operator.svc" [serving] validServingFor=[observability-operator.openshift-cluster-observability-operator.svc,observability-operator.openshift-cluster-observability-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1761921036" (2025-10-31 16:17:44 +0000 UTC to 2027-10-31 16:17:45 +0000 UTC (now=2025-10-31 16:17:50.634822127 +0000 UTC))
2025-10-31T16:17:50Z	INFO	setup	starting manager
2025-10-31T16:17:50Z	INFO	controller-runtime.metrics	Starting metrics server
I1031 16:17:50.659412       1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
2025-10-31T16:17:50Z	INFO	starting server	{"name": "pprof", "addr": "127.0.0.1:8083"}
I1031 16:17:50.659429       1 tlsconfig.go:243] "Starting DynamicServingCertificateController"
I1031 16:17:50.659789       1 shared_informer.go:349] "Waiting for caches to sync" controller="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
2025-10-31T16:17:50Z	INFO	starting server	{"name": "health probe", "addr": "[::]:8081"}
I1031 16:17:50.760510       1 shared_informer.go:356] "Caches are synced" controller="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.PodDisruptionBudget"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1alpha1.MonitoringStack"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1.ConsolePlugin"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.Prometheus"}
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"client-ca::kube-system::extension-apiserver-authentication::client-ca-file", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file"]: "admin-kubeconfig-signer" [] issuer="<self>" (2025-10-31 14:14:44 +0000 UTC to 2035-10-29 14:14:44 +0000 UTC (now=2025-10-31 16:17:50.760900639 +0000 UTC))
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"client-ca::kube-system::extension-apiserver-authentication::client-ca-file", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded client CA [1/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file"]: "kubelet-signer" [] issuer="<self>" (2025-10-31 14:14:48 +0000 UTC to 2025-11-01 14:14:48 +0000 UTC (now=2025-10-31 16:17:50.760941294 +0000 UTC))
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"client-ca::kube-system::extension-apiserver-authentication::client-ca-file", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded client CA [2/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file"]: "kube-control-plane-signer" [] issuer="<self>" (2025-10-31 14:14:48 +0000 UTC to 2026-10-31 14:14:48 +0000 UTC (now=2025-10-31 16:17:50.760956268 +0000 UTC))
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"client-ca::kube-system::extension-apiserver-authentication::client-ca-file", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded client CA [3/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file"]: "kube-apiserver-to-kubelet-signer" [] issuer="<self>" (2025-10-31 14:14:48 +0000 UTC to 2026-10-31 14:14:48 +0000 UTC (now=2025-10-31 16:17:50.760969783 +0000 UTC))
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"client-ca::kube-system::extension-apiserver-authentication::client-ca-file", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded client CA [4/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file"]: "kubelet-bootstrap-kubeconfig-signer" [] issuer="<self>" (2025-10-31 14:14:46 +0000 UTC to 2035-10-29 14:14:46 +0000 UTC (now=2025-10-31 16:17:50.760982799 +0000 UTC))
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"client-ca::kube-system::extension-apiserver-authentication::client-ca-file", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded client CA [5/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file"]: "kube-csr-signer_@1761921037" [] issuer="kubelet-signer" (2025-10-31 14:30:36 +0000 UTC to 2025-11-01 14:14:48 +0000 UTC (now=2025-10-31 16:17:50.761000193 +0000 UTC))
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"client-ca::kube-system::extension-apiserver-authentication::client-ca-file", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded client CA [6/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file"]: "openshift-kube-apiserver-operator_node-system-admin-signer@1761921035" [] issuer="<self>" (2025-10-31 14:30:35 +0000 UTC to 2028-10-30 14:30:36 +0000 UTC (now=2025-10-31 16:17:50.761016118 +0000 UTC))
2025-10-31T16:17:50Z	INFO	events	Event(v1.ObjectReference{Kind:"", Namespace:"", Name:"serving-cert::/etc/tls/private/tls.crt::/etc/tls/private/tls.key", UID:"", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'TLSConfigChanged' loaded serving cert ["serving-cert::/etc/tls/private/tls.crt::/etc/tls/private/tls.key"]: "observability-operator.openshift-cluster-observability-operator.svc" [serving] validServingFor=[observability-operator.openshift-cluster-observability-operator.svc,observability-operator.openshift-cluster-observability-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1761921036" (2025-10-31 16:17:44 +0000 UTC to 2027-10-31 16:17:45 +0000 UTC (now=2025-10-31 16:17:50.76126921 +0000 UTC))
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.ServiceAccount"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.RoleBinding"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.ServiceMonitor"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.Alertmanager"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.Service"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "source": "kind source: *v1.Role"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1.Role"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "source": "kind source: *v1.ClusterRoleBinding"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1alpha1.UIPlugin"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1.Deployment"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1.Service"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1.ServiceAccount"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "observability-operator", "controllerGroup": "", "controllerKind": "Service", "source": "kind source: *v1.ServiceMonitor"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "source": "kind source: *v1alpha1.ObservabilityInstaller"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "source": "kind source: *v1alpha1.Subscription"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "source": "kind source: *v1.Secret"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "source": "kind source: *v1.Namespace"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "source": "kind source: *v1.Secret"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "source": "kind source: *v1alpha1.UIPlugin"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1.RoleBinding"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1alpha1.PersesDashboard"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "source": "kind source: *v1alpha1.PersesDatasource"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "source": "kind source: *v1alpha1.ThanosQuerier"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "source": "kind source: *v1.Deployment"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "source": "kind source: *v1.ServiceAccount"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "source": "kind source: *v1.Service"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "source": "kind source: *v1.ConfigMap"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "source": "kind source: *v1alpha1.MonitoringStack"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "source": "kind source: *v1.ClusterRole"}
2025-10-31T16:17:50Z	INFO	Starting EventSource	{"controller": "observability-operator", "controllerGroup": "", "controllerKind": "Service", "source": "kind source: *v1.Service"}
2025-10-31T16:17:51Z	INFO	controller-runtime.metrics	Serving metrics server	{"bindAddress": ":8080", "secure": true}
2025-10-31T16:17:52Z	INFO	Starting Controller	{"controller": "observability-operator", "controllerGroup": "", "controllerKind": "Service"}
2025-10-31T16:17:52Z	INFO	Starting workers	{"controller": "observability-operator", "controllerGroup": "", "controllerKind": "Service", "worker count": 1}
2025-10-31T16:17:52Z	INFO	Starting Controller	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack"}
2025-10-31T16:17:52Z	INFO	Starting workers	{"controller": "monitoringstack", "controllerGroup": "monitoring.rhobs", "controllerKind": "MonitoringStack", "worker count": 1}
2025-10-31T16:17:52Z	INFO	Starting Controller	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller"}
2025-10-31T16:17:52Z	INFO	Starting workers	{"controller": "cluster-observability", "controllerGroup": "observability.openshift.io", "controllerKind": "ObservabilityInstaller", "worker count": 1}
2025-10-31T16:17:52Z	INFO	Starting Controller	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin"}
2025-10-31T16:17:52Z	INFO	Starting workers	{"controller": "uiplugin", "controllerGroup": "observability.openshift.io", "controllerKind": "UIPlugin", "worker count": 1}
2025-10-31T16:17:52Z	INFO	Starting Controller	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier"}
2025-10-31T16:17:52Z	INFO	Starting workers	{"controller": "thanosquerier", "controllerGroup": "monitoring.rhobs", "controllerKind": "ThanosQuerier", "worker count": 1}
2025-10-31T16:17:52Z	INFO	observability-operator	Reconciling operator resources	{"operator": {"name":"observability-operator","namespace":"openshift-cluster-observability-operator"}}
2025-10-31T16:17:52Z	INFO	observability-operator	Reconciling operator resources	{"operator": {"name":"observability-operator","namespace":"openshift-cluster-observability-operator"}}
2025-10-31T16:17:58Z	INFO	observability-operator	Reconciling operator resources	{"operator": {"name":"observability-operator","namespace":"openshift-cluster-observability-operator"}}
2025-10-31T16:17:58Z	INFO	observability-operator	Reconciling operator resources	{"operator": {"name":"observability-operator","namespace":"openshift-cluster-observability-operator"}}
2025-10-31T16:20:57Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}
2025-10-31T16:20:57Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}
2025-10-31T16:20:58Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}
2025-10-31T16:20:58Z	INFO	observability-ui	Failed to update status	{"plugin": {"name":"monitoring"}, "err": "Operation cannot be fulfilled on uiplugins.observability.openshift.io \"monitoring\": the object has been modified; please apply your changes to the latest version and try again"}
2025-10-31T16:20:58Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}
2025-10-31T16:20:59Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}
2025-10-31T16:21:00Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}
2025-10-31T16:21:00Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}
2025-10-31T16:21:01Z	INFO	observability-ui	Reconciling observability UI plugin	{"plugin": {"name":"monitoring"}}