2025-06-03T13:42:56Z INFO setup starting manager 2025-06-03T13:42:56Z INFO controller-runtime.metrics Starting metrics server 2025-06-03T13:42:56Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8082", "secure": false} 2025-06-03T13:42:56Z INFO starting server {"name": "health probe", "addr": "[::]:8081"} 2025-06-03T13:42:56Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1alpha1.Perses"} 2025-06-03T13:42:56Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.Service"} 2025-06-03T13:42:56Z INFO Starting EventSource {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "source": "kind source: *v1alpha1.PersesDashboard"} 2025-06-03T13:42:56Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.StatefulSet"} 2025-06-03T13:42:56Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.ConfigMap"} 2025-06-03T13:42:56Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.Deployment"} 2025-06-03T13:42:56Z INFO Starting EventSource {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "source": "kind source: *v1alpha1.PersesDatasource"} 2025-06-03T13:42:57Z INFO Starting Controller {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard"} 2025-06-03T13:42:57Z INFO Starting workers {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "worker count": 1} 2025-06-03T13:42:57Z INFO Starting Controller {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource"} 2025-06-03T13:42:57Z INFO Starting workers {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "worker count": 1} 2025-06-03T13:42:57Z INFO Starting Controller {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses"} 2025-06-03T13:42:57Z INFO Starting workers {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "worker count": 1} time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "3f48bc72-cb39-4244-a103-4777b75688e6"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "3f48bc72-cb39-4244-a103-4777b75688e6", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "505c4a89-788b-40d0-ad4f-7363f1bd8092"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "505c4a89-788b-40d0-ad4f-7363f1bd8092", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "a5f1e40a-7b6c-409c-977d-5accb310161c"} time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "a5f1e40a-7b6c-409c-977d-5accb310161c", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "19e20e6a-1e6c-4fea-b04d-a4a503b975e6"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "19e20e6a-1e6c-4fea-b04d-a4a503b975e6", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "07f12a30-8863-480b-8db6-ec9d7007e713"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "07f12a30-8863-480b-8db6-ec9d7007e713", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "3db0411d-f8e0-4f98-a10e-7379e8b5a98c"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "3db0411d-f8e0-4f98-a10e-7379e8b5a98c", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "271af599-caf1-4b4c-b3ec-b98b172f715b"} time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "271af599-caf1-4b4c-b3ec-b98b172f715b", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "fd694c86-ffdd-4234-b752-ccb2e4b2519e"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "fd694c86-ffdd-4234-b752-ccb2e4b2519e", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "942a41da-5822-4771-b703-b9bbe8192c2f"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "942a41da-5822-4771-b703-b9bbe8192c2f", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "3978f243-b203-4a45-80f3-e2a3460f7ded"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "3978f243-b203-4a45-80f3-e2a3460f7ded", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "7cdd42c4-ea9e-4624-a233-b77a256f5ea1"} time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "7cdd42c4-ea9e-4624-a233-b77a256f5ea1", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "dcaeadee-e4b2-48b1-aa8a-8448d4d380a3"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "dcaeadee-e4b2-48b1-aa8a-8448d4d380a3", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "f6b2b75f-f306-42cc-ad6c-e7f9147844c4"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "f6b2b75f-f306-42cc-ad6c-e7f9147844c4", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "c411e9db-2941-424b-96a4-a16863b3fc55"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "c411e9db-2941-424b-96a4-a16863b3fc55", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "56b6beb2-ff34-46df-8fe4-b2430966d448"} time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "56b6beb2-ff34-46df-8fe4-b2430966d448", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "17b721ff-5f7d-445d-9813-c6618542b16a"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "17b721ff-5f7d-445d-9813-c6618542b16a", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "5ddbc13d-1c03-48ca-8431-6f8a6ca7f70c"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "5ddbc13d-1c03-48ca-8431-6f8a6ca7f70c", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "503fcf23-d5a7-4b49-ad2a-1c9d9edf2245"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "503fcf23-d5a7-4b49-ad2a-1c9d9edf2245", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "5aca5b91-012e-4d06-a276-8a216439601b"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "5aca5b91-012e-4d06-a276-8a216439601b", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "0c737cac-5c7c-4d8a-a428-34dc637704fd"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "0c737cac-5c7c-4d8a-a428-34dc637704fd", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "a1a989ad-cda1-4f08-8508-67c70766a7a9"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "a1a989ad-cda1-4f08-8508-67c70766a7a9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "8304e71d-0a38-4713-9a79-5f8b80eaa4be"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "8304e71d-0a38-4713-9a79-5f8b80eaa4be", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "7f541a31-b6ad-40a0-a2bb-2fc4dfaa45d3"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "7f541a31-b6ad-40a0-a2bb-2fc4dfaa45d3", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "61c09ce0-69cc-4719-8a70-58de501a36d9"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "61c09ce0-69cc-4719-8a70-58de501a36d9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "a8e86e43-ca48-457b-9ef0-bd213e6b1834"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "a8e86e43-ca48-457b-9ef0-bd213e6b1834", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "978da03c-ec33-4dfe-80dd-17aff855ecf3"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "978da03c-ec33-4dfe-80dd-17aff855ecf3", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "aee63997-883b-41a8-b0fc-09e1a6616674"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "aee63997-883b-41a8-b0fc-09e1a6616674", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "8e239a22-d4b3-46f4-8519-4a3d05a6855a"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "8e239a22-d4b3-46f4-8519-4a3d05a6855a", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "95f9c234-0036-4277-ba82-2593cb6ed59e"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "95f9c234-0036-4277-ba82-2593cb6ed59e", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "be3a8043-fe72-41ff-8352-677f99cd82f9"} time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "be3a8043-fe72-41ff-8352-677f99cd82f9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "1bf8d5e2-c765-4ad7-a088-bcbfa84bedc2"} time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "1bf8d5e2-c765-4ad7-a088-bcbfa84bedc2", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "32d0cb49-17ee-4056-8dd4-e2abdab2abd9"} time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "32d0cb49-17ee-4056-8dd4-e2abdab2abd9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "431331d4-00db-474f-a6d4-bdeacca234cb"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "431331d4-00db-474f-a6d4-bdeacca234cb", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "6a449357-d75b-461a-be36-c4024c0303a9"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "6a449357-d75b-461a-be36-c4024c0303a9", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "25d9f8c7-67ad-408f-b6e1-5cf5d63cfd0b"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "25d9f8c7-67ad-408f-b6e1-5cf5d63cfd0b", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "081224be-2a1e-4fd5-a8f7-d8dc61043622"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "081224be-2a1e-4fd5-a8f7-d8dc61043622", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "3dfecd14-be28-4598-a34e-bcfd84a5f37d"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "3dfecd14-be28-4598-a34e-bcfd84a5f37d", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:57Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "10266125-424c-4e0d-b095-c5d26f5dfd95"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "10266125-424c-4e0d-b095-c5d26f5dfd95", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:57Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "4503832c-ff1b-4387-845b-437b450af45f"} 2025-06-03T13:42:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "4503832c-ff1b-4387-845b-437b450af45f", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:58Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:58Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "1756e209-a71f-4062-ab90-a0dce1736e3b"} 2025-06-03T13:42:58Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "1756e209-a71f-4062-ab90-a0dce1736e3b", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:58Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:58Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:58Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "56630d4d-49a1-493d-88e6-579a768592cf"} 2025-06-03T13:42:58Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "56630d4d-49a1-493d-88e6-579a768592cf", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:58Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:58Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "72248ac6-1d09-46f7-bbfb-a65ae017757b"} 2025-06-03T13:42:58Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "72248ac6-1d09-46f7-bbfb-a65ae017757b", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:58Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:58Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "7303697e-db21-4163-9970-f256fa8d6836"} 2025-06-03T13:42:58Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "7303697e-db21-4163-9970-f256fa8d6836", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:58Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "b270b68b-31ca-4019-bb81-42ab4fd5a1fb"} 2025-06-03T13:42:58Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "b270b68b-31ca-4019-bb81-42ab4fd5a1fb", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:58Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:42:58Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:42:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e012ad6b-85f3-45e5-a32d-54a505835fef"} 2025-06-03T13:42:58Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e012ad6b-85f3-45e5-a32d-54a505835fef", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:59Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:59Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:59Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "4f943819-d206-4b44-9cda-a4e6c09ead68"} 2025-06-03T13:42:59Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "4f943819-d206-4b44-9cda-a4e6c09ead68", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:59Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:42:59Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "422d141e-fb9e-4ac7-a645-38090916467a"} 2025-06-03T13:42:59Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "422d141e-fb9e-4ac7-a645-38090916467a", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:42:59Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:42:59Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:42:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "0352f473-2288-476d-9f85-7e6cc1dc2b33"} time="2025-06-03T13:42:59Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:42:59Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "0352f473-2288-476d-9f85-7e6cc1dc2b33", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:42:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "662c1bba-d680-4db4-bcc7-3fa92a2505c9"} 2025-06-03T13:42:59Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "662c1bba-d680-4db4-bcc7-3fa92a2505c9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:00Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:43:00Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:43:00Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "ab0ffa78-f095-426f-b7ce-8633d0a8b2b6"} 2025-06-03T13:43:00Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "ab0ffa78-f095-426f-b7ce-8633d0a8b2b6", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:02Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:02Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:02Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "7f8b6c77-5542-4025-83d7-1d2b4dd231ac"} 2025-06-03T13:43:02Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "7f8b6c77-5542-4025-83d7-1d2b4dd231ac", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:02Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:02Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "6925c2a8-8874-4a83-853d-d1c7adc1dfbb"} 2025-06-03T13:43:02Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "6925c2a8-8874-4a83-853d-d1c7adc1dfbb", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:02Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:02Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "9dec6e7b-155b-4c04-a860-eb6700868bd0"} 2025-06-03T13:43:02Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "9dec6e7b-155b-4c04-a860-eb6700868bd0", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:02Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "c3fc15f1-fbbb-4dcd-90e5-3deca468bf66"} 2025-06-03T13:43:02Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "c3fc15f1-fbbb-4dcd-90e5-3deca468bf66", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:02Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:43:02Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:43:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "7506b228-d1ef-452c-9ec2-0db527c38ac6"} 2025-06-03T13:43:02Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "7506b228-d1ef-452c-9ec2-0db527c38ac6", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:07Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:07Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:07Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "78b3203f-c4bc-4af4-885c-ca23c8807534"} 2025-06-03T13:43:07Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "78b3203f-c4bc-4af4-885c-ca23c8807534", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:07Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "6ff859dc-a3d0-4bbc-93ff-5a0dcfec2073"} 2025-06-03T13:43:07Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "6ff859dc-a3d0-4bbc-93ff-5a0dcfec2073", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:07Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:07Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:07Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "48958573-e4fb-4ad8-b430-2589a5f8882c"} time="2025-06-03T13:43:07Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "0936819a-ae98-4510-b966-da19b4b6fda2"} 2025-06-03T13:43:07Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "48958573-e4fb-4ad8-b430-2589a5f8882c", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T13:43:07Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "0936819a-ae98-4510-b966-da19b4b6fda2", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:07Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:43:07Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:43:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "84ff8f9d-58f8-4751-9355-ccc042d66862"} 2025-06-03T13:43:07Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "84ff8f9d-58f8-4751-9355-ccc042d66862", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:17Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:17Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:17Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:17Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "9bf51833-0a95-4fda-b26a-7ca5afca279b"} 2025-06-03T13:43:17Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "9bf51833-0a95-4fda-b26a-7ca5afca279b", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:17Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:17Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:17Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "c65ada59-3b8c-4e06-a9d7-ad4dd86ab349"} 2025-06-03T13:43:17Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "c65ada59-3b8c-4e06-a9d7-ad4dd86ab349", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:17Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:17Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:17Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "3e88cc5d-1542-482e-a4fd-84c47eadc713"} 2025-06-03T13:43:17Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "3e88cc5d-1542-482e-a4fd-84c47eadc713", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:17Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:17Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ebae8bf2-589c-4497-88d4-8fd23d04bbf0"} 2025-06-03T13:43:17Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ebae8bf2-589c-4497-88d4-8fd23d04bbf0", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:18Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:43:18Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:43:18Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "91075e3b-591e-4824-bb7d-e07cda829b9e"} 2025-06-03T13:43:18Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "91075e3b-591e-4824-bb7d-e07cda829b9e", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:38Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:38Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:38Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:38Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "26811237-0ce9-4eb5-9587-68cd8af9eb0e"} 2025-06-03T13:43:38Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "26811237-0ce9-4eb5-9587-68cd8af9eb0e", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:38Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:38Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "2769f44a-117a-4604-a14d-99aaf2603954"} 2025-06-03T13:43:38Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "2769f44a-117a-4604-a14d-99aaf2603954", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:38Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:43:38Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:43:38Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:43:38Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "1dc86931-dfc0-454d-8e78-ca1cc5bc90c9"} 2025-06-03T13:43:38Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "1dc86931-dfc0-454d-8e78-ca1cc5bc90c9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:38Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:43:38Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "93e20471-6994-47c8-9137-e6e537751721"} 2025-06-03T13:43:38Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "93e20471-6994-47c8-9137-e6e537751721", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:43:38Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:43:38Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:43:38Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "4045d5b8-2124-4c6c-a8cb-4be7486d8a5b"} 2025-06-03T13:43:38Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "4045d5b8-2124-4c6c-a8cb-4be7486d8a5b", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:44:19Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:44:19Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:44:19Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:44:19Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "62bae3c9-3b68-4aca-8ac8-2bb36dc29866"} 2025-06-03T13:44:19Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "62bae3c9-3b68-4aca-8ac8-2bb36dc29866", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:44:19Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:44:19Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "9c9a8770-9c94-47d8-bb39-6d072df59ec3"} 2025-06-03T13:44:19Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "9c9a8770-9c94-47d8-bb39-6d072df59ec3", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:44:19Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:44:19Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:44:19Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:44:19Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "5a703f27-025f-41a9-87e7-ff2cdd8d0bf9"} 2025-06-03T13:44:19Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "5a703f27-025f-41a9-87e7-ff2cdd8d0bf9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:44:19Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:44:19Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "88bc6a93-405d-4f79-9cb9-61c5a59d229d"} 2025-06-03T13:44:19Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "88bc6a93-405d-4f79-9cb9-61c5a59d229d", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:44:19Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:44:19Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:44:19Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e801d31e-6faf-4cd2-9c83-888bf64dc2c8"} 2025-06-03T13:44:19Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e801d31e-6faf-4cd2-9c83-888bf64dc2c8", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:45:41Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:45:41Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:45:41Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:45:41Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "1a633326-7698-47ee-933d-eb87ca5a615e"} 2025-06-03T13:45:41Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "1a633326-7698-47ee-933d-eb87ca5a615e", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:45:41Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:45:41Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "f4ef5daf-5729-4754-a963-28caf2a50548"} time="2025-06-03T13:45:41Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller 2025-06-03T13:45:41Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "f4ef5daf-5729-4754-a963-28caf2a50548", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:45:41Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:45:41Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:45:41Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "70a006d6-9798-4341-bc51-33a62018be43"} 2025-06-03T13:45:41Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "70a006d6-9798-4341-bc51-33a62018be43", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:45:41Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:45:41Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "b41954a8-f29a-4a79-a1a0-74e9b43223f6"} 2025-06-03T13:45:41Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "b41954a8-f29a-4a79-a1a0-74e9b43223f6", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:45:41Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:45:41Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:45:41Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "43c563c9-0df2-48fa-81fe-2d2b5a85153f"} 2025-06-03T13:45:41Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "43c563c9-0df2-48fa-81fe-2d2b5a85153f", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:24Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:48:24Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:48:24Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:48:24Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "8a98cdc1-cfa4-402a-9989-951b71ed33c8"} 2025-06-03T13:48:24Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "8a98cdc1-cfa4-402a-9989-951b71ed33c8", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:24Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:48:24Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:48:24Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "c4c42a38-e06b-4103-b865-41f17b5c9bc8"} 2025-06-03T13:48:24Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "c4c42a38-e06b-4103-b865-41f17b5c9bc8", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:24Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:48:24Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:48:24Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ff1759ec-9e12-4c58-8040-abd64cbb5bd1"} 2025-06-03T13:48:24Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ff1759ec-9e12-4c58-8040-abd64cbb5bd1", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:24Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:48:24Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "5b1574c4-6f29-4196-b625-dd1bfffa3221"} 2025-06-03T13:48:24Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "5b1574c4-6f29-4196-b625-dd1bfffa3221", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:25Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:48:25Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T13:48:25Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "57030a2f-be59-4c64-88c8-054cbea12e11"} 2025-06-03T13:48:25Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "57030a2f-be59-4c64-88c8-054cbea12e11", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:29Z" level=info msg="Performing Finalizer Operations for Perses before delete CR" module=perses_controller time="2025-06-03T13:48:29Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:48:29Z" level=info msg="perses datasource resource not found. Deleting 'accelerators-thanos-querier-datasource' in 'openshift-cluster-observability-operator'" module=perses_datasource_controller time="2025-06-03T13:48:29Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=datasource_controller 2025-06-03T13:48:29Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "609e32ff-25f2-478a-b536-0e2485cc7d2a"} 2025-06-03T13:48:29Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "609e32ff-25f2-478a-b536-0e2485cc7d2a", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:29Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:48:29Z" level=info msg="perses dashboard resource not found. Deleting 'accelerators-dashboard' in 'openshift-cluster-observability-operator'" module=perses_dashboards_controller time="2025-06-03T13:48:29Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused" module=dashboard_controller 2025-06-03T13:48:29Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "dfc1bf90-15f7-40f1-8a63-06b82ea47a8f"} 2025-06-03T13:48:29Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "dfc1bf90-15f7-40f1-8a63-06b82ea47a8f", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.71.79:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:48:29Z" level=info msg="Removing Finalizer for Perses after successfully perform the operations" module=perses_controller time="2025-06-03T13:48:29Z" level=info msg="perses resource not found. Ignoring since object must be deleted" module=perses_controller time="2025-06-03T13:48:29Z" level=info msg="perses resource not found. Ignoring since object must be deleted" module=perses_controller time="2025-06-03T13:48:29Z" level=info msg="perses resource not found. Ignoring since object must be deleted" module=perses_controller time="2025-06-03T13:48:29Z" level=info msg="perses resource not found. Ignoring since object must be deleted" module=perses_controller time="2025-06-03T13:49:50Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:49:50Z" level=info msg="Adding Finalizer for Perses" module=perses_controller time="2025-06-03T13:49:50Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host" module=datasource_controller 2025-06-03T13:49:50Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "63fc133b-4410-41e2-af98-47b0931a81b6"} 2025-06-03T13:49:50Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "63fc133b-4410-41e2-af98-47b0931a81b6", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:49:50Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:49:50Z" level=error msg="Failed to update custom resource to add finalizer" error="Operation cannot be fulfilled on perses.perses.dev \"perses\": the object has been modified; please apply your changes to the latest version and try again" module=perses_controller 2025-06-03T13:49:50Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "00ce009e-8429-4581-af4e-584a2fd546e9"} 2025-06-03T13:49:50Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "00ce009e-8429-4581-af4e-584a2fd546e9", "error": "Operation cannot be fulfilled on perses.perses.dev \"perses\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:49:50Z" level=info msg="Adding Finalizer for Perses" module=perses_controller time="2025-06-03T13:49:50Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host" module=datasource_controller 2025-06-03T13:49:50Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "36626145-cd84-4b04-b571-1d96479c3113"} 2025-06-03T13:49:50Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "36626145-cd84-4b04-b571-1d96479c3113", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:49:50Z" level=info msg="Creating a new Service: Service.Namespace openshift-cluster-observability-operator Service.Name perses" module=service_controller time="2025-06-03T13:49:50Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:49:50Z" level=info msg="Creating a new ConfigMap: ConfigMap.Namespace openshift-cluster-observability-operator ConfigMap.Name perses-config" module=configmap_controller time="2025-06-03T13:49:50Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host" module=dashboard_controller 2025-06-03T13:49:50Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "187ddc70-e70f-4365-9292-7a7933a348c1"} 2025-06-03T13:49:50Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "187ddc70-e70f-4365-9292-7a7933a348c1", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:49:50Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:49:50Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:49:50Z" level=info msg="Creating a new StatefulSet: StatefulSet.Namespace openshift-cluster-observability-operator StatefulSet.Name perses" module=statefulset_controller time="2025-06-03T13:49:50Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host" module=dashboard_controller 2025-06-03T13:49:50Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "3b103805-43d0-44e9-b3d6-94252e359c28"} 2025-06-03T13:49:50Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "3b103805-43d0-44e9-b3d6-94252e359c28", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp: lookup perses.openshift-cluster-observability-operator.svc.cluster.local on 172.30.0.10:53: no such host"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:49:50Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:49:50Z" level=error msg="Operation cannot be fulfilled on statefulsets.apps \"perses\": the object has been modified; please apply your changes to the latest version and try againFailed to update StatefulSet" module=statefulset_controller 2025-06-03T13:49:50Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e6beb7e3-34bd-4430-8e40-94d3975596c2"} 2025-06-03T13:49:50Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e6beb7e3-34bd-4430-8e40-94d3975596c2", "error": "Operation cannot be fulfilled on statefulsets.apps \"perses\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T13:49:51Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:49:51Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:49:51Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:49:51Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:50:01Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T13:53:52Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T13:53:52Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Project created: openshift-cluster-observability-operator" module=datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Secret created: accelerators-thanos-querier-datasource-secret" module=datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Datasource created: accelerators-thanos-querier-datasource" module=datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Secret updated: thanos-querier-datasource-secret" module=datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Datasource updated: thanos-querier-datasource" module=datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Dashboard updated: openshift-cluster-sample-dashboard" module=dashboard_controller time="2025-06-03T13:53:52Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Secret updated: accelerators-thanos-querier-datasource-secret" module=datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Datasource updated: accelerators-thanos-querier-datasource" module=datasource_controller time="2025-06-03T13:53:52Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:53:52Z" level=info msg="Dashboard created: accelerators-dashboard" module=dashboard_controller time="2025-06-03T13:53:52Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T13:53:52Z" level=info msg="Dashboard updated: accelerators-dashboard" module=dashboard_controller