2025-06-03T11:10:53Z INFO setup starting manager 2025-06-03T11:10:53Z INFO controller-runtime.metrics Starting metrics server 2025-06-03T11:10:53Z INFO starting server {"name": "health probe", "addr": "[::]:8081"} 2025-06-03T11:10:53Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8082", "secure": false} 2025-06-03T11:10:53Z INFO Starting EventSource {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "source": "kind source: *v1alpha1.PersesDashboard"} 2025-06-03T11:10:53Z INFO Starting EventSource {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "source": "kind source: *v1alpha1.PersesDatasource"} 2025-06-03T11:10:53Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.Service"} 2025-06-03T11:10:53Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.StatefulSet"} 2025-06-03T11:10:53Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.ConfigMap"} 2025-06-03T11:10:53Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1alpha1.Perses"} 2025-06-03T11:10:53Z INFO Starting EventSource {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "source": "kind source: *v1.Deployment"} 2025-06-03T11:10:54Z INFO Starting Controller {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource"} 2025-06-03T11:10:54Z INFO Starting workers {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "worker count": 1} 2025-06-03T11:10:54Z INFO Starting Controller {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard"} 2025-06-03T11:10:54Z INFO Starting workers {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "worker count": 1} time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "713010a5-e207-44c8-ab9e-2c28ab4cde06"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "713010a5-e207-44c8-ab9e-2c28ab4cde06", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T11:10:54Z INFO Starting Controller {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses"} 2025-06-03T11:10:54Z INFO Starting workers {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "worker count": 1} time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "88f5bd94-4dd4-4f8b-8280-d297fbd65e6a"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "88f5bd94-4dd4-4f8b-8280-d297fbd65e6a", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ebadcd21-869d-4e75-96b9-0a125364c552"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ebadcd21-869d-4e75-96b9-0a125364c552", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "6ef4579d-a2ad-4b03-befc-c29a754ff822"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "6ef4579d-a2ad-4b03-befc-c29a754ff822", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "c4cf5a82-47fe-4ef5-ba23-eb17e6dd1c0c"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "c4cf5a82-47fe-4ef5-ba23-eb17e6dd1c0c", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "10e6bbca-fa59-4aba-b89f-85c0f71f42ab"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "10e6bbca-fa59-4aba-b89f-85c0f71f42ab", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "7c60b7d5-72f5-4df0-aac8-69acb1f6cbdc"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "7c60b7d5-72f5-4df0-aac8-69acb1f6cbdc", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "4cef83b7-db09-43e8-b990-8f20a178c4e5"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "4cef83b7-db09-43e8-b990-8f20a178c4e5", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "19268bd8-965a-43c2-bdf2-ad97540884a7"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "19268bd8-965a-43c2-bdf2-ad97540884a7", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "ebcc7d50-8854-427a-9ecb-ae093a9fe795"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "ebcc7d50-8854-427a-9ecb-ae093a9fe795", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ad5c0b02-a44d-42fe-b96c-6a3468bebef1"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "ad5c0b02-a44d-42fe-b96c-6a3468bebef1", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "88045664-5f9b-4851-bd17-eb1a5170340c"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "88045664-5f9b-4851-bd17-eb1a5170340c", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "eb338138-c361-4f9f-a568-29f00292353a"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "eb338138-c361-4f9f-a568-29f00292353a", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "889f56fc-8cbf-4f1a-a507-8ddd3dd2cc69"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "889f56fc-8cbf-4f1a-a507-8ddd3dd2cc69", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "50c1b02a-5802-42a7-98c1-7d65b8188f59"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "50c1b02a-5802-42a7-98c1-7d65b8188f59", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:54Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:54Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "b98c7589-946b-4828-9d45-07ab99f1d602"} 2025-06-03T11:10:54Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "b98c7589-946b-4828-9d45-07ab99f1d602", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:54Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:55Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:55Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:55Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "7903946c-e5a0-4913-8dba-caafe6575af4"} 2025-06-03T11:10:55Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "7903946c-e5a0-4913-8dba-caafe6575af4", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:55Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:55Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:55Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "0eb27999-9851-49de-a7c6-9454309612a3"} 2025-06-03T11:10:55Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "0eb27999-9851-49de-a7c6-9454309612a3", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:56Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=datasource_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "4ec6854d-d36d-4afe-9db1-49ebba1bd70c"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "4ec6854d-d36d-4afe-9db1-49ebba1bd70c", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:56Z" level=error msg="Failed to create perses rest client" error="unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory" module=dashboard_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "cdcc7edc-9b86-4823-94ab-b50d70532a10"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "cdcc7edc-9b86-4823-94ab-b50d70532a10", "error": "unable to read CA cert: unable to read file /ca/ca.crt: open /ca/ca.crt: no such file or directory"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "e7454dba-c790-4f5d-8d9d-3eadb08a13f4"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "e7454dba-c790-4f5d-8d9d-3eadb08a13f4", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "878580ca-ebb0-4fcc-8371-8b4a5fbc7f5e"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "878580ca-ebb0-4fcc-8371-8b4a5fbc7f5e", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "663d168a-f34a-4cc4-917a-93557e78462f"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "663d168a-f34a-4cc4-917a-93557e78462f", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "fe99514a-5c95-4857-aaa3-98f1181363f9"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "fe99514a-5c95-4857-aaa3-98f1181363f9", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "2ba5fd1c-adc0-4aa2-8c4f-ec23c1e376de"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "2ba5fd1c-adc0-4aa2-8c4f-ec23c1e376de", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "ca65399c-0465-407d-b067-0299c25ba05e"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "ca65399c-0465-407d-b067-0299c25ba05e", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "544a0834-6bf8-4245-ad1c-601debb3747a"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "544a0834-6bf8-4245-ad1c-601debb3747a", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "60ffe371-5d7d-4e75-a26f-0cd41c7909f3"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "60ffe371-5d7d-4e75-a26f-0cd41c7909f3", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "f5c5efc1-dbfb-4c15-824d-07da05cf6e09"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "f5c5efc1-dbfb-4c15-824d-07da05cf6e09", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:56Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:56Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:56Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "a54fab3c-c267-44f5-a181-f44d96433927"} 2025-06-03T11:10:56Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "a54fab3c-c267-44f5-a181-f44d96433927", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "876199da-4a21-4789-82d3-a90dad5bf339"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "876199da-4a21-4789-82d3-a90dad5bf339", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "7a3cc794-ef8b-42e2-b102-5318b25c2079"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "7a3cc794-ef8b-42e2-b102-5318b25c2079", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "4e5979f5-ed40-4d4e-a7f3-aee71d01d033"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "4e5979f5-ed40-4d4e-a7f3-aee71d01d033", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "ab1df372-eb86-473e-99af-8ea7fc9316ff"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "ab1df372-eb86-473e-99af-8ea7fc9316ff", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "8ee96e39-e6f7-479c-a10a-8329afbe36ea"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "8ee96e39-e6f7-479c-a10a-8329afbe36ea", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "c71b1243-2a8a-41d8-934b-3c02d9fd6568"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "c71b1243-2a8a-41d8-934b-3c02d9fd6568", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "021fc271-3c2d-455a-9a57-26c324bfd699"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "021fc271-3c2d-455a-9a57-26c324bfd699", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "311bbe59-e2d7-4cdc-ac23-0f3740bb2260"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "311bbe59-e2d7-4cdc-ac23-0f3740bb2260", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "f0fc5bba-6e91-46fb-81d7-e42097d32ae2"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "f0fc5bba-6e91-46fb-81d7-e42097d32ae2", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "f9345315-7de4-4aaf-b7a8-6fd8d2ed5640"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "f9345315-7de4-4aaf-b7a8-6fd8d2ed5640", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "643316dc-3697-467d-962d-8e200124d99a"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "643316dc-3697-467d-962d-8e200124d99a", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:57Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "ce572ce8-e1e6-4a08-82bc-c192192d1585"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "ce572ce8-e1e6-4a08-82bc-c192192d1585", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "3c86c8dd-7778-4e85-9851-6cc8bc2e1b04"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "3c86c8dd-7778-4e85-9851-6cc8bc2e1b04", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:57Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:57Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:57Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e09433fc-e607-4be2-9c11-e342779cd7dd"} 2025-06-03T11:10:57Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "e09433fc-e607-4be2-9c11-e342779cd7dd", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:58Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:58Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "e48910cb-a8e5-4a94-a87d-3c1579d6c34a"} 2025-06-03T11:10:58Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "e48910cb-a8e5-4a94-a87d-3c1579d6c34a", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:58Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:58Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "9b2f313d-66d8-4312-9b17-bbbc95b54c15"} 2025-06-03T11:10:58Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "9b2f313d-66d8-4312-9b17-bbbc95b54c15", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:58Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:58Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:58Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "78b3ff17-8d70-4a85-86a6-4d397012295e"} 2025-06-03T11:10:58Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "78b3ff17-8d70-4a85-86a6-4d397012295e", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:59Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:59Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "6b7222d7-f0ca-4ef2-91dc-1a3ae39283fa"} 2025-06-03T11:10:59Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "6b7222d7-f0ca-4ef2-91dc-1a3ae39283fa", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:59Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:59Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "b557cb49-fa4b-4d4c-99eb-1c7611bcbac4"} 2025-06-03T11:10:59Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "b557cb49-fa4b-4d4c-99eb-1c7611bcbac4", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:59Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:10:59Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:10:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "ae076297-8ba9-4845-8497-112d0b7d2f86"} 2025-06-03T11:10:59Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "ae076297-8ba9-4845-8497-112d0b7d2f86", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:59Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:10:59Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:10:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "58b1cd54-9df2-4f5b-9917-5a7fea627da3"} 2025-06-03T11:10:59Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "58b1cd54-9df2-4f5b-9917-5a7fea627da3", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:10:59Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:10:59Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:10:59Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "bf8ba04c-4939-4ad3-9f5a-b084b00ab156"} 2025-06-03T11:10:59Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "bf8ba04c-4939-4ad3-9f5a-b084b00ab156", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:02Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:11:02Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:11:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "38c320fa-ee28-4a06-911c-5ac787758ea6"} 2025-06-03T11:11:02Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "38c320fa-ee28-4a06-911c-5ac787758ea6", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:02Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:11:02Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:11:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "3de7c9ac-552a-4af7-ba27-f4e5f91743aa"} 2025-06-03T11:11:02Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "3de7c9ac-552a-4af7-ba27-f4e5f91743aa", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:02Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:11:02Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:11:02Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "7e9988a4-44c7-4308-b75b-cd73484d5597"} 2025-06-03T11:11:02Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "7e9988a4-44c7-4308-b75b-cd73484d5597", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:04Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:11:04Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:11:04Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "d2ec3050-a147-407b-8f4a-af293b3e4363"} 2025-06-03T11:11:04Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "d2ec3050-a147-407b-8f4a-af293b3e4363", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:04Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:11:04Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:11:04Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "73f8d154-c1cd-4f27-89f0-43edf8031ea7"} 2025-06-03T11:11:04Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "73f8d154-c1cd-4f27-89f0-43edf8031ea7", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:07Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:11:07Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:11:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "105ea515-5565-4409-a1a6-47a11ce8169b"} 2025-06-03T11:11:07Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "105ea515-5565-4409-a1a6-47a11ce8169b", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:07Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:11:07Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:11:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "2c13721f-dd68-4fda-a4ac-e2df8e37d0ce"} 2025-06-03T11:11:07Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "2c13721f-dd68-4fda-a4ac-e2df8e37d0ce", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:07Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:11:07Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:11:07Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "da569fa9-49c2-4b33-af2d-f0df1df35f34"} 2025-06-03T11:11:07Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "da569fa9-49c2-4b33-af2d-f0df1df35f34", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:14Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:11:14Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:11:14Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "cfec160e-df61-4c60-a852-be336d01e091"} 2025-06-03T11:11:14Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "cfec160e-df61-4c60-a852-be336d01e091", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:14Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:11:14Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:11:14Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "5e575c08-9e2d-4135-8399-6dd16fb159a8"} 2025-06-03T11:11:14Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "5e575c08-9e2d-4135-8399-6dd16fb159a8", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:17Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:11:17Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:11:17Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "63eeb880-93f2-4962-8da5-ba3b3778c971"} 2025-06-03T11:11:17Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "63eeb880-93f2-4962-8da5-ba3b3778c971", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:17Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:11:17Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:11:17Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "151c142d-9dfe-40ce-a9d3-494fd6ddac17"} 2025-06-03T11:11:17Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "151c142d-9dfe-40ce-a9d3-494fd6ddac17", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:17Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:11:17Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:11:17Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "81f25e6a-ac55-49ff-a7bd-77e8814c8485"} 2025-06-03T11:11:17Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "81f25e6a-ac55-49ff-a7bd-77e8814c8485", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:35Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:11:35Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller time="2025-06-03T11:11:35Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller 2025-06-03T11:11:35Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "8e0a11b2-3c1b-450e-9567-beb812b956c0"} 2025-06-03T11:11:35Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "8e0a11b2-3c1b-450e-9567-beb812b956c0", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:35Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:11:35Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "a75a26a9-39b9-43d7-97bc-87e42e6dc758"} 2025-06-03T11:11:35Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "a75a26a9-39b9-43d7-97bc-87e42e6dc758", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:37Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:11:37Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:11:37Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "240d17be-116b-4ebf-bf59-f21284395e8e"} 2025-06-03T11:11:37Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "240d17be-116b-4ebf-bf59-f21284395e8e", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:37Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:11:37Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:11:37Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "73a42641-e2c6-439a-a2a2-54c34a9b7d7d"} 2025-06-03T11:11:37Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "73a42641-e2c6-439a-a2a2-54c34a9b7d7d", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:11:38Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:11:38Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:11:38Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "aec0c9b1-63a4-49bd-93d8-8cb8e53f5c3e"} 2025-06-03T11:11:38Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "aec0c9b1-63a4-49bd-93d8-8cb8e53f5c3e", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:12:16Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:12:16Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:12:16Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:12:16Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "5e1aed65-4c4d-4228-b714-f0c26cf12d20"} 2025-06-03T11:12:16Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "5e1aed65-4c4d-4228-b714-f0c26cf12d20", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:12:16Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:12:16Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "a835cd46-a9bb-4ed4-85e0-31a212fdec56"} 2025-06-03T11:12:16Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "a835cd46-a9bb-4ed4-85e0-31a212fdec56", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:12:18Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:12:18Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:12:18Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "bd35a685-ec2d-4baf-ac14-519a098a1b17"} 2025-06-03T11:12:18Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "bd35a685-ec2d-4baf-ac14-519a098a1b17", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:12:18Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:12:18Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:12:18Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "4285739f-a3cc-4802-af99-85dc0a924ba7"} 2025-06-03T11:12:18Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "4285739f-a3cc-4802-af99-85dc0a924ba7", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:12:19Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:12:19Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:12:19Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "63dc965b-9ec2-4e1d-9667-22500b761ac3"} 2025-06-03T11:12:19Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "63dc965b-9ec2-4e1d-9667-22500b761ac3", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:13:37Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:13:37Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:13:37Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:13:37Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "8f85f8b7-8c1a-4b81-b2ae-90f217447c2c"} time="2025-06-03T11:13:37Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:13:37Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "c5f60100-0b1c-4d12-a7ce-1c1ea1d4a038"} 2025-06-03T11:13:37Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "8f85f8b7-8c1a-4b81-b2ae-90f217447c2c", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 2025-06-03T11:13:37Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "c5f60100-0b1c-4d12-a7ce-1c1ea1d4a038", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:13:40Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:13:40Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:13:40Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "5da5cd3a-cf88-4dfe-8489-2b72958a0a56"} 2025-06-03T11:13:40Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "5da5cd3a-cf88-4dfe-8489-2b72958a0a56", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:13:40Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:13:40Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:13:40Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "2f9c2835-0555-4d1f-9bc7-ad1a187dfa02"} 2025-06-03T11:13:40Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "2f9c2835-0555-4d1f-9bc7-ad1a187dfa02", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:13:41Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:13:41Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:13:41Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "29355e79-7567-4ebe-823a-c1efd1aa0e33"} 2025-06-03T11:13:41Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "29355e79-7567-4ebe-823a-c1efd1aa0e33", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:16:21Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:16:21Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:16:21Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:16:21Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "72b74bfc-3696-482b-b404-e07946ffec0b"} 2025-06-03T11:16:21Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "72b74bfc-3696-482b-b404-e07946ffec0b", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:16:21Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:16:21Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "0c1330d7-a1d5-4640-9889-f891ed1bd452"} 2025-06-03T11:16:21Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "0c1330d7-a1d5-4640-9889-f891ed1bd452", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:16:24Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:16:24Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:16:24Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "5345c750-ae31-400b-a322-0af628a1763b"} 2025-06-03T11:16:24Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "5345c750-ae31-400b-a322-0af628a1763b", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:16:24Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:16:24Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:16:24Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "9b7c4679-98c1-4b5a-bc43-42b5e16953b0"} 2025-06-03T11:16:24Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "9b7c4679-98c1-4b5a-bc43-42b5e16953b0", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:16:25Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:16:25Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:16:25Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "ad60b82a-1339-41ba-abb1-8a01c8f68877"} 2025-06-03T11:16:25Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "ad60b82a-1339-41ba-abb1-8a01c8f68877", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:21:49Z" level=info msg="Reconciling PersesDatasource: perses-dev/thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:21:49Z" level=info msg="Reconciling PersesDashboard: perses-dev/openshift-cluster-sample-dashboard" module=perses_dashboards_controller time="2025-06-03T11:21:49Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:21:49Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "b4bd8e6a-3464-48b2-a44e-af0e13a47bed"} 2025-06-03T11:21:49Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"openshift-cluster-sample-dashboard","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "openshift-cluster-sample-dashboard", "reconcileID": "b4bd8e6a-3464-48b2-a44e-af0e13a47bed", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:21:49Z" level=error msg="project error: perses-dev" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:21:49Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "9e4c909e-dbaa-4861-b968-1c34a6c4ddaf"} 2025-06-03T11:21:49Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"thanos-querier-datasource","namespace":"perses-dev"}, "namespace": "perses-dev", "name": "thanos-querier-datasource", "reconcileID": "9e4c909e-dbaa-4861-b968-1c34a6c4ddaf", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/perses-dev\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:21:52Z" level=info msg="Reconciling PersesDatasource: openshift-cluster-observability-operator/accelerators-thanos-querier-datasource" module=perses_datasource_controller time="2025-06-03T11:21:52Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=datasource_controller 2025-06-03T11:21:52Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "d95f1866-b8f4-44f2-bf23-769fb3f13f0a"} 2025-06-03T11:21:52Z ERROR Reconciler error {"controller": "persesdatasource", "controllerGroup": "perses.dev", "controllerKind": "PersesDatasource", "PersesDatasource": {"name":"accelerators-thanos-querier-datasource","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-thanos-querier-datasource", "reconcileID": "d95f1866-b8f4-44f2-bf23-769fb3f13f0a", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:21:52Z" level=info msg="Reconciling PersesDashboard: openshift-cluster-observability-operator/accelerators-dashboard" module=perses_dashboards_controller time="2025-06-03T11:21:52Z" level=error msg="project error: openshift-cluster-observability-operator" error="something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused" module=dashboard_controller 2025-06-03T11:21:52Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "45a13a89-3982-4ae4-a366-6e6eeb94dbfb"} 2025-06-03T11:21:52Z ERROR Reconciler error {"controller": "persesdashboard", "controllerGroup": "perses.dev", "controllerKind": "PersesDashboard", "PersesDashboard": {"name":"accelerators-dashboard","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "accelerators-dashboard", "reconcileID": "45a13a89-3982-4ae4-a366-6e6eeb94dbfb", "error": "something wrong happened with the request to the API. Error: Get \"https://perses.openshift-cluster-observability-operator.svc.cluster.local:8080/api/v1/projects/openshift-cluster-observability-operator\": dial tcp 172.30.87.121:8080: connect: connection refused"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249 time="2025-06-03T11:21:52Z" level=info msg="Database SQL configuration is not set, skipping Deployment creation" module=statefulset_controller time="2025-06-03T11:21:52Z" level=error msg="StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbiddenFailed to update StatefulSet with dry run" module=statefulset_controller 2025-06-03T11:21:52Z INFO Warning: Reconciler returned both a non-zero result and a non-nil error. The result will always be ignored if the error is non-nil and the non-nil error causes requeuing with exponential backoff. For more details, see: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/reconcile#Reconciler {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "db26ab68-c804-40ea-baed-ff0c6ddb84b2"} 2025-06-03T11:21:52Z ERROR Reconciler error {"controller": "perses", "controllerGroup": "perses.dev", "controllerKind": "Perses", "Perses": {"name":"perses","namespace":"openshift-cluster-observability-operator"}, "namespace": "openshift-cluster-observability-operator", "name": "perses", "reconcileID": "db26ab68-c804-40ea-baed-ff0c6ddb84b2", "error": "StatefulSet.apps \"perses\" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:341 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:288 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.20.3/pkg/internal/controller/controller.go:249