Login successful. You have access to 70 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". Login successful. You have access to 83 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". + oc config view --minify --raw=true + //cat userfile ./execute_obs_interop_commands.sh: line 51: //cat: No such file or directory + whoami 1008020000 + rm -rf /alabama/.kube/config + cp userfile /alabama/.kube/config + //cat /alabama/.kube/config ./execute_obs_interop_commands.sh: line 55: //cat: No such file or directory + export KUBECONFIG=/alabama/.kube/config + KUBECONFIG=/alabama/.kube/config + go mod vendor go: downloading github.com/oklog/run v1.1.0 go: downloading github.com/prometheus/common v0.30.0 go: downloading github.com/go-kit/kit v0.11.0 go: downloading github.com/prometheus/prometheus v1.8.2-0.20210811141203-dcb07e8eac34 go: downloading github.com/go-kit/log v0.1.0 go: downloading sigs.k8s.io/controller-runtime v0.10.0 go: downloading sigs.k8s.io/kustomize/v3 v3.3.1 go: downloading open-cluster-management.io/api v0.0.0-20210916013819-2e58cdb938f9 go: downloading github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.48.1 go: downloading open-cluster-management.io/addon-framework v0.0.0-20211014025435-1f42884cdd53 go: downloading github.com/IBM/controller-filtered-cache v0.3.3 go: downloading github.com/openshift/api v0.0.0-20210331193751-3acddb19d360 go: downloading github.com/stolostron/multicloud-operators-foundation v0.0.0-20220112005209-c7d642306a99 go: downloading github.com/stolostron/multiclusterhub-operator v0.0.0-20220106205009-2af6f43fd562 go: downloading github.com/cenkalti/backoff v2.2.1+incompatible go: downloading github.com/stolostron/observatorium-operator v0.0.0-20220307015247-f9eb849e218e go: downloading github.com/openshift/cluster-monitoring-operator v0.1.1-0.20210611103744-7168290cd660 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/cloudflare/cfssl v1.6.0 go: downloading github.com/prometheus-operator/prometheus-operator/pkg/client v0.47.1 go: downloading k8s.io/kubectl v0.21.2 go: downloading github.com/prometheus-community/prom-label-proxy v0.3.0 go: downloading sigs.k8s.io/kustomize/api v0.8.8 go: downloading github.com/openshift/kubernetes-kube-storage-version-migrator v0.0.3-0.20210302135122-481bd04dbc78 go: downloading github.com/onsi/ginkgo v1.16.4 go: downloading github.com/hashicorp/go-version v1.3.0 go: downloading github.com/onsi/gomega v1.16.0 go: downloading github.com/prometheus/alertmanager v0.22.2 go: downloading github.com/google/uuid v1.2.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading k8s.io/klog/v2 v2.10.0 go: downloading golang.org/x/sys v0.0.0-20210817190340-bfb29a6856f2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading golang.org/x/net v0.0.0-20211209124913-491a49abca63 go: downloading github.com/go-logfmt/logfmt v0.5.0 go: downloading github.com/gobuffalo/flect v0.2.2 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/google/go-cmp v0.5.6 go: downloading k8s.io/utils v0.0.0-20210802155522-efc7438f0176 go: downloading google.golang.org/protobuf v1.27.1 go: downloading github.com/go-logr/zapr v0.4.0 go: downloading go.uber.org/zap v1.19.0 go: downloading github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f go: downloading golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914 go: downloading github.com/brancz/locutus v0.0.0-20210511124350-7a84f4d1bcb3 go: downloading sigs.k8s.io/kustomize/kyaml v0.10.17 go: downloading github.com/efficientgo/tools/core v0.0.0-20210201224146-3d78f4d30648 go: downloading github.com/go-openapi/strfmt v0.20.1 go: downloading github.com/go-openapi/runtime v0.19.28 go: downloading k8s.io/apiserver v0.22.1 go: downloading k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7 go: downloading github.com/fsnotify/fsnotify v1.5.1 go: downloading github.com/Azure/go-autorest/autorest v0.11.19 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.14 go: downloading github.com/go-openapi/errors v0.20.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/go-openapi/swag v0.19.15 go: downloading github.com/go-openapi/validate v0.20.2 go: downloading github.com/zmap/zlint/v3 v3.0.0 go: downloading github.com/google/certificate-transparency-go v1.0.21 go: downloading github.com/zmap/zcrypto v0.0.0-20201128221613-3719af1573cf go: downloading github.com/edsrzf/mmap-go v1.0.0 go: downloading github.com/opentracing/opentracing-go v1.2.0 go: downloading github.com/uber/jaeger-client-go v2.29.1+incompatible go: downloading github.com/alecthomas/units v0.0.0-20210208195552-ff826a37aa15 go: downloading github.com/jpillora/backoff v1.0.0 go: downloading go.uber.org/atomic v1.9.0 go: downloading go.uber.org/multierr v1.6.0 go: downloading cloud.google.com/go v0.87.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/go-openapi/spec v0.19.5 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/asaskevich/govalidator v0.0.0-20200907205600-7a23bdc65eef go: downloading github.com/mitchellh/mapstructure v1.4.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading go.mongodb.org/mongo-driver v1.5.1 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/form3tech-oss/jwt-go v3.2.5+incompatible go: downloading golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e go: downloading github.com/jmoiron/sqlx v1.3.1 go: downloading go.uber.org/goleak v1.1.10 go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c go: downloading golang.org/x/text v0.3.5 go: downloading github.com/robfig/cron v1.2.0 go: downloading github.com/go-openapi/analysis v0.19.5 go: downloading github.com/go-openapi/loads v0.19.5 go: downloading github.com/uber/jaeger-lib v2.4.1+incompatible go: downloading github.com/emicklei/go-restful v2.14.2+incompatible go: downloading github.com/dennwc/varint v1.0.0 go: downloading golang.org/x/lint v0.0.0-20210508222113-6edffad5e616 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/weppos/publicsuffix-go v0.13.0 go: downloading golang.org/x/tools v0.1.5 go: downloading github.com/go-stack/stack v1.8.0 + ginkgo build ./tests/pkg/tests/ Compiling tests... compiled tests.test + cd tests + cp resources/options.yaml.template resources/options.yaml + /usr/local/bin/yq e -i '.options.hub.name="ci-op-d2sgicmg-219ee"' resources/options.yaml + /usr/local/bin/yq e -i '.options.hub.baseDomain="ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com"' resources/options.yaml + /usr/local/bin/yq e -i '.options.clusters.name="acm-interops-opp-ci-gcp-1699253092166"' resources/options.yaml + /usr/local/bin/yq e -i '.options.clusters.baseDomain="acm-interops-opp-ci-gcp-1699253092166.gcp.dev09.red-chesterfield.com"' resources/options.yaml + /usr/local/bin/yq e -i '.options.clusters.kubeconfig="/alabama/.kube/managed_kubeconfig"' resources/options.yaml + cat resources/options.yaml options: hub: name: ci-op-d2sgicmg-219ee baseDomain: ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com clusters: name: acm-interops-opp-ci-gcp-1699253092166 baseDomain: acm-interops-opp-ci-gcp-1699253092166.gcp.dev09.red-chesterfield.com kubeconfig: /alabama/.kube/managed_kubeconfig + ginkgo --focus=@ocpInterop -v pkg/tests/ -- -options=../../resources/options.yaml -v=5 I1106 07:24:23.330296 7975 utils.go:211] Kubeconfig path /alabama/.kube/config Running Suite: Observability E2E Suite ====================================== Random Seed: 1699255462 Will run 45 of 62 specs I1106 07:24:23.333165 7975 observability-e2e-test_suite_test.go:158] options filename=../../resources/options.yaml file preview: ../../resources/options.yaml E1106 07:24:23.334182 7975 observability-e2e-test_suite_test.go:170] --options error: yaml: unmarshal errors: line 6: cannot unmarshal !!map into []utils.Cluster I1106 07:24:23.334204 7975 observability-e2e-test_suite_test.go:198] ownerPrefix=ginkgo I1106 07:24:23.334211 7975 observability-e2e-test_suite_test.go:205] ocpRelease=4.4.4 I1106 07:24:23.334216 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.334221 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.334895 7975 utils.go:73] Create kubeclient dynamic for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.334920 7975 utils.go:211] Kubeconfig path /alabama/.kube/config STEP: Deploy CM cluster-monitoring-config I1106 07:24:23.341288 7975 utils.go:256] obj:map[] I1106 07:24:23.341374 7975 utils.go:269] kind: ConfigMap I1106 07:24:23.341387 7975 utils.go:277] apiVersion: v1 I1106 07:24:23.341391 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.341394 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.342007 7975 utils.go:88] Create kubeclient apiextension for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.342019 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.342433 7975 utils.go:374] Install ConfigMap: apiVersion: v1 data: config.yaml: |- alertmanagerMain: enableUserAlertmanagerConfig: true kind: ConfigMap metadata: name: cluster-monitoring-config namespace: openshift-monitoring W1106 07:24:23.407884 7975 utils.go:389] ConfigMap openshift-monitoring/cluster-monitoring-config already exists, updating! STEP: Checking MCO operator is started up and running STEP: Checking Required CRDs are created I1106 07:24:23.476943 7975 utils.go:88] Create kubeclient apiextension for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.476960 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.477568 7975 utils.go:636] Check if multiclusterobservabilities.observability.open-cluster-management.io exists I1106 07:24:23.507664 7975 utils.go:636] Check if observatoria.core.observatorium.io exists I1106 07:24:23.540212 7975 utils.go:636] Check if observabilityaddons.observability.open-cluster-management.io exists I1106 07:24:23.555517 7975 mco_deploy.go:798] Create open-cluster-management-observability namespaces I1106 07:24:23.555530 7975 utils.go:256] obj:map[] I1106 07:24:23.555594 7975 utils.go:269] kind: Namespace I1106 07:24:23.555603 7975 utils.go:277] apiVersion: v1 I1106 07:24:23.555606 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.555609 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.556368 7975 utils.go:88] Create kubeclient apiextension for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.556379 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.556863 7975 utils.go:304] Install Namespace: apiVersion: v1 kind: Namespace metadata: name: open-cluster-management-observability I1106 07:24:23.592227 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.592244 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.593059 7975 utils.go:73] Create kubeclient dynamic for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.593075 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.684932 7975 mco_deploy.go:778] Delete existing pullSecret - secrets "multiclusterhub-operator-pull-secret" not found I1106 07:24:23.684959 7975 mco_deploy.go:785] Create MCO pull secret I1106 07:24:23.702622 7975 mco_deploy.go:849] Create MCO object storage secret I1106 07:24:23.702640 7975 utils.go:256] obj:map[] I1106 07:24:23.702731 7975 utils.go:269] kind: Secret I1106 07:24:23.702746 7975 utils.go:277] apiVersion: v1 I1106 07:24:23.702749 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.702753 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.703552 7975 utils.go:88] Create kubeclient apiextension for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:23.703563 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:23.703975 7975 utils.go:357] Install Secret: apiVersion: v1 kind: Secret metadata: name: thanos-object-storage namespace: open-cluster-management-observability stringData: thanos.yaml: | type: s3 config: bucket: obs-v1 endpoint: s3.us-east-1.amazonaws.com insecure: false access_key: XXXXXXXXXXXXXXXXXXXX secret_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX type: Opaque I1106 07:24:24.436788 7975 utils.go:256] obj:map[] I1106 07:24:24.436881 7975 utils.go:269] kind: LimitRange I1106 07:24:24.436891 7975 utils.go:277] apiVersion: v1 I1106 07:24:24.436895 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.436899 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.437737 7975 utils.go:88] Create kubeclient apiextension for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.437748 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.438241 7975 utils.go:452] Install LimitRange: apiVersion: v1 kind: LimitRange metadata: name: limit-range namespace: open-cluster-management-observability spec: limits: - defaultRequest: cpu: 10m memory: 50Mi max: cpu: "5.25" memory: 12Gi type: Container I1106 07:24:24.470939 7975 utils.go:256] obj:map[] I1106 07:24:24.471038 7975 utils.go:269] kind: ResourceQuota I1106 07:24:24.471047 7975 utils.go:277] apiVersion: v1 I1106 07:24:24.471051 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.471054 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.472402 7975 utils.go:88] Create kubeclient apiextension for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.472420 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.473453 7975 utils.go:471] Install ResourceQuota: apiVersion: v1 kind: ResourceQuota metadata: name: resource-quota namespace: open-cluster-management-observability spec: hard: cpu: "5.25" memory: 15Gi STEP: Creating the MCO testing RBAC resources I1106 07:24:24.505492 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.505506 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.523761 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.523776 7975 utils.go:211] Kubeconfig path /alabama/.kube/config STEP: Creating MCO instance of v1beta1 I1106 07:24:24.546115 7975 utils.go:256] obj:map[] I1106 07:24:24.546211 7975 utils.go:269] kind: MultiClusterObservability I1106 07:24:24.546223 7975 utils.go:277] apiVersion: observability.open-cluster-management.io/v1beta1 I1106 07:24:24.546228 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.546231 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.547005 7975 utils.go:88] Create kubeclient apiextension for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.547016 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.547523 7975 utils.go:514] Install MultiClusterObservability: apiVersion: observability.open-cluster-management.io/v1beta1 kind: MultiClusterObservability metadata: annotations: null name: observability spec: nodeSelector: kubernetes.io/os: linux observabilityAddonSpec: {} retentionResolution1h: 30d retentionResolution5m: 14d retentionResolutionRaw: 5d storageConfigObject: metricObjectStorage: key: thanos.yaml name: thanos-object-storage statefulSetSize: 2Gi statefulSetStorageClass: gp2 I1106 07:24:24.547535 7975 utils.go:73] Create kubeclient dynamic for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.547538 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:24:24.580005 7975 utils.go:73] Create kubeclient dynamic for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:24:24.580023 7975 utils.go:211] Kubeconfig path /alabama/.kube/config W1106 07:24:24.595467 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:24.627147 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability STEP: Waiting for MCO ready status W1106 07:24:24.642932 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:29.659947 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:34.679614 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:39.696619 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:44.714549 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:49.732016 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:54.749766 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:24:59.766710 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:04.784638 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:09.801783 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:14.820010 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:19.838739 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:24.856288 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:29.874006 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:34.891941 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:39.910219 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:44.928217 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:49.948806 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:54.966239 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:25:59.984739 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:05.003521 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:10.021368 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:15.039841 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:20.058079 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:25.100725 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:30.135039 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:35.153414 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:40.172447 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:45.190198 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:50.208018 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:26:55.226008 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:00.243448 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:05.262234 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:10.280892 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:15.298914 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:20.317441 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:25.334786 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:30.353095 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:35.372201 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:40.391180 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:45.411540 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:50.429455 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:27:55.448491 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:00.467747 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:05.488757 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:10.507480 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:15.525744 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:20.544505 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:25.592205 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:30.611906 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:35.631806 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:40.650064 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:45.667426 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:50.685896 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:28:55.707665 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:00.726532 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:05.745040 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:10.761809 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:15.780937 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:20.798973 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:25.816779 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:30.835604 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:35.852530 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:40.871398 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:45.889726 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:50.909465 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:29:55.928237 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:00.946483 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:05.963699 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:10.982219 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:16.001561 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:21.020543 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:26.061106 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:31.079836 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:36.098670 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:41.119172 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:46.137183 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:51.155549 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:30:56.173653 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:01.191922 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:06.210834 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:11.231428 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:16.249567 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:21.269376 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:26.289552 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:31.309384 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:36.329625 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:41.347144 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:46.366518 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:51.389836 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:31:56.409527 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:01.430184 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:06.446884 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:11.466831 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:16.486216 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:21.505195 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:26.551147 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:31.569075 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:36.588248 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:41.608496 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:46.625998 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:51.645427 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:32:56.662963 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:01.682557 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:06.702458 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:11.723167 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:16.741739 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:21.760383 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:26.780275 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:31.797756 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:36.817621 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:41.838346 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:46.856011 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:51.874555 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:33:56.895439 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:01.916116 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:06.935961 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:11.953202 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:16.973105 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:21.993194 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:27.045012 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:32.064921 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:37.082164 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:42.102386 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:47.120646 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:52.140747 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:34:57.160768 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:02.178153 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:07.198452 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:12.218710 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:17.239357 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:22.260123 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:27.276920 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:32.295862 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:37.315971 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:42.336020 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:47.357999 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:52.376165 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:35:57.397830 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:02.418959 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:07.439392 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:12.461675 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:17.479845 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:22.500780 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:27.549532 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:32.568997 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:37.589916 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:42.606998 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:47.624549 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:52.645124 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:36:57.665971 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:02.686144 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:07.704738 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:12.726895 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:17.746732 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:22.767035 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:27.786784 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:32.804307 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:37.824929 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:42.844978 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:47.865193 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:52.886486 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:37:57.903805 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:02.924407 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:07.941490 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:12.960169 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:17.983444 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:23.000865 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:28.064351 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:33.084550 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:38.109138 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:43.129043 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:48.146547 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:53.167134 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:38:58.188242 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:03.209655 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:08.230479 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:13.251105 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:18.271707 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:23.293849 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:28.310878 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:33.328817 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:38.346766 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:43.364521 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:48.382823 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:53.401377 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:39:58.419317 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:03.437324 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:08.455211 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:13.472968 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:18.489910 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:23.508207 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:28.551036 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:33.567750 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:38.585941 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:43.603970 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:48.621320 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:53.638743 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:40:58.656846 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:03.675223 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:08.692369 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:13.709019 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:18.727919 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:23.745818 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:28.764014 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:33.781780 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:38.799728 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:43.817528 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:48.834719 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:53.852840 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:41:58.870299 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:03.888120 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:08.911033 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:13.928782 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:18.946444 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:23.964180 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:29.000962 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:34.019085 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:39.037380 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:44.055204 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:49.072231 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:54.090695 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:42:59.108453 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:04.126861 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:09.144625 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:14.163103 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:19.180618 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:24.198314 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:29.215995 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:34.233082 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:39.250982 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:44.267980 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:49.284989 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:54.303325 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:43:59.321497 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:44:04.339680 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:44:09.356883 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:44:14.374079 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:44:19.391326 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability W1106 07:44:24.410092 7975 warnings.go:70] observability.open-cluster-management.io/v1beta1 MultiClusterObservability is deprecated in v2.3+, unavailable in v2.6+; use observability.open-cluster-management.io/v1beta2 MultiClusterObservability I1106 07:44:24.627676 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:44:24.627708 7975 utils.go:211] Kubeconfig path /alabama/.kube/config [DEBUG] MCO is installed failed, checking MCO operator logs: 2023-11-06T07:25:18.442Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:25:18.442Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:25:18.443Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "local-cluster", "request.namespace": ""} 2023-11-06T07:25:18.443Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:25:18.443Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:25:18.443Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:25:18.443Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:25:18.443Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:25:18.443Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:25:18.443Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:25:18.685Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:20.686Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:22.687Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:24.687Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:26.688Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:28.689Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:30.689Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:32.690Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:34.352Z INFO controller_placementrule UpdateFunc {"managedCluster": "local-cluster"} 2023-11-06T07:25:34.352Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "", "Request.Name": "local-cluster"} 2023-11-06T07:25:34.352Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:25:34.352Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:25:34.352Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:25:34.352Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:25:34.352Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:25:34.352Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:25:34.352Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:25:34.352Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:25:34.352Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:25:34.353Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:25:34.353Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "local-cluster", "request.namespace": ""} 2023-11-06T07:25:34.353Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:25:34.353Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:25:34.353Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:25:34.353Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:25:34.353Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:25:34.353Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:25:34.353Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:25:34.691Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:36.692Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:38.693Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:40.694Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:42.695Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:44.696Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:46.696Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:48.697Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:50.698Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:52.699Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:54.700Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:56.701Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:25:58.702Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:00.703Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:02.703Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:04.704Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:06.705Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:08.706Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:10.707Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:12.708Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:14.709Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:16.710Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:18.711Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:20.712Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:22.712Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:24.713Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:26.714Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:28.715Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:30.717Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:32.717Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:34.718Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:36.719Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:38.720Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:40.721Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:42.722Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:44.723Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:46.724Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:48.725Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:50.726Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:52.727Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:54.728Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:56.729Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:26:58.730Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:00.731Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:02.733Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:04.734Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:06.735Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:08.736Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:10.736Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:12.738Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:14.738Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:16.739Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:18.740Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:20.741Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:22.742Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:24.743Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:26.743Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:28.744Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:30.752Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:32.753Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:34.754Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:36.756Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:38.757Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:40.758Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:42.759Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:44.759Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:46.760Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:48.762Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:50.763Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:52.764Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:54.765Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:56.766Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:27:58.769Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:00.770Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:02.771Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:04.771Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:06.773Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:08.774Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:10.774Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:12.776Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:14.777Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:16.777Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:18.778Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:20.779Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:22.779Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:24.780Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:26.781Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:28.782Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:30.783Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:32.784Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:34.785Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:36.786Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:38.787Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:40.787Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:42.789Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:44.790Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:46.791Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:48.792Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:50.793Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:52.795Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:54.796Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:56.796Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:28:58.797Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:00.798Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:02.799Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:04.800Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:06.801Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:08.802Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:10.803Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:12.804Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:14.805Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:16.807Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:18.808Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:20.808Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:22.808Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:24.809Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:26.809Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:28.810Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:30.811Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:32.812Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:34.813Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:36.814Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:38.814Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:40.815Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:42.816Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:44.817Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:46.818Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:48.819Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:50.820Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:52.821Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:54.822Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:56.823Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:29:57.497Z INFO controller_multiclustermonitoring Reconciling MultiClusterObservability {"Request.Namespace": "", "Request.Name": "observability"} 2023-11-06T07:29:57.498Z DEBUG config operand is {"key": "grafana", "name": "observability-grafana"} 2023-11-06T07:29:57.498Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-grafana-rhel8@sha256:0251758b149b0ce13a2b8133876f891499041c4e8360b8a2e30a01dca83636f1"} 2023-11-06T07:29:57.498Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/grafana-dashboard-loader-rhel8@sha256:c65e3d2641a5ad6707a82aef285bdd08935c626a5dd20142d25838dbf2e65d7e"} 2023-11-06T07:29:57.498Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:29:57.498Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:29:57.498Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:29:57.498Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:29:57.498Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:29:57.498Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-alertmanager-rhel8@sha256:c41965a22e042dff2beeca00246f6fb0bcffa00304a089081dce30e078d71160"} 2023-11-06T07:29:57.498Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:29:57.498Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:29:57.502Z DEBUG config operand is {"key": "rbac-query-proxy", "name": "observability-rbac-query-proxy"} 2023-11-06T07:29:57.502Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:29:57.502Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:29:57.502Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/rbac-query-proxy-rhel8@sha256:845ca4f7af7f7ed517bfd2a379825d66a055cc954e29279c1c12206956704562"} 2023-11-06T07:29:57.502Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:29:57.502Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8-operator@sha256:5502f270c41fd6a50146af22808b1767265d65074b59425ac99f69f862ee99be"} 2023-11-06T07:29:57.502Z DEBUG config operand is {"key": "observatorium-operator", "name": "observability-observatorium-operator"} 2023-11-06T07:29:57.515Z INFO deploying deployerFn not found {"kind:": "Role"} 2023-11-06T07:29:57.518Z INFO deploying deployerFn not found {"kind:": "RoleBinding"} 2023-11-06T07:29:57.522Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:29:57.531Z INFO deploying Skip creation {"Kind:": "/v1, Kind=ConfigMap", "Name:": "observability-managed-cluster-label-allowlist"} 2023-11-06T07:29:57.546Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:29:57.550Z INFO deploying Update {"Kind:": "/v1, Kind=Secret", "Name:": "grafana-config"} 2023-11-06T07:29:57.750Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:29:57.754Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-config"} 2023-11-06T07:29:57.758Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-proxy"} 2023-11-06T07:29:57.796Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:29:57.821Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "rbac-proxy-cookie-secret"} 2023-11-06T07:29:57.829Z INFO deploying deployerFn not found {"kind:": "Ingress"} 2023-11-06T07:29:57.836Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:29:57.853Z INFO controller_certificates CA secrets already existed {"name": "observability-server-ca-certs"} 2023-11-06T07:29:57.853Z INFO controller_certificates CA secrets already existed {"name": "observability-client-ca-certs"} 2023-11-06T07:29:57.853Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:29:57.853Z INFO controller_certificates Certificate secrets already existed {"name": "observability-server-certs"} 2023-11-06T07:29:57.853Z INFO controller_certificates Certificate secrets already existed {"name": "observability-grafana-certs"} 2023-11-06T07:29:57.854Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:29:57.854Z INFO controller_multiclustermonitoring storageClassSelected {"storageClassSelected": "gp3-csi"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8@sha256:5adb986a2ed36fa9805b046c94b7bf09b9b3d34c21d9945e6efc2a7212b1788d"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-receive-controller-rhel8@sha256:ddb7fae1f274ceb449a902d0d2dac3ea51214f2091c7cdd2a252ff813967327d"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:29:57.854Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-rhel8@sha256:9aee6ab2f4d6c3ad1fdfcc0f8ba78b7994663d6787fcc73fb02bf907283b274f"} 2023-11-06T07:29:57.854Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:29:57.855Z INFO controller_multiclustermonitoring StorageVersionMigration already existed/unchanged {"name": "storage-version-migration-observability"} 2023-11-06T07:29:57.855Z DEBUG controller_multiclustermonitoring status update goroutine is triggered. 2023-11-06T07:29:58.824Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:00.825Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:02.826Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:04.827Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:06.828Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:08.829Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:10.832Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:12.474Z INFO controller_multiclustermonitoring Reconciling MultiClusterObservability {"Request.Namespace": "", "Request.Name": "observability"} 2023-11-06T07:30:12.475Z DEBUG config operand is {"key": "grafana", "name": "observability-grafana"} 2023-11-06T07:30:12.475Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-grafana-rhel8@sha256:0251758b149b0ce13a2b8133876f891499041c4e8360b8a2e30a01dca83636f1"} 2023-11-06T07:30:12.475Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/grafana-dashboard-loader-rhel8@sha256:c65e3d2641a5ad6707a82aef285bdd08935c626a5dd20142d25838dbf2e65d7e"} 2023-11-06T07:30:12.475Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:12.476Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.476Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.476Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.476Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.476Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-alertmanager-rhel8@sha256:c41965a22e042dff2beeca00246f6fb0bcffa00304a089081dce30e078d71160"} 2023-11-06T07:30:12.476Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:12.476Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:12.477Z DEBUG config operand is {"key": "rbac-query-proxy", "name": "observability-rbac-query-proxy"} 2023-11-06T07:30:12.486Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:12.486Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:12.486Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/rbac-query-proxy-rhel8@sha256:845ca4f7af7f7ed517bfd2a379825d66a055cc954e29279c1c12206956704562"} 2023-11-06T07:30:12.486Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:12.487Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8-operator@sha256:5502f270c41fd6a50146af22808b1767265d65074b59425ac99f69f862ee99be"} 2023-11-06T07:30:12.487Z DEBUG config operand is {"key": "observatorium-operator", "name": "observability-observatorium-operator"} 2023-11-06T07:30:12.499Z INFO deploying deployerFn not found {"kind:": "Role"} 2023-11-06T07:30:12.503Z INFO deploying deployerFn not found {"kind:": "RoleBinding"} 2023-11-06T07:30:12.506Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:12.515Z INFO deploying Skip creation {"Kind:": "/v1, Kind=ConfigMap", "Name:": "observability-managed-cluster-label-allowlist"} 2023-11-06T07:30:12.530Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:12.534Z INFO deploying Update {"Kind:": "/v1, Kind=Secret", "Name:": "grafana-config"} 2023-11-06T07:30:12.716Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:12.720Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-config"} 2023-11-06T07:30:12.723Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-proxy"} 2023-11-06T07:30:12.764Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:12.788Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "rbac-proxy-cookie-secret"} 2023-11-06T07:30:12.801Z INFO deploying deployerFn not found {"kind:": "Ingress"} 2023-11-06T07:30:12.812Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:12.831Z INFO controller_certificates CA secrets already existed {"name": "observability-server-ca-certs"} 2023-11-06T07:30:12.831Z INFO controller_certificates CA secrets already existed {"name": "observability-client-ca-certs"} 2023-11-06T07:30:12.831Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:12.831Z INFO controller_certificates Certificate secrets already existed {"name": "observability-server-certs"} 2023-11-06T07:30:12.831Z INFO controller_certificates Certificate secrets already existed {"name": "observability-grafana-certs"} 2023-11-06T07:30:12.831Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:12.832Z INFO controller_multiclustermonitoring storageClassSelected {"storageClassSelected": "gp3-csi"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8@sha256:5adb986a2ed36fa9805b046c94b7bf09b9b3d34c21d9945e6efc2a7212b1788d"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-receive-controller-rhel8@sha256:ddb7fae1f274ceb449a902d0d2dac3ea51214f2091c7cdd2a252ff813967327d"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:12.832Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-rhel8@sha256:9aee6ab2f4d6c3ad1fdfcc0f8ba78b7994663d6787fcc73fb02bf907283b274f"} 2023-11-06T07:30:12.832Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:12.832Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:12.834Z INFO controller_multiclustermonitoring StorageVersionMigration already existed/unchanged {"name": "storage-version-migration-observability"} 2023-11-06T07:30:12.834Z INFO controller_multiclustermonitoring Reconciling MultiClusterObservability {"Request.Namespace": "", "Request.Name": "observability"} 2023-11-06T07:30:12.834Z DEBUG config operand is {"key": "grafana", "name": "observability-grafana"} 2023-11-06T07:30:12.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-grafana-rhel8@sha256:0251758b149b0ce13a2b8133876f891499041c4e8360b8a2e30a01dca83636f1"} 2023-11-06T07:30:12.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/grafana-dashboard-loader-rhel8@sha256:c65e3d2641a5ad6707a82aef285bdd08935c626a5dd20142d25838dbf2e65d7e"} 2023-11-06T07:30:12.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:12.835Z DEBUG controller_multiclustermonitoring status update goroutine is triggered. 2023-11-06T07:30:12.835Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.835Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.835Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.835Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:12.835Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-alertmanager-rhel8@sha256:c41965a22e042dff2beeca00246f6fb0bcffa00304a089081dce30e078d71160"} 2023-11-06T07:30:12.835Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:12.835Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:12.835Z DEBUG config operand is {"key": "rbac-query-proxy", "name": "observability-rbac-query-proxy"} 2023-11-06T07:30:12.835Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:12.835Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:12.835Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/rbac-query-proxy-rhel8@sha256:845ca4f7af7f7ed517bfd2a379825d66a055cc954e29279c1c12206956704562"} 2023-11-06T07:30:12.835Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:12.836Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8-operator@sha256:5502f270c41fd6a50146af22808b1767265d65074b59425ac99f69f862ee99be"} 2023-11-06T07:30:12.836Z DEBUG config operand is {"key": "observatorium-operator", "name": "observability-observatorium-operator"} 2023-11-06T07:30:12.848Z INFO deploying deployerFn not found {"kind:": "Role"} 2023-11-06T07:30:12.851Z INFO deploying deployerFn not found {"kind:": "RoleBinding"} 2023-11-06T07:30:12.855Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:12.862Z INFO deploying Skip creation {"Kind:": "/v1, Kind=ConfigMap", "Name:": "observability-managed-cluster-label-allowlist"} 2023-11-06T07:30:12.878Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:12.881Z INFO deploying Update {"Kind:": "/v1, Kind=Secret", "Name:": "grafana-config"} 2023-11-06T07:30:14.023Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:14.027Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-config"} 2023-11-06T07:30:14.030Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-proxy"} 2023-11-06T07:30:14.136Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:14.184Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "rbac-proxy-cookie-secret"} 2023-11-06T07:30:14.193Z INFO deploying deployerFn not found {"kind:": "Ingress"} 2023-11-06T07:30:14.222Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:14.244Z INFO controller_certificates CA secrets already existed {"name": "observability-server-ca-certs"} 2023-11-06T07:30:14.244Z INFO controller_certificates CA secrets already existed {"name": "observability-client-ca-certs"} 2023-11-06T07:30:14.244Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:14.244Z INFO controller_certificates Certificate secrets already existed {"name": "observability-server-certs"} 2023-11-06T07:30:14.244Z INFO controller_certificates Certificate secrets already existed {"name": "observability-grafana-certs"} 2023-11-06T07:30:14.244Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:14.245Z INFO controller_multiclustermonitoring storageClassSelected {"storageClassSelected": "gp3-csi"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8@sha256:5adb986a2ed36fa9805b046c94b7bf09b9b3d34c21d9945e6efc2a7212b1788d"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-receive-controller-rhel8@sha256:ddb7fae1f274ceb449a902d0d2dac3ea51214f2091c7cdd2a252ff813967327d"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:14.245Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-rhel8@sha256:9aee6ab2f4d6c3ad1fdfcc0f8ba78b7994663d6787fcc73fb02bf907283b274f"} 2023-11-06T07:30:14.245Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:14.246Z INFO controller_multiclustermonitoring StorageVersionMigration already existed/unchanged {"name": "storage-version-migration-observability"} 2023-11-06T07:30:14.246Z DEBUG controller_multiclustermonitoring status update goroutine is triggered. 2023-11-06T07:30:14.833Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:16.834Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:18.835Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:20.836Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:22.837Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:24.838Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:26.838Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:28.839Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:30.842Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:32.843Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:34.844Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:36.845Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:37.887Z INFO controller_multiclustermonitoring Reconciling MultiClusterObservability {"Request.Namespace": "", "Request.Name": "observability"} 2023-11-06T07:30:37.887Z DEBUG config operand is {"key": "grafana", "name": "observability-grafana"} 2023-11-06T07:30:37.892Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-grafana-rhel8@sha256:0251758b149b0ce13a2b8133876f891499041c4e8360b8a2e30a01dca83636f1"} 2023-11-06T07:30:37.892Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/grafana-dashboard-loader-rhel8@sha256:c65e3d2641a5ad6707a82aef285bdd08935c626a5dd20142d25838dbf2e65d7e"} 2023-11-06T07:30:37.892Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:37.893Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:37.893Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:37.893Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:37.893Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:37.893Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-alertmanager-rhel8@sha256:c41965a22e042dff2beeca00246f6fb0bcffa00304a089081dce30e078d71160"} 2023-11-06T07:30:37.893Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:37.893Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:37.894Z DEBUG config operand is {"key": "rbac-query-proxy", "name": "observability-rbac-query-proxy"} 2023-11-06T07:30:37.894Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:37.894Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:37.894Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/rbac-query-proxy-rhel8@sha256:845ca4f7af7f7ed517bfd2a379825d66a055cc954e29279c1c12206956704562"} 2023-11-06T07:30:37.894Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:37.894Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8-operator@sha256:5502f270c41fd6a50146af22808b1767265d65074b59425ac99f69f862ee99be"} 2023-11-06T07:30:37.894Z DEBUG config operand is {"key": "observatorium-operator", "name": "observability-observatorium-operator"} 2023-11-06T07:30:37.908Z INFO deploying deployerFn not found {"kind:": "Role"} 2023-11-06T07:30:37.912Z INFO deploying deployerFn not found {"kind:": "RoleBinding"} 2023-11-06T07:30:37.923Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:37.932Z INFO deploying Skip creation {"Kind:": "/v1, Kind=ConfigMap", "Name:": "observability-managed-cluster-label-allowlist"} 2023-11-06T07:30:37.952Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:37.955Z INFO deploying Update {"Kind:": "/v1, Kind=Secret", "Name:": "grafana-config"} 2023-11-06T07:30:38.163Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:38.166Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-config"} 2023-11-06T07:30:38.170Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-proxy"} 2023-11-06T07:30:38.206Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:38.231Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "rbac-proxy-cookie-secret"} 2023-11-06T07:30:38.240Z INFO deploying deployerFn not found {"kind:": "Ingress"} 2023-11-06T07:30:38.247Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:38.267Z INFO controller_certificates CA secrets already existed {"name": "observability-server-ca-certs"} 2023-11-06T07:30:38.267Z INFO controller_certificates CA secrets already existed {"name": "observability-client-ca-certs"} 2023-11-06T07:30:38.267Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:38.267Z INFO controller_certificates Certificate secrets already existed {"name": "observability-server-certs"} 2023-11-06T07:30:38.267Z INFO controller_certificates Certificate secrets already existed {"name": "observability-grafana-certs"} 2023-11-06T07:30:38.267Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:38.268Z INFO controller_multiclustermonitoring storageClassSelected {"storageClassSelected": "gp3-csi"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8@sha256:5adb986a2ed36fa9805b046c94b7bf09b9b3d34c21d9945e6efc2a7212b1788d"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-receive-controller-rhel8@sha256:ddb7fae1f274ceb449a902d0d2dac3ea51214f2091c7cdd2a252ff813967327d"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:38.268Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-rhel8@sha256:9aee6ab2f4d6c3ad1fdfcc0f8ba78b7994663d6787fcc73fb02bf907283b274f"} 2023-11-06T07:30:38.268Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:38.269Z INFO controller_multiclustermonitoring StorageVersionMigration already existed/unchanged {"name": "storage-version-migration-observability"} 2023-11-06T07:30:38.269Z DEBUG controller_multiclustermonitoring status update goroutine is triggered. 2023-11-06T07:30:38.281Z INFO controller_multiclustermonitoring Reconciling MultiClusterObservability {"Request.Namespace": "", "Request.Name": "observability"} 2023-11-06T07:30:38.282Z DEBUG config operand is {"key": "grafana", "name": "observability-grafana"} 2023-11-06T07:30:38.282Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-grafana-rhel8@sha256:0251758b149b0ce13a2b8133876f891499041c4e8360b8a2e30a01dca83636f1"} 2023-11-06T07:30:38.282Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/grafana-dashboard-loader-rhel8@sha256:c65e3d2641a5ad6707a82aef285bdd08935c626a5dd20142d25838dbf2e65d7e"} 2023-11-06T07:30:38.282Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:38.282Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:38.283Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:38.283Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:38.283Z DEBUG config operand is {"key": "alertmanager", "name": "observability-alertmanager"} 2023-11-06T07:30:38.283Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-alertmanager-rhel8@sha256:c41965a22e042dff2beeca00246f6fb0bcffa00304a089081dce30e078d71160"} 2023-11-06T07:30:38.283Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:38.283Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:38.283Z DEBUG config operand is {"key": "rbac-query-proxy", "name": "observability-rbac-query-proxy"} 2023-11-06T07:30:38.283Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:38.283Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:38.283Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/rbac-query-proxy-rhel8@sha256:845ca4f7af7f7ed517bfd2a379825d66a055cc954e29279c1c12206956704562"} 2023-11-06T07:30:38.283Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-oauth-proxy@sha256:41762eda03298362d3f97e1fd0be4af429d22658b520f94023941655e062d1a1"} 2023-11-06T07:30:38.283Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8-operator@sha256:5502f270c41fd6a50146af22808b1767265d65074b59425ac99f69f862ee99be"} 2023-11-06T07:30:38.283Z DEBUG config operand is {"key": "observatorium-operator", "name": "observability-observatorium-operator"} 2023-11-06T07:30:38.295Z INFO deploying deployerFn not found {"kind:": "Role"} 2023-11-06T07:30:38.299Z INFO deploying deployerFn not found {"kind:": "RoleBinding"} 2023-11-06T07:30:38.302Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:38.311Z INFO deploying Skip creation {"Kind:": "/v1, Kind=ConfigMap", "Name:": "observability-managed-cluster-label-allowlist"} 2023-11-06T07:30:38.327Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:38.331Z INFO deploying Update {"Kind:": "/v1, Kind=Secret", "Name:": "grafana-config"} 2023-11-06T07:30:38.846Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:39.442Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:39.446Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-config"} 2023-11-06T07:30:39.450Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "alertmanager-proxy"} 2023-11-06T07:30:39.555Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:39.602Z INFO deploying Skip creation {"Kind:": "/v1, Kind=Secret", "Name:": "rbac-proxy-cookie-secret"} 2023-11-06T07:30:39.611Z INFO deploying deployerFn not found {"kind:": "Ingress"} 2023-11-06T07:30:39.637Z INFO deploying deployerFn not found {"kind:": "ServiceAccount"} 2023-11-06T07:30:39.656Z INFO controller_certificates CA secrets already existed {"name": "observability-server-ca-certs"} 2023-11-06T07:30:39.656Z INFO controller_certificates CA secrets already existed {"name": "observability-client-ca-certs"} 2023-11-06T07:30:39.656Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:39.657Z INFO controller_certificates Certificate secrets already existed {"name": "observability-server-certs"} 2023-11-06T07:30:39.657Z INFO controller_certificates Certificate secrets already existed {"name": "observability-grafana-certs"} 2023-11-06T07:30:39.657Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:39.657Z INFO controller_multiclustermonitoring storageClassSelected {"storageClassSelected": "gp3-csi"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/observatorium-rhel8@sha256:5adb986a2ed36fa9805b046c94b7bf09b9b3d34c21d9945e6efc2a7212b1788d"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/openshift4/ose-configmap-reloader@sha256:fa5b2e42a27e3ab5f0a0d2a103b5602b81d1b29eb788b54cc60ad547744ad053"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-receive-controller-rhel8@sha256:ddb7fae1f274ceb449a902d0d2dac3ea51214f2091c7cdd2a252ff813967327d"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-rhel8@sha256:44af8559fea3f293f8649ae52bf88d0aacb1c53fa0bc84adf66c1f165ec23337"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/memcached-exporter-rhel8@sha256:bb8071451fe2dad89b1cfbf849ccb4e1e5169a69cc0a112b118a5120fa4b3429"} 2023-11-06T07:30:39.657Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/thanos-rhel8@sha256:9aee6ab2f4d6c3ad1fdfcc0f8ba78b7994663d6787fcc73fb02bf907283b274f"} 2023-11-06T07:30:39.657Z DEBUG config operand is {"key": "observatorium", "name": "observability"} 2023-11-06T07:30:39.659Z INFO controller_multiclustermonitoring StorageVersionMigration already existed/unchanged {"name": "storage-version-migration-observability"} 2023-11-06T07:30:39.660Z DEBUG controller_multiclustermonitoring status update goroutine is triggered. 2023-11-06T07:30:40.846Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:42.848Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:44.852Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:46.852Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:48.854Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:50.855Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:52.856Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:54.857Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:56.858Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:30:58.859Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:00.860Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:02.861Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:04.862Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:06.863Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:08.863Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:10.864Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:12.864Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:14.865Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:16.867Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:18.868Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:20.868Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:22.869Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:24.869Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:26.870Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:28.870Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:30.871Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:32.872Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:34.872Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:36.873Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:38.874Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:40.875Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:42.876Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:44.877Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:46.879Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:48.879Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:50.880Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:52.881Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:54.883Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:56.884Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:31:58.885Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:00.885Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:02.887Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:04.888Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:06.888Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:08.891Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:10.893Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:12.894Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:14.894Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:16.896Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:18.897Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:20.897Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:22.898Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:24.899Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:26.900Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:28.900Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:30.901Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:32.901Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:34.902Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:36.903Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:38.904Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:40.905Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:42.906Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:44.908Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:46.908Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:48.909Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:50.911Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:52.911Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:54.913Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:56.914Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:32:58.915Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:00.915Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:02.916Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:04.917Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:06.918Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:08.919Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:10.920Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:12.921Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:14.922Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:16.923Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:18.924Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:20.925Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:22.925Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:24.926Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:26.927Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:28.928Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:30.928Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:32.929Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:34.930Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:36.930Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:38.932Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:40.932Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:42.934Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:44.935Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:46.936Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:48.936Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:50.937Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:52.939Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:54.940Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:56.940Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:33:58.941Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:00.942Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:02.943Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:04.944Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:06.945Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:08.945Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:10.947Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:12.947Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:14.948Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:16.949Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:18.950Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:20.951Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:22.951Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:24.952Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:26.953Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:28.954Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:30.955Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:32.956Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:34.957Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:36.958Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:38.958Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:40.960Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:42.961Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:44.962Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:46.963Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:48.964Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:50.966Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:52.967Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:54.968Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:56.969Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:34:58.970Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:00.971Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:02.972Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:04.973Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:06.974Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:08.976Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:10.977Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:12.979Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:14.979Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:16.980Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:18.981Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:20.983Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:22.984Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:24.985Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:26.986Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:28.987Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:30.988Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:32.988Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:34.989Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:36.990Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:38.991Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:40.992Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:42.993Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:44.994Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:46.995Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:48.996Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:50.997Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:52.998Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:54.999Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:57.000Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:35:59.001Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:01.002Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:03.002Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:05.003Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:07.004Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:09.005Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:11.006Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:13.007Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:15.009Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:17.010Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:19.011Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:21.012Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:23.013Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:25.014Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:27.015Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:29.016Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:31.017Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:33.018Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:35.018Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:37.020Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:39.020Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:41.021Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:43.022Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:45.023Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:47.024Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:49.025Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:51.025Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:53.027Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:55.027Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:57.028Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:36:59.030Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:01.031Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:03.032Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:05.033Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:07.034Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:09.035Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:11.035Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:13.036Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:15.037Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:17.038Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:19.038Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:21.039Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:23.039Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:25.040Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:27.042Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:29.043Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:31.044Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:33.045Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:35.046Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:37.046Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:39.047Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:41.048Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:43.048Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:45.049Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:47.051Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:49.051Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:51.052Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:53.054Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:55.055Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:57.056Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:37:59.057Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:01.058Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:03.059Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:05.061Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:07.061Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:09.062Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:11.063Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:13.064Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:15.065Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:17.066Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:19.067Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:21.068Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:23.069Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:25.070Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:27.071Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:29.072Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:31.072Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:33.073Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:35.074Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:37.075Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:39.076Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:41.077Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:43.079Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:45.080Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:47.081Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:49.082Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:51.083Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:53.084Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:55.084Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:57.085Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:38:59.086Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:01.087Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:03.088Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:05.090Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:07.090Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:09.091Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:11.093Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:13.094Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:15.095Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:17.095Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:19.096Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:21.097Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:23.098Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:25.099Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:27.100Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:29.100Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:31.101Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:33.102Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:35.102Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:37.103Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:39.104Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:41.105Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:43.106Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:45.106Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:47.109Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:49.110Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:51.111Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:53.114Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:55.115Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:57.116Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:39:59.119Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:01.119Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:03.120Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:05.121Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:07.123Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:09.124Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:11.125Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:13.126Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:15.127Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:17.128Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:19.129Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:21.130Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:23.131Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:25.132Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:27.133Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:29.134Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:31.135Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:33.136Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:35.137Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:37.138Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:39.139Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:41.139Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:43.140Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:45.141Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:47.142Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:49.143Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:51.144Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:53.145Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:55.146Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:57.147Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:40:59.147Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:01.148Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:03.149Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:05.150Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:07.151Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:09.152Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:09.786Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "local-cluster", "Request.Name": "observability-addon"} 2023-11-06T07:41:09.786Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:41:09.786Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:41:09.786Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:41:09.786Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:41:09.786Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:41:09.786Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:41:09.786Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:41:09.786Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:41:09.786Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:41:09.787Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:41:09.787Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "observability-addon", "request.namespace": "local-cluster"} 2023-11-06T07:41:09.787Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:09.787Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:09.787Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:09.787Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:41:09.787Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:41:09.787Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:41:09.787Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:41:09.832Z INFO controller_placementrule Updated status for managedclusteraddon {"namespace": "local-cluster"} 2023-11-06T07:41:09.875Z INFO controller_placementrule UpdateFunc {"managedCluster": "local-cluster"} 2023-11-06T07:41:09.875Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "", "Request.Name": "local-cluster"} 2023-11-06T07:41:09.875Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:41:09.875Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:41:09.875Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:41:09.875Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:41:09.875Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:41:09.876Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:41:09.876Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:41:09.876Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:41:09.876Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:41:09.876Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:41:09.876Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "local-cluster", "request.namespace": ""} 2023-11-06T07:41:09.876Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:09.876Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:09.876Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:09.876Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:41:09.877Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:41:09.877Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:41:09.877Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:41:11.153Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:13.154Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:15.155Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:17.156Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:19.156Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:19.954Z INFO controller_placementrule UpdateFunc {"managedCluster": "local-cluster"} 2023-11-06T07:41:19.954Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "", "Request.Name": "local-cluster"} 2023-11-06T07:41:19.955Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:41:19.955Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:41:19.955Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:41:19.955Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:41:19.955Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:41:19.955Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:41:19.955Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:41:19.955Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:41:19.955Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:41:19.956Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:41:19.956Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "local-cluster", "request.namespace": ""} 2023-11-06T07:41:19.956Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:19.956Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:19.956Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:19.956Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:41:19.956Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:41:19.956Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:41:19.956Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:41:21.157Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:23.158Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:23.688Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "local-cluster", "Request.Name": "observability-addon"} 2023-11-06T07:41:23.688Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:41:23.688Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:41:23.688Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:41:23.688Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "observability-addon", "request.namespace": "local-cluster"} 2023-11-06T07:41:23.688Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:23.688Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:23.688Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:23.688Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:41:23.688Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:41:23.688Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:41:23.689Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:41:23.717Z INFO controller_placementrule Updated status for managedclusteraddon {"namespace": "local-cluster"} 2023-11-06T07:41:23.741Z INFO controller_placementrule UpdateFunc {"managedCluster": "local-cluster"} 2023-11-06T07:41:23.741Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "", "Request.Name": "local-cluster"} 2023-11-06T07:41:23.741Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:41:23.741Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:41:23.741Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:41:23.741Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:41:23.741Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:41:23.741Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:41:23.741Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:41:23.741Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:41:23.741Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:41:23.742Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:41:23.742Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "local-cluster", "request.namespace": ""} 2023-11-06T07:41:23.742Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:23.742Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:23.742Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:23.742Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:41:23.742Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:41:23.742Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:41:23.742Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:41:25.159Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:27.160Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:29.161Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:31.162Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:33.164Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:35.165Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:37.166Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:39.167Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:41.168Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:43.169Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:45.170Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:47.171Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:49.172Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:49.944Z INFO controller_placementrule UpdateFunc {"managedCluster": "local-cluster"} 2023-11-06T07:41:49.944Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "", "Request.Name": "local-cluster"} 2023-11-06T07:41:49.944Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:41:49.944Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:41:49.944Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:41:49.944Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:41:49.944Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:41:49.945Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:41:49.945Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:41:49.945Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:41:49.945Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:41:49.945Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:41:49.945Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "local-cluster", "request.namespace": ""} 2023-11-06T07:41:49.945Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:49.945Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:49.945Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:41:49.945Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:41:49.945Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:41:49.945Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:41:49.946Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:41:51.173Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:53.174Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:55.175Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:57.176Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:41:59.177Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:01.177Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:03.179Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:05.179Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:07.180Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:09.181Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:11.182Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:13.186Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:15.187Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:17.188Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:19.189Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:21.190Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:23.191Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:24.833Z INFO controller_placementrule Reconciling PlacementRule {"Request.Namespace": "local-cluster", "Request.Name": "observability-addon"} 2023-11-06T07:42:24.834Z INFO util observability-controller clustermanagementaddon is present 2023-11-06T07:42:24.834Z INFO controller_placementrule load and update templates for managedcluster resources 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-config-reloader-rhel8@sha256:42398b6bb7faee3c1573449f859498754c5fca1aa3d514d1aaac35a2a8cb2eb0"} 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/acm-prometheus-rhel8@sha256:a873f302cd0abde70d7f493424706c24dfc414ce38a6b609aade3ab72ab313cd"} 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-rbac-proxy-rhel8@sha256:c9c935c682fc31df05674a071131ca87f70bb3857871dc0372ed23121ff84d88"} 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/kube-state-metrics-rhel8@sha256:2a49e0e7166530487ef6bb794082ff093a663e13549a0b2fb3533e3c4efa5630"} 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/metrics-collector-rhel8@sha256:1024aa5b771d4644394cf9f58d4ca46df2b9cf889beed5f55c073682dcbd6c84"} 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/node-exporter-rhel8@sha256:df1c5dbfa3a96e55426c7deeb4a0b80ef8797784681ec7b21dfdb56aef432b2d"} 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/prometheus-rhel8@sha256:6122baa8a0c92771e5a1a2385ce34a6337b0afeb75300ba63115d6e64c213041"} 2023-11-06T07:42:24.834Z DEBUG config image replacement {"componentName": "registry.redhat.io/rhacm2/endpoint-monitoring-rhel8-operator@sha256:cbf87d47a575866c134f19820b3c470d4f9b1cd5caaf14363375a514275c912d"} 2023-11-06T07:42:24.834Z INFO controller_placementrule Monitoring operator should be installed in cluster {"cluster_name": "local-cluster", "request.name": "observability-addon", "request.namespace": "local-cluster"} 2023-11-06T07:42:24.834Z INFO controller_placementrule observabilityaddon already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:42:24.834Z INFO controller_placementrule clusterrolebinding endpoint-observability-mco-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:42:24.834Z INFO controller_placementrule rolebinding endpoint-observability-res-rolebinding already existed/unchanged {"namespace": "local-cluster"} 2023-11-06T07:42:24.834Z INFO util ManagedClusterAddOn is created or updated successfully {"name": "observability-controller", "namespace": "local-cluster"} 2023-11-06T07:42:24.834Z INFO controller_placementrule Cluster: local-cluster, Spec.NodeSelector (after): map[kubernetes.io/os:linux] 2023-11-06T07:42:24.834Z INFO controller_placementrule Cluster: local-cluster, Spec.Tolerations (after): [] 2023-11-06T07:42:24.835Z INFO controller_placementrule Updating manifestwork {"local-cluster": "local-cluster", "name": "local-cluster-observability"} 2023-11-06T07:42:24.864Z INFO controller_placementrule Updated status for managedclusteraddon {"namespace": "local-cluster"} 2023-11-06T07:42:25.192Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:27.192Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:29.193Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:31.194Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:33.195Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:35.196Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:37.197Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:39.198Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:41.199Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:43.199Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:45.201Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:47.201Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:49.202Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:51.203Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:53.203Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:55.205Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:57.206Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:42:59.207Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:01.208Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:03.209Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:05.210Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:07.210Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:09.211Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:11.213Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:13.214Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:15.215Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:17.216Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:19.216Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:21.217Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:23.218Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:25.219Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:27.220Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:29.221Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:31.222Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:33.222Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:35.222Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:37.224Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:39.225Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:41.226Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:43.226Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:45.227Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:47.228Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:49.229Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:51.230Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:53.231Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:55.232Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:57.232Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:43:59.233Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:01.234Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:03.235Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:05.236Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:07.237Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:09.237Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:11.238Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:13.239Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:15.240Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:17.241Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:19.242Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:21.243Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. 2023-11-06T07:44:23.244Z DEBUG controller_multiclustermonitoring check status ready goroutine is triggered. Failure [1201.357 seconds] [BeforeSuite] BeforeSuite  /tmp/obs/tests/pkg/tests/observability-e2e-test_suite_test.go:132 Timed out after 1200.000s. Expected success, but got an error: <*errors.errorString | 0xc000fc0e70>: { s: "MCO componnets cannot be running in 20 minutes. check the MCO CR status for the details: map[conditions:[map[lastTransitionTime:2023-11-06T07:24:33Z message:Installation is in progress reason:Installing status:True type:Installing] map[lastTransitionTime:2023-11-06T07:24:33Z message:StatefulSet observability-thanos-receive-default is not ready reason:StatefulSetNotReady status:False type:Failed]]]", } MCO componnets cannot be running in 20 minutes. check the MCO CR status for the details: map[conditions:[map[lastTransitionTime:2023-11-06T07:24:33Z message:Installation is in progress reason:Installing status:True type:Installing] map[lastTransitionTime:2023-11-06T07:24:33Z message:StatefulSet observability-thanos-receive-default is not ready reason:StatefulSetNotReady status:False type:Failed]]] /tmp/obs/tests/pkg/tests/observability_install_test.go:160 ------------------------------ I1106 07:44:24.689836 7975 utils.go:58] Create kubeclient for url https://api.ci-op-d2sgicmg-219ee.cspilp.interop.ccitredhat.com:6443 using kubeconfig path /alabama/.kube/config I1106 07:44:24.689860 7975 utils.go:211] Kubeconfig path /alabama/.kube/config I1106 07:44:24.716009 7975 mco_deploy.go:161] Get <31> pods in namespace JUnit report was created: /tmp/obs/tests/pkg/tests/junit_results.xml Ran 45 of 0 Specs in 1201.383 seconds FAIL! -- 0 Passed | 45 Failed | 0 Pending | 0 Skipped --- FAIL: TestObservabilityE2E (1201.39s) You're using deprecated Ginkgo functionality: ============================================= Ginkgo 2.0 is under active development and will introduce (a small number of) breaking changes. To learn more, view the migration guide at https://github.com/onsi/ginkgo/blob/v2/docs/MIGRATING_TO_V2.md To comment, chime in at https://github.com/onsi/ginkgo/issues/711 You are using a custom reporter. Support for custom reporters will likely be removed in V2. Most users were using them to generate junit or teamcity reports and this functionality will be merged into the core reporter. In addition, Ginkgo 2.0 will support emitting a JSON-formatted report that users can then manipulate to generate custom reports. If this change will be impactful to you please leave a comment on https://github.com/onsi/ginkgo/issues/711 Learn more at: https://github.com/onsi/ginkgo/blob/v2/docs/MIGRATING_TO_V2.md#removed-custom-reporters To silence deprecations that can be silenced set the following environment variable: ACK_GINKGO_DEPRECATIONS=1.16.4 FAIL Ginkgo ran 1 suite in 20m1.893074992s Test Suite Failed