-
Bug
-
Resolution: Done
-
Critical
-
ACM 2.10.0, MCE 2.5.0
-
None
Description of problem:
The hypershift-addon agent is showing the following error upon startup:
I0108 18:26:12.896595 1 request.go:682] Waited for 1.020741761s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/hive.openshift.io/v1?timeout=32s 2024-01-08T18:26:21.481Z INFO agent.agent-reconciler install/upgrade.go:72 check if HyperShift operator re-installation is required (startup=true, installfailed=false) 2024-01-08T18:26:21.481Z INFO agent.agent-reconciler install/upgrade.go:77 hypershift operator re-installation is required 2024-01-08T18:26:21.481Z INFO agent.agent-reconciler agent/agent.go:318 heath probes server is running... 2024-01-08T18:26:21.489Z INFO agent.agent-reconciler install/hypershift.go:187 reinstallCheckRequired = false 2024-01-08T18:26:21.523Z INFO agent.controller-manager-setup agent/agent.go:232 the maximum hosted cluster count set to 80 2024-01-08T18:26:21.524Z INFO agent.controller-manager-setup agent/agent.go:233 the threshold hosted cluster count set to 60 2024-01-08T18:26:21.535Z INFO agent.agent-reconciler install/hypershift.go:616 createorupdate the pull secret (hypershift/open-cluster-management-image-pull-credentials) 2024-01-08T18:26:21.582Z INFO agent.agent-reconciler install/hypershift.go:218 bucket secret(local-cluster/hypershift-operator-oidc-provider-s3-credentials) not found on the hub. 2024-01-08T18:26:21.593Z INFO agent.agent-reconciler install/hypershift.go:230 private link secret(local-cluster/hypershift-operator-private-link-credentials) not found on the hub. 2024-01-08T18:26:21.593Z INFO agent.agent-reconciler install/hypershift.go:241 bucket secret(local-cluster/hypershift-operator-oidc-provider-s3-credentials) and private link secret(local-cluster/hypershift-operator-private-link-credentials) not found on the hub, installing hypershift operator for non-AWS platform. 2024-01-08T18:26:21.602Z INFO agent.agent-reconciler install/hypershift.go:313 external dns secret(local-cluster/hypershift-operator-external-dns-credentials) was not found I0108 18:26:22.933230 1 request.go:682] Waited for 1.3504795s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/infrastructure.cluster.x-k8s.io/v1beta2?timeout=32s 2024-01-08T18:26:27.686Z INFO agent.agent-reconciler agent/agent.go:912 this is the initial agent startup and the hypershift CRDs are not installed yet, no matches for kind "HostedCluster" in version "hypershift.openshift.io/v1beta1" 2024-01-08T18:26:27.686Z INFO agent.agent-reconciler agent/agent.go:913 going to continue updating AddOnPlacementScore and cluster claims with zero HC count 2024-01-08T18:26:27.693Z INFO agent.agent-reconciler agent/agent.go:991 updated the addOnPlacementScore for local-cluster: 0 2024-01-08T18:26:27.693Z INFO agent.agent-reconciler agent/clusterclaim.go:73 the hosted cluster count has not reached the maximum 80 yet. current count is 0 2024-01-08T18:26:27.694Z INFO agent.agent-reconciler install/install_job.go:20 HyperShift install args: [--namespace hypershift --image-refs /tmp/hypershift-operator-imagestream --platform-monitoring OperatorOnly --enable-uwm-telemetry-remote-write --enable-defaulting-webhook --enable-validating-webhook] 2024-01-08T18:26:27.725Z INFO agent.agent-reconciler install/install_job.go:106 created HyperShift install job: hypershift-install-job-kzvx9 2024-01-08T18:26:27.725Z INFO agent.agent-reconciler agent/agent.go:1012 updated the hosted cluster cound cluster claims successfully 2024-01-08T18:26:27.730Z ERROR agent.agent-reconciler agent/agent.go:788 failed to get configmap from the hub. Setting the HCP sizing baseline with default values. {"error": "configmaps \"hcp-sizing-baseline\" not found"} github.com/stolostron/hypershift-addon-operator/pkg/agent.(*agentController).SetHCPSizingBaseline /remote-source/hypershift-addon-operator/app/pkg/agent/agent.go:788 github.com/stolostron/hypershift-addon-operator/pkg/agent.(*AgentOptions).runControllerManager /remote-source/hypershift-addon-operator/app/pkg/agent/agent.go:244 github.com/stolostron/hypershift-addon-operator/pkg/agent.NewAgentCommand.func1 /remote-source/hypershift-addon-operator/app/pkg/agent/agent.go:82 github.com/spf13/cobra.(*Command).execute /remote-source/hypershift-addon-operator/app/vendor/github.com/spf13/cobra/command.go:856 github.com/spf13/cobra.(*Command).ExecuteC /remote-source/hypershift-addon-operator/app/vendor/github.com/spf13/cobra/command.go:974 github.com/spf13/cobra.(*Command).Execute /remote-source/hypershift-addon-operator/app/vendor/github.com/spf13/cobra/command.go:902 main.main /remote-source/hypershift-addon-operator/app/cmd/main.go:41 runtime.main /usr/lib/golang/src/runtime/proc.go:267 I0108 18:26:32.983146 1 request.go:682] Waited for 5.197303059s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/controlplane.operator.openshift.io/v1alpha1?timeout=32s 2024-01-08T18:26:33.937Z ERROR agent.agent-reconciler agent/hcp_capacity_calculation.go:75 failed to list hosted control planes {"error": "no matches for kind \"HostedControlPlane\" in version \"hypershift.openshift.io/v1beta1\""} github.com/stolostron/hypershift-addon-operator/pkg/agent.(*agentController).calculateCapacitiesToHostHCPs /remote-source/hypershift-addon-operator/app/pkg/agent/hcp_capacity_calculation.go:75 github.com/stolostron/hypershift-addon-operator/pkg/agent.(*AgentOptions).runControllerManager /remote-source/hypershift-addon-operator/app/pkg/agent/agent.go:246 github.com/stolostron/hypershift-addon-operator/pkg/agent.NewAgentCommand.func1 /remote-source/hypershift-addon-operator/app/pkg/agent/agent.go:82 github.com/spf13/cobra.(*Command).execute /remote-source/hypershift-addon-operator/app/vendor/github.com/spf13/cobra/command.go:856 github.com/spf13/cobra.(*Command).ExecuteC /remote-source/hypershift-addon-operator/app/vendor/github.com/spf13/cobra/command.go:974 github.com/spf13/cobra.(*Command).Execute /remote-source/hypershift-addon-operator/app/vendor/github.com/spf13/cobra/command.go:902 main.main /remote-source/hypershift-addon-operator/app/cmd/main.go:41 runtime.main /usr/lib/golang/src/runtime/proc.go:267 2024-01-08T18:26:33.937Z INFO agent.controller-manager-setup agent/agent.go:248 starting manager I0108 18:26:43.009006 1 request.go:682] Waited for 4.188887235s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/machine.openshift.io/v1beta1?timeout=32s 2024-01-08T18:26:44.965Z INFO agent.agent-reconciler.external-secret-controller agent/external_secret_controller.go:54 reconciling klusterlet: klusterlet 2024-01-08T18:26:44.965Z INFO agent.agent-reconciler.external-secret-controller agent/external_secret_controller.go:58 klusterlet not from a hosted cluster 2024-01-08T18:26:44.965Z INFO agent.agent-reconciler.external-secret-controller agent/external_secret_controller.go:59 done reconciling klusterlet: klusterlet 2024-01-08T18:26:45.066Z INFO agent.agent-reconciler.addon-status-controller agent/addon_status_controller.go:65 reconciling Deployment hypershift/operator 2024-01-08T18:26:45.096Z INFO agent.agent-reconciler.addon-status-controller agent/addon_status_controller.go:98 updated ManagedClusterAddOnStatus
This results in the new cluster capacity metrics all showing 0 (since no baselines are set).
Version-Release number of selected component (if applicable):
ACM 2.10.0-DOWNSTREAM-2024-01-05-20-32-42
Hub is on OCP 4.15.0-rc.0 with fips enabled
How reproducible:
always
Steps to Reproduce:
- install latest acm/mce
- check hypershift addon agent log
- ...
Actual results:
Expected results:
Additional info:
- is caused by
-
ACM-8738 Hosting cluster capacity monitoring dashboard
- Closed