-
Bug
-
Resolution: Done
-
Blocker
-
MCE 2.4.0, MCE 2.3.0
-
False
-
None
-
False
-
-
-
Low
-
+
-
No
Description of problem:
Update 2.8.3 -> 2.9.0
seeing ocm-controller logs with:
W1121 18:20:58.174443 1 client_config.go:618] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
2023-11-21T18:20:58Z INFO controller-runtime.metrics Metrics server is starting to listen {"addr": ":8080"}
2023-11-21T18:20:58Z INFO Starting server {"kind": "health probe", "addr": "[::]:8000"}
2023-11-21T18:20:58Z INFO starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
I1121 18:20:58.180742 1 leaderelection.go:245] attempting to acquire leader lease multicluster-engine/foundation-controller...
I1121 18:21:15.310612 1 leaderelection.go:255] successfully acquired lease multicluster-engine/foundation-controller
{{2023-11-21T18:21:15Z DEBUG events ocm-controller-67fdf7c846-7btmg_3201d60e-8b15-4675-a73b-ab382a111dbd became leader {"type": "Normal", "object":
, "reason": "LeaderElection"}}}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterinfo-controller", "source": "kind source: *v1.ManagedCluster"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterdetector-controller", "source": "kind source: *v1.ManagedCluster"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterinfo-controller", "source": "kind source: *v1beta1.ManagedClusterInfo"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterdetector-controller", "source": "kind source: *v1beta1.ManagedClusterInfo"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterinfo-controller", "source": "kind source: *v1.Secret"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterdetector-controller", "source": "kind source: *v1.Secret"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clusterinfo-controller"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clusterdetector-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "global-clusterset-controller", "source": "kind source: *v1beta2.ManagedClusterSet"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clustercapcity-controller", "source": "kind source: *v1.ManagedCluster"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "global-clusterset-controller", "source": "kind source: *v1beta2.ManagedClusterSetBinding"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "global-clusterset-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clustercapcity-controller", "source": "kind source: *v1beta1.ManagedClusterInfo"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clustercapcity-controller", "source": "kind source: *v1.Secret"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterrole-controller", "source": "kind source: *v1beta2.ManagedClusterSet"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterrole-controller", "source": "kind source: *v1.ClusterPool"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterclaim-mapper-controller", "source": "kind source: *v1.ClusterClaim"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterrole-controller", "source": "kind source: *v1.ClusterClaim"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterclaim-mapper-controller", "source": "kind source: *v1.ClusterDeployment"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterrole-controller", "source": "kind source: *v1.ClusterDeployment"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterdeployment-mapper-controller", "source": "kind source: *v1.ClusterDeployment"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterrole-controller", "source": "kind source: *v1.ManagedCluster"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterdeployment-mapper-controller", "source": "kind source: *v1.ClusterPool"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterset-clusterdeployment-mapper-controller", "source": "kind source: *v1.ManagedCluster"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clusterset-clusterdeployment-mapper-controller"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clustercapcity-controller"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clusterset-clusterrole-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterrole-controller", "source": "kind source: *v1.ManagedCluster"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clusterset-clusterclaim-mapper-controller"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clusterrole-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "gc-controller", "source": "kind source: *v1beta1.ManagedClusterAction"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "gc-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "certrotation-controller", "source": "kind source: *v1.Secret"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "certrotation-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "clusterca-controller", "source": "kind source: *v1beta1.ManagedClusterInfo"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "addon-install-controller", "source": "kind source: *v1.ManagedCluster"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "clusterca-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "addon-install-controller", "source": "kind source: *v1alpha1.ManagedClusterAddOn"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "addon-install-controller"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "imageregistry-controller", "source": "kind source: *v1alpha1.ManagedClusterImageRegistry"}
2023-11-21T18:21:15Z INFO Starting EventSource {"controller": "imageregistry-controller", "source": "kind source: *v1beta1.PlacementDecision"}
2023-11-21T18:21:15Z INFO Starting Controller {"controller": "imageregistry-controller"}
W1121 18:21:15.357429 1 reflector.go:533] k8s.io/client-go@v12.0.0+incompatible/tools/cache/reflector.go:231: failed to list *v1beta2.ManagedClusterSetBinding: request to convert CR from an invalid group/version: cluster.open-cluster-management.io/v1beta1
E1121 18:21:15.357465 1 reflector.go:148] k8s.io/client-go@v12.0.0+incompatible/tools/cache/reflector.go:231: Failed to watch *v1beta2.ManagedClusterSetBinding: failed to list *v1beta2.ManagedClusterSetBinding: request to convert CR from an invalid group/version: cluster.open-cluster-management.io/v1beta1
I1121 18:21:15.369334 1 base_controller.go:34] Waiting for caches to sync for addon-deploy-controller
I1121 18:21:15.369366 1 base_controller.go:34] Waiting for caches to sync for addon-registration-controller
I1121 18:21:15.369408 1 base_controller.go:34] Waiting for caches to sync for addon-install-controller
I1121 18:21:15.369518 1 base_controller.go:34] Waiting for caches to sync for addon-owner-controller
I1121 18:21:15.369530 1 base_controller.go:34] Waiting for caches to sync for addon-config-controller
I1121 18:21:15.369603 1 base_controller.go:34] Waiting for caches to sync for management-addon-config-controller
I1121 18:21:15.369615 1 base_controller.go:34] Waiting for caches to sync for addon-configuration-controller
I1121 18:21:15.369690 1 base_controller.go:34] Waiting for caches to sync for CSRApprovingController
I1121 18:21:15.369758 1 base_controller.go:34] Waiting for caches to sync for CSRApprovingController
I1121 18:21:15.411784 1 syncclusterrolebinding_controller.go:59] ClusterRolebinding informer synced
I1121 18:21:15.411957 1 syncrolebinding_controller.go:63] Rolebinding Informer synced
{{I1121 18:21:15.469714 1 base_controller.go:40] Caches are synced for management-addon-config-controller }}
{{I1121 18:21:15.469736 1 base_controller.go:40] Caches are synced for addon-configuration-controller }}
{{I1121 18:21:15.469735 1 base_controller.go:40] Caches are synced for addon-install-controller }}
{{I1121 18:21:15.469750 1 base_controller.go:40] Caches are synced for addon-owner-controller }}
I1121 18:21:15.469756 1 base_controller.go:78] Starting #1 worker of addon-owner-controller controller ...
I1121 18:21:15.469756 1 base_controller.go:78] Starting #1 worker of addon-install-controller controller ...
I1121 18:21:15.469741 1 base_controller.go:78] Starting #1 worker of management-addon-config-controller controller ...
{{I1121 18:21:15.469772 1 base_controller.go:40] Caches are synced for addon-config-controller }}
{{I1121 18:21:15.469752 1 base_controller.go:40] Caches are synced for addon-deploy-controller }}
{{I1121 18:21:15.469798 1 base_controller.go:40] Caches are synced for CSRApprovingController }}
I1121 18:21:15.469800 1 base_controller.go:78] Starting #1 worker of addon-deploy-controller controller ...
I1121 18:21:15.469803 1 base_controller.go:78] Starting #1 worker of CSRApprovingController controller ...
I1121 18:21:15.469749 1 base_controller.go:78] Starting #1 worker of addon-configuration-controller controller ...
{{I1121 18:21:15.469769 1 base_controller.go:40] Caches are synced for addon-registration-controller }}
I1121 18:21:15.469842 1 base_controller.go:78] Starting #1 worker of addon-registration-controller controller ...
{{I1121 18:21:15.469724 1 base_controller.go:40] Caches are synced for CSRApprovingController }}
I1121 18:21:15.470242 1 base_controller.go:78] Starting #1 worker of CSRApprovingController controller ...
I1121 18:21:15.469786 1 base_controller.go:78] Starting #1 worker of addon-config-controller controller ...
W1121 18:21:16.684799 1 reflector.go:533] k8s.io/client-go@v12.0.0+incompatible/tools/cache/reflector.go:231: failed to list *v1beta2.ManagedClusterSetBinding: request to convert CR from an invalid group/version: cluster.open-cluster-management.io/v1beta1
E1121 18:21:16.684831 1 reflector.go:148] k8s.io/client-go@v12.0.0+incompatible/tools/cache/reflector.go:231: Failed to watch *v1beta2.ManagedClusterSetBinding: failed to list *v1beta2.ManagedClusterSetBinding: request to convert CR from an invalid group/version: cluster.open-cluster-management...
Version-Release number of selected component (if applicable):
2.9.0
How reproducible:
this is a long-running demonstration environment that has been running since April 2023.
Originally installed ACM 2.7.2 in April 2023.
Steps to Reproduce:
- Start with Hub on 2.7.z
- Manage and synchronize ManagedClusterSet and ManagedClusterSetBinding resources via GitOps
- Upgrade to 2.8.3
- Upgrade to 2.9.0
- check ocm-controller pods they are in CLBO
- ...
Actual results:
ocm-controller pods in CLBO
Expected results:
I do not expect this error and the ocm-controller pods in CLBO
Additional info:
- is related to
-
ACM-8767 Advanced Cluster Management for Kubernetes no longer available in OperatorHub
- Closed
- links to
-
RHSA-2023:124140 Multicluster Engine for Kubernetes 2.4.1 bug fixes