-
Bug
-
Resolution: Done
-
Undefined
-
ACM 2.9.0
-
False
-
-
False
-
-
-
Critical
-
Yes
Description of problem:
While testing ACM 2.9 deploying many managed clusters and applying the du profile. 100% of the clusters failed to apply the du profile because TALM could not detect there was an installplan waiting for approval on the managed clusters using a Managedclusterview object
Version-Release number of selected component (if applicable):
Hub 4.13.10
Deployed/managed cluster 4.14.0-rc.0
ACM 2.9 Build- 2.9.0-DOWNSTREAM-2023-09-11-04-26-48
It seems this build does not have the issue, so the issue must have been introduced very recently - 2.9.0-DOWNSTREAM-2023-09-07-04-47-52
How reproducible:
Steps to Reproduce:
- ...
Actual results:
Expected results:
Additional info:
Logs from the managed cluster klusterlet-addon-workmgr:
# oc --kubeconfig /root/hv-vm/kc/vm00116/kubeconfig logs -n open-cluster-management-agent-addon klusterlet-addon-workmgr-6d5b4b57f5-wpsn2
...
2023-09-12T13:00:02Z ERROR controllers.ManagedClusterView failed to query resource {"ManagedClusterView": {"name":"vm00116-ztp-install-subscription-ptp-operator-subscription-97lm7","namespace":"vm00116"}, "error": "the server doesn't have a resource type \"Subscription\""}
github.com/stolostron/multicloud-operators-foundation/pkg/klusterlet/view.(*ViewReconciler).Reconcile
github.com/stolostron/multicloud-operators-foundation/pkg/klusterlet/view/view_controller.go:57
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:118
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:314
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226
2023-09-12T13:00:02Z ERROR controllers.ManagedClusterView failed to query resource {"ManagedClusterView": {"name":"vm00116-ztp-install-subscription-cluster-logging-ch8lh","namespace":"vm00116"}, "error": "the server doesn't have a resource type \"Subscription\""}
github.com/stolostron/multicloud-operators-foundation/pkg/klusterlet/view.(*ViewReconciler).Reconcile
github.com/stolostron/multicloud-operators-foundation/pkg/klusterlet/view/view_controller.go:57
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:118
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:314
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226
2023-09-12T13:00:02Z ERROR controllers.ManagedClusterView failed to query resource {"ManagedClusterView": {"name":"vm00116-ztp-install-subscription-sriov-network-operator-subscription-227jk","namespace":"vm00116"}, "error": "the server doesn't have a resource type \"Subscription\""}
github.com/stolostron/multicloud-operators-foundation/pkg/klusterlet/view.(*ViewReconciler).Reconcile
github.com/stolostron/multicloud-operators-foundation/pkg/klusterlet/view/view_controller.go:57
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:118
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:314
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226
It seems the workmgr is complaining it can not access subscriptions despite them existing on the managed cluster:
# oc --kubeconfig /root/hv-vm/kc/vm00116/kubeconfig get subscriptions -A NAMESPACE NAME PACKAGE SOURCE CHANNEL openshift-local-storage local-storage-operator local-storage-operator rh-du-operators stable openshift-logging cluster-logging cluster-logging rh-du-operators stable openshift-ptp ptp-operator-subscription ptp-operator rh-du-operators stable openshift-sriov-network-operator sriov-network-operator-subscription sriov-network-operator rh-du-operators stable