-
Bug
-
Resolution: Done
-
Critical
-
None
-
ACM 2.7.0
-
None
-
1
-
False
-
None
-
False
-
-
-
ACM Sprint 22
-
No
Description of problem:
ile provisioning and applying the du profile on 270 SNOs, the policy part of the du profile fails to set a status on du profile policies:
# oc get policy -n sno00270 NAME REMEDIATION ACTION COMPLIANCE STATE AGE ztp-common.common-config-policy inform Compliant 2d17h ztp-common.common-subscriptions-policy inform 2d17h ztp-group.group-du-sno-config-log-policy inform NonCompliant 2d17h ztp-group.group-du-sno-config-policy inform NonCompliant 2d17h ztp-group.group-du-sno-config-storage-policy inform NonCompliant 2d17h ztp-install.sno00270-common-subscriptions-policy-5m2lb enforce 55m
While looking at the container running on the SNO we find a stack trace indicating why perhaps no status is being set which blocks the du profile from completing:
# oc --kubeconfig=/root/hv-vm/sno/manifests/sno00270/kubeconfig get po -n open-cluster-management-agent-addon
NAME READY STATUS RESTARTS AGE
cluster-proxy-proxy-agent-7b99f4c97f-75c97 2/2 Running 0 2d16h
cluster-proxy-proxy-agent-7b99f4c97f-pnzz2 2/2 Running 0 2d16h
config-policy-controller-56b5f78866-nvjgw 2/2 Running 9 (41h ago) 2d16h
governance-policy-framework-5b89bf4c98-c4998 1/1 Running 5 (41h ago) 2d16h
klusterlet-addon-workmgr-7b7c6c54df-4xppg 1/1 Running 0 2d16h
# oc --kubeconfig=/root/hv-vm/sno/manifests/sno00270/kubeconfig logs -n open-cluster-management-agent-addon config-policy-controller-56b5f78866-nvjgw -c config-policy-controller
...
2022-11-07T14:54:57.895Z info configuration-policy-controller controllers/configurationpolicy_controller.go:903 Handling the object template for the relevant namespace {"policy": "sno00270-common-subscriptions-policy-config-527qb", "namespace": "openshift-local-storage", "desiredName": "openshift-local-storage", "index": 13}
2022-11-07T14:54:57.900Z info configuration-policy-controller controllers/configurationpolicy_controller.go:1086 Will update the policy status {"policy": "sno00270-common-subscriptions-policy-config-527qb", "complianceState": "Compliant"}
2022-11-07T14:54:57.900Z info configuration-policy-controller controllers/configurationpolicy_controller.go:903 Handling the object template for the relevant namespace {"policy": "sno00270-common-subscriptions-policy-config-527qb", "namespace": "openshift-local-storage", "desiredName": "local-storage-operator", "index": 14}
2022-11-07T14:54:57.900Z info configuration-policy-controller controllers/configurationpolicy_controller.go:903 Handling the object template for the relevant namespace {"policy": "common-config-policy-config", "namespace": "openshift-marketplace", "desiredName": "rh-du-operators", "index": 2}
2022-11-07T14:54:57.907Z info configuration-policy-controller controllers/configurationpolicy_controller.go:1086 Will update the policy status {"policy": "sno00270-common-subscriptions-policy-config-527qb", "complianceState": "Compliant"}
2022-11-07T14:54:57.907Z info configuration-policy-controller controllers/configurationpolicy_controller.go:903 Handling the object template for the relevant namespace {"policy": "sno00270-common-subscriptions-policy-config-527qb", "namespace": "", "desiredName": "local-storage-operator.openshift-local-storage", "index": 15}
2022-11-07T14:54:57.907Z info configuration-policy-controller controllers/configurationpolicy_controller.go:903 Handling the object template for the relevant namespace {"policy": "common-config-policy-config", "namespace": "", "desiredName": "disconnected-internal-icsp", "index": 3}
2022-11-07T14:54:57.916Z info configuration-policy-controller controllers/configurationpolicy_controller.go:1086 Will update the policy status {"policy": "sno00270-common-subscriptions-policy-config-527qb", "complianceState": "Compliant"}
2022-11-07T14:54:57.921Z error configuration-policy-controller controllers/configurationpolicy_controller.go:2428 Could not update status, retrying next loop {"name": "sno00270-common-subscriptions-policy-config-527qb", "namespace": "sno00270", "error": "Event \"ztp-install.sno00270-common-subscriptions-policy-5m2lb.1725552f210cc4a3\" is invalid: message: Invalid value: \"\": can have at most 1024 characters"}
open-cluster-management.io/config-policy-controller/controllers.(*ConfigurationPolicyReconciler).addForUpdate
/remote-source/app/controllers/configurationpolicy_controller.go:2428
open-cluster-management.io/config-policy-controller/controllers.(*ConfigurationPolicyReconciler).checkRelatedAndUpdate
/remote-source/app/controllers/configurationpolicy_controller.go:978
open-cluster-management.io/config-policy-controller/controllers.(*ConfigurationPolicyReconciler).handleObjectTemplates
/remote-source/app/controllers/configurationpolicy_controller.go:969
open-cluster-management.io/config-policy-controller/controllers.(*ConfigurationPolicyReconciler).handlePolicyWorker
/remote-source/app/controllers/configurationpolicy_controller.go:283
Version-Release number of selected component (if applicable):
Hub OCP cluster 4.11.13
SNO OCP Cluster 4.11.13
ACM version 2.7.0-DOWNSTREAM-2022-11-03-17-37-17
How reproducible:
All SNOs 270/270 are not progressing
Steps to Reproduce:
- ...
Actual results:
Expected results:
Additional info:
- relates to
-
-
- ON_QA
-
-
-
- ON_QA
-
-
-
- ON_QA
-
-
-
- ON_QA
-
-
-
- ON_QA
-