-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
ACM 2.12.2
-
False
-
None
-
False
-
-
-
-
Moderate
-
None
// code placeholder
Description of problem:
If customers change
image.registry.spec.managementState from Managed to Removed, then they reconfigure storage and change back from Removed to Managed from the hosted cluster, the cluster is not able to start new builds or push/pull images to the internal image registry since their corresponding controllers from openshiftcontrollermanager stay down.
Version-Release number of selected component (if applicable):
multicluster-engine.v2.7.3
advanced-cluster-management.v2.12.2
ocp 4.16.10
Baremetal hosted control plane.
How reproducible:
In a hosted cluster change configs.image.registry to Managed, then revert to removed. The cluster wont be able to start buildconfigs or pull/push anymore.
Steps to Reproduce:
1.- From hosted cluster:
$ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec": {"managementState": "Removed"}}'
2.- After some minutes, change back to Managed:
$ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec": {"managementState": "Managed"}}'
3.- Registry pod is up:
NAME READY STATUS RESTARTS AGE image-registry-c97f4765d-hzgzz 1/1 Running 0 82s node-ca-lmgh2 1/1 Running 0 81s node-ca-qqfjn 1/1 Running 0 81s
4.- However, for every new project that is created, token secrets are not created for the default service accounts anymore:
[auron@bastion ~]$ oc new-project issue NAME SECRETS AGE builder 0 6s default 0 6s deployer 0 6s [auron@bastion ~]$ oc get secret No resources found in issue namespace.
5.- Also, bc won't start.
5.- From HCP in the hub cluster, the openshiftcontrollermanager config cm is not reverted:
//Before changing config.imageregistry/cluster $ oc get cm openshift-controller-manager-config -o yaml |grep controller -A2 controllers: null //After changing config.imageregistry/cluster $ oc get cm openshift-controller-manager-config -o yaml |grep controller -A2 controllers: - '*' - -openshift.io/serviceaccount-pull-secrets // After reverting to Managed again: oc get cm openshift-controller-manager-config -o yaml |grep controller -A2 controllers: - '*' - -openshift.io/serviceaccount-pull-secrets // From logs, from openshift-controller-manager, required controller are missing from the Started controllers: $ oc logs openshift-controller-manager-9fc4877d7-lh57f |grep -i started I0320 14:26:45.468986 1 standalone_apiserver.go:105] Started health checks at 0.0.0.0:8443 I0320 14:26:45.504181 1 controller_manager.go:155] Started "openshift.io/image-trigger" I0320 14:26:45.507216 1 controller_manager.go:155] Started "openshift.io/deployer-serviceaccount" I0320 14:26:45.512487 1 controller_manager.go:155] Started "openshift.io/templateinstancefinalizer" I0320 14:26:45.519372 1 controller_manager.go:155] Started "openshift.io/unidling" I0320 14:26:45.523185 1 controller_manager.go:155] Started "openshift.io/origin-namespace" I0320 14:26:45.528728 1 controller_manager.go:155] Started "openshift.io/build-config-change" I0320 14:26:45.532233 1 controller_manager.go:155] Started "openshift.io/builder-rolebindings" I0320 14:26:45.536680 1 controller_manager.go:155] Started "openshift.io/image-signature-import" I0320 14:26:45.567940 1 controller_manager.go:155] Started "openshift.io/templateinstance" I0320 14:26:45.571997 1 controller_manager.go:155] Started "openshift.io/default-rolebindings" I0320 14:26:45.578941 1 controller_manager.go:155] Started "openshift.io/builder-serviceaccount" I0320 14:26:45.593608 1 controller_manager.go:155] Started "openshift.io/deployer-rolebindings" I0320 14:26:45.603683 1 controller_manager.go:155] Started "openshift.io/image-import" I0320 14:26:45.608471 1 controller_manager.go:155] Started "openshift.io/image-puller-rolebindings" I0320 14:26:45.618529 1 controller_manager.go:155] Started "openshift.io/build" I0320 14:26:45.630363 1 controller_manager.go:155] Started "openshift.io/deployer" I0320 14:26:45.643274 1 controller_manager.go:155] Started "openshift.io/deploymentconfig" I0320 14:26:45.643361 1 controller_manager.go:157] Started Origin Controllers //the following controllers are missing: name="openshift.io/internal-image-registry-pull-secrets_image-pull-secret" name="openshift.io/internal-image-registry-pull-secrets_kids" name="openshift.io/internal-image-registry-pull-secrets_legacy-image-pull-secret" name="openshift.io/internal-image-registry-pull-secrets_legacy-token-secret" name="openshift.io/internal-image-registry-pull-secrets_service-account" name="openshift.io/internal-image-registry-pull-secrets_urls"
Actual results:
- Configmap from openshiftcontrollermanager was not reverted and pull, push and build actions fails.
Expected results:
- Pull, push and build actions should work correctly after the image registry is set to Managed again.
Additional info:
Current workaround is to manually edit the cm and change to controllers: null again.
