Cluster upgrade from 4.15.21 to 4.16.8 is hanged. 

The following error can be seen in cluster version operator:

    - lastTransitionTime: "2024-09-03T13:51:44Z"
      message: 'Could not update customresourcedefinition "clusterserviceversions.operators.coreos.com"
        (649 of 903): the object is invalid, possibly due to local cluster configuration'
      reason: UpdatePayloadResourceInvalid
      status: "True"
      type: Failing

in the pod logs:

2024-09-02T16:22:06.562594589Z E0902 16:22:06.562523       1 task.go:122] error running apply for customresourcedefinition "clusterserviceversions.operators.coreos.com" (649 of 903): CustomResourceDefinition.apiextensions.k8s.io "clusterserviceversions.operators.coreos.com" is invalid: metadata.ownerReferences: Invalid value: []v1.OwnerReference{v1.OwnerReference{APIVersion:"config.openshift.io/v1", Kind:"ClusterServiceVersion", Name:"rhsso-operator.7.6.9-opr-002", UID:"00f0a902-a305-40bd-b277-2de22dca78ba", Controller:(*bool)(0xc0f2234eb8), BlockOwnerDeletion:(*bool)(nil)}, v1.OwnerReference{APIVersion:"config.openshift.io/v1", Kind:"ClusterVersion", Name:"version", UID:"6412f9f6-7ecf-4bfc-8277-813c9a4ef48d", Controller:(*bool)(0xc0f2234eb9), BlockOwnerDeletion:(*bool)(nil)}}: Only one reference can have Controller set to true. Found "true" in references for ClusterServiceVersion/rhsso-operator.7.6.9-opr-002 and ClusterVersion/version

sso operator has been upgraded manually and successfully to rhsso-operator.7.6.9-opr-003

Checking oc get crd clusterserviceversions.operators.coreos.com 

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    capability.openshift.io/name: OperatorLifecycleManager
    controller-gen.kubebuilder.io/version: v0.8.0
    include.release.openshift.io/ibm-cloud-managed: "true"
    include.release.openshift.io/self-managed-high-availability: "true"
  creationTimestamp: "2024-08-26T20:38:58Z"
  generation: 1
  name: clusterserviceversions.operators.coreos.com
  ownerReferences:
  - apiVersion: config.openshift.io/v1
    controller: true
    kind: ClusterServiceVersion
    name: rhsso-operator.7.6.9-opr-002
    uid: 00f0a902-a305-40bd-b277-2de22dca78ba
  resourceVersion: "615055403"
  uid: cb376acc-c191-4108-8f45-9ea75939f600
spec:
  conversion:
    strategy: None
  group: operators.coreos.com
  names:
    categories:
    - olm
...


Please, feel free to change the subcomponent if you believe it's not related to OLM.


    

 4.16.8

    

    1.
    2.
    3.