Upgrade a TP cluster from 4.18.0-ec.3 to 4.18.0-0.nightly-multi-2024-11-19-013314, it stuck on customresourcedefinition \"olms.operator.openshift.io\" update. It seems the spec.versions changed from v1alpha1 to v1.

$ omc get clusterversion
NAME      VERSION       AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.18.0-ec.3   True        True          2h16m   Unable to apply 4.18.0-0.nightly-multi-2024-11-19-013314: some cluster configuration is invalid

$ omc get clusterversion version -ojson|jq .status.conditions[]
...
{
  "lastTransitionTime": "2024-11-19T12:36:20Z",
  "message": "Could not update customresourcedefinition \"olms.operator.openshift.io\" (74 of 954): the object is invalid, possibly due to local cluster configuration",
  "reason": "UpdatePayloadResourceInvalid",
  "status": "True",
  "type": "Failing"
}
...
{
  "lastTransitionTime": "2024-11-19T12:36:20Z",
  "message": "{\"message\":\"Could not update customresourcedefinition \\\"olms.operator.openshift.io\\\" (74 of 954): the object is invalid, possibly due to local cluster configuration\",\"children\":[{\"message\":\"CustomResourceDefinition.apiextensions.k8s.io \\\"olms.operator.openshift.io\\\" is invalid: status.storedVersions[0]: Invalid value: \\\"v1alpha1\\\": must appear in spec.versions\"}],\"manifest\":{\"originalFilename\":\"0000_10_operator-lifecycle-manager_01_olms-TechPreviewNoUpgrade.crd.yaml\",\"group\":\"apiextensions.k8s.io\",\"kind\":\"CustomResourceDefinition\",\"name\":\"olms.operator.openshift.io\"}}",
  "reason": "IssuesFound",
  "status": "True",
  "type": "ReconciliationIssues"
}

    4.18.0-0.nightly-multi-2024-11-19-013314

    always

    1. Build a TP cluster 4.18.0-ec.3
    2. Upgrade it to 4.18.0-0.nightly-multi-2024-11-19-013314
    3.
    

    Upgrade stuck at customresourcedefinition \"olms.operator.openshift.io\"

    upgrade succeed

    Must-gather logs: https://gcsweb-qe-private-deck-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/qe-private-deck/logs/periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-4.18-upgrade-from-stable-4.18-azure-ipi-proxy-tp-arm-f60/1858786982260379648/artifacts/azure-ipi-proxy-tp-arm-f60/gather-must-gather/artifacts/