Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-56805

cluster-capi-operator tries updating the clusteroperator status in a hot loop

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • No
    • None
    • CLOUD Sprint 271, CLOUD Sprint 272, CLOUD Sprint 273
    • 3
    • In Progress
    • Release Note Not Required
    • Covering together with OCPBUGS-58880
    • None
    • None
    • None
    • None

      Description of problem:

      At the moment the `cluster-capi-operator` manager is running in a hot loop trying to Update() the ClusterOperator status with the most recent conditions.

      E0527 19:07:44.303923       1 controller.go:316] "Reconciler error" err="failed to set conditions for InfraCluster controller: failed to sync status: failed to update cluster operator status: Operation cannot be fulfilled on clusteroperators.config.openshift.io \"cluster-api\": the object has been modified; please apply your changes to the latest version and try again" controller="InfraClusterController" controllerGroup="config.openshift.io" controllerKind="ClusterOperator" ClusterOperator="cluster-api" namespace="" name="cluster-api" reconcileID="434ea0cf-83d3-442c-bde6-1fedddf4ffef"
...
E0527 19:07:44.357363       1 controller.go:316] "Reconciler error" err="failed to set status available: failed to update cluster operator status: Operation cannot be fulfilled on clusteroperators.config.openshift.io \"cluster-api\": the object has been modified; please apply your changes to the latest version and try again" controller="CoreClusterController" controllerGroup="config.openshift.io" controllerKind="ClusterOperator" ClusterOperator="cluster-api" namespace="" name="cluster-api" reconcileID="9f4c2c13-ede1-4981-8761-988bfb4f66bb"
...
E0527 19:07:44.443030       1 controller.go:316] "Reconciler error" err="failed to set conditions for CAPI Installer Controller: failed to sync status: failed to update cluster operator status: Operation cannot be fulfilled on clusteroperators.config.openshift.io \"cluster-api\": the object has been modified; please apply your changes to the latest version and try again" controller="CapiInstallerController" controllerGroup="config.openshift.io" controllerKind="ClusterOperator" ClusterOperator="cluster-api" namespace="" name="cluster-api" reconcileID="eded91df-8801-4313-ab1e-321813387601"
```
(more logs [here](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-capi-operator/299/pull-ci-openshift-cluster-capi-operator-main-e2e-aws-capi-techpreview/1927428378592284672/artifacts/e2e-aws-capi-techpreview/gather-extra/artifacts/pods/openshift-cluster-api_cluster-capi-operator-84f4597b4b-2h2b8_cluster-capi-operator.log))

      This should instead be using a Patch() to do so to avoid conflicts on all the non relevant fields.

      We need a short term fix until we reintroduce a more refined status updating mechanism + SSA which was originally merged with: https://github.com/openshift/cluster-capi-operator/pull/256 but got reverted by https://github.com/openshift/cluster-capi-operator/pull/273

      Version-Release number of selected component (if applicable):

      How reproducible:

          Always

      Steps to Reproduce:

          1. Check the logs of the cluster-capi-operator's cluster-capi-operator container    

      Actual results:

          CO updates are erroring in a hot loop

      Expected results:

          No CO update errors in a loop

      Additional info:

              ddonati@redhat.com Damiano Donati
              ddonati@redhat.com Damiano Donati
              None
              None
              Zhaohua Sun Zhaohua Sun
              None
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: