Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-28388

Redundant reconciles by CCO's status controller

    XMLWordPrintable

Details

    • No
    • False
    • Hide

      None

      Show
      None
    • Release Note Not Required
    • In Progress

    Description

      Description of problem:

      The status controller of CCO reconciles 500+ times/h on average on a resting 6-node mint-mode OCP cluster on AWS.

      Steps to Reproduce:

      1. Install a 6-node mint-mode OCP cluster on AWS
2. Do nothing with it and wait for a couple of hours
3. Plot the following metric in the metrics dashboard of OCP console:
rate(controller_runtime_reconcile_total{controller="status"}[1h]) * 3600    

      Actual results:

      500+ reconciles/h on a resting cluster

      Expected results:

      12-50 reconciles/h on a resting cluster
Note: the reconcile() function always requeues after 5min so the theoretical minimum is 12 reconciles/h

      Attachments

        Issue Links

          blocks

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-29155 Redundant reconciles by CCO's status controller

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-29155 Redundant reconciles by CCO's status controller

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-29156 Redundant reconciles by CCO's status controller

          • Normal - To be worked after urgent and high priorities are resolved. This term is chosen when the severity of an issue is relatively close to the level of effort to fix it. The existence of an easily implemented workaround can also lead to this priority level instead of a higher priority.
          • Closed
          links to

          GitHub openshift/cloud-credential-operator#665: OCPBUGS-28388: Fix the ClusterOperator watch of the status controller

          Web Link RHEA-2024:0041 OpenShift Container Platform 4.16.z bug fix update

          Activity

            People

              fxierh Feilian Xie
              fxierh Feilian Xie
              Jianping Shu Jianping Shu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: