Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-34972

PerformaneProfile Reconciles/Updates Every ~15 Minutes

XMLWordPrintable

    • No
    • CNF Compute Sprint 255, CNF Compute Sprint 256, CNF Compute Sprint 257, CNF Compute Sprint 258, CNF Compute Sprint 259, CNF Compute Sprint 260, CNF Compute Sprint 261, CNF Compute Sprint 262, CNF Compute Sprint 263, CNF Compute Sprint 264, CNF Compute Sprint 265, CNF Compute Sprint 266
    • 12
    • False
    • Hide

      None

      Show
      None

      Description of problem:

          After deploying a node with ZTP and applying a PerformanceProfile, the PerformanceProfile continually transitions between conditions (Available, Upgradeable, Progressing, Degraded) and the cluster tuning operator logs show a cycle of Reconciling/Updating the profile every 15 minutes or so.
      
      No apparent impact to the cluster, but it is generating a lot of noise and concern with one of our partners

      Version-Release number of selected component (if applicable):

      Observed in 4.14.5 and 4.14.25    

      How reproducible:

          Always

      Steps to Reproduce:

          1.Deploy a node via ZTP
          2.Apply a performanceprofile via ACM/policies    

      Actual results:

          PerformanceProfile is applied, but logs show repeated reconcile/update attempts, generating noise in the logs

      Expected results:

          PerformaneProfile is applied and reconciled, but without the Updates and state transitions.

      Additional info:

        Logs show that the MachineConfig for the perf profile is getting updated every 15 minutes, but nothing has been changed i.e. no change in the applied PerformanceProfile:
      
      I0605 18:52:08.786257       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:08.786568       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:08.786899       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:08.788604       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      I0605 18:52:08.812015       1 status.go:83] Updating the performance profile "openshift-node-performance-profile" status
      I0605 18:52:08.823836       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:08.824049       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:08.824994       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:08.826478       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      I0605 18:52:29.069218       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:29.069349       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:29.069571       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:29.074617       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      I0605 18:52:29.088866       1 status.go:83] Updating the performance profile "openshift-node-performance-profile" status
      I0605 18:52:29.096390       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:29.096506       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:29.096834       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:29.097912       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      
      # oc get performanceprofile -o yaml
      <...snip...>
        status:
          conditions:
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            message: cgroup=v1;
            status: "True"
            type: Available
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            status: "True"
            type: Upgradeable
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            status: "False"
            type: Progressing
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            status: "False"
            type: Degraded
      

              rh-ee-rbaturov Ronny Baturov
              dcritch1@redhat.com David Critch
              Mallapadi Niranjan Mallapadi Niranjan
              Francesco Romani
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: