Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-34972

PerformaneProfile Reconciles/Updates Every ~15 Minutes

XMLWordPrintable

    • No
    • CNF Compute Sprint 255, CNF Compute Sprint 256, CNF Compute Sprint 257, CNF Compute Sprint 258, CNF Compute Sprint 259, CNF Compute Sprint 260
    • 6
    • False
    • Hide

      None

      Show
      None

      Description of problem:

          After deploying a node with ZTP and applying a PerformanceProfile, the PerformanceProfile continually transitions between conditions (Available, Upgradeable, Progressing, Degraded) and the cluster tuning operator logs show a cycle of Reconciling/Updating the profile every 15 minutes or so.
      
      No apparent impact to the cluster, but it is generating a lot of noise and concern with one of our partners

      Version-Release number of selected component (if applicable):

      Observed in 4.14.5 and 4.14.25    

      How reproducible:

          Always

      Steps to Reproduce:

          1.Deploy a node via ZTP
          2.Apply a performanceprofile via ACM/policies    

      Actual results:

          PerformanceProfile is applied, but logs show repeated reconcile/update attempts, generating noise in the logs

      Expected results:

          PerformaneProfile is applied and reconciled, but without the Updates and state transitions.

      Additional info:

        Logs show that the MachineConfig for the perf profile is getting updated every 15 minutes, but nothing has been changed i.e. no change in the applied PerformanceProfile:
      
      I0605 18:52:08.786257       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:08.786568       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:08.786899       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:08.788604       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      I0605 18:52:08.812015       1 status.go:83] Updating the performance profile "openshift-node-performance-profile" status
      I0605 18:52:08.823836       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:08.824049       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:08.824994       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:08.826478       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      I0605 18:52:29.069218       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:29.069349       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:29.069571       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:29.074617       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      I0605 18:52:29.088866       1 status.go:83] Updating the performance profile "openshift-node-performance-profile" status
      I0605 18:52:29.096390       1 performanceprofile_controller.go:390] Reconciling PerformanceProfile
      I0605 18:52:29.096506       1 resources.go:41] checking staged MachineConfig "rendered-master-478a5ff5b6e20bdac08368b380ae69c8"
      I0605 18:52:29.096834       1 performanceprofile_controller.go:461] using "crun" as high-performance runtime class container-runtime for profile "openshift-node-performance-profile"
      I0605 18:52:29.097912       1 resources.go:109] Update machine-config "50-performance-openshift-node-performance-profile"
      
      # oc get performanceprofile -o yaml
      <...snip...>
        status:
          conditions:
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            message: cgroup=v1;
            status: "True"
            type: Available
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            status: "True"
            type: Upgradeable
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            status: "False"
            type: Progressing
          - lastHeartbeatTime: "2024-06-05T19:09:08Z"
            lastTransitionTime: "2024-06-05T19:09:08Z"
            status: "False"
            type: Degraded
      

            rh-ee-rbaturov Ronny Baturov
            dcritch1@redhat.com David Critch
            Mallapadi Niranjan Mallapadi Niranjan
            Francesco Romani
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: