Uploaded image for project: 'Red Hat OpenShift Control Planes'
  1. Red Hat OpenShift Control Planes
  2. CNTRLPLANE-1702

Track control plane rollout time with dedicated e2e test check

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • False
    • None

      User Story

      As a HyperShift platform engineer, I want to independently measure and track control plane rollout time in e2e tests, so that we can establish SLOs for HCP readiness and compare performance metrics between ROSA HCP and ARO HCP.

      Background

      Currently, there is no SLO or e2e test that measures:

      • HCP is ready within 5 minutes
      • HostedCluster should be complete within xyz minutes

      This information is needed by the Microsoft team to compare ARO HCP and ROSA HCP performance metrics.

      The metric should capture the delta between HC creationTimestamp and the Available condition lastTransition to true.

      Currently, once the kube-apiserver is up in e2e tests, we wait until the cluster fully rolls out, but we do not independently wait for the CP (control plane) to roll out. The CP rollout is a subset of the full cluster rollout, which includes nodes joining and all CVO reporting complete.

      Acceptance Criteria

      • Given a HostedCluster is being created in an e2e test, when the control plane rolls out, then the test independently measures and records the delta between HC creationTimestamp and the Available condition lastTransition to true
      • Verify that WaitForControlPlaneComponentRollout() is called before the full cluster rollout check executes
      • Test that the control plane rollout check executes after the kube-apiserver is up but before waiting for the full cluster rollout (nodes joining + CVO reporting complete)
      • Demonstrate that the control plane readiness metric can be used to establish SLOs (e.g., HCP ready within 5 minutes)
      • Demonstrate that metrics can be collected and compared between ROSA HCP and ARO HCP deployments

      Engineering Notes

      A function WaitForControlPlaneComponentRollout() already exists in test/e2e/util/util.go:602 that waits for control plane components to complete rollout independently.

      Two implementation options:

      • Option 1 (Easier): Add WaitForControlPlaneComponentRollout() at the beginning of WaitForImageRollout() function
      • Option 2 (More explicit): Call WaitForControlPlaneComponentRollout() before each WaitForImageRollout() call in the test suite

      Recommended approach: Option 1 - Add the call at the beginning of WaitForImageRollout() as it is easier to implement and ensures all tests automatically get the control plane rollout check.

              hypershift-automation hypershift-team automation
              rh-ee-brcox Bryan Cox
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: