Uploaded image for project: 'Machine Config Operator'
  1. Machine Config Operator
  2. MCO-1907

Evaluate if mco-disruptive suites are ready for MCO component readiness

XMLWordPrintable

    • Icon: Spike Spike
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • None
    • 5
    • False
    • Hide

      None

      Show
      None
    • False
    • MCO Sprint 279
    • 0

      This will require investigating the performance of our new disruptive suites to see if they're ready to contribute to MCO's component readiness. Currently, these suites are classified as candidate(MCO-1858) and do not appear in the default MCO CR view

      Here is an exhaustive list of our 4.21 disruptive suites:

      Platform/Variant Default TechPreview
      AWS History History
      GCP History History
      vsphere History History
      Azure History History
      metal-ipi-ovn-ipv4 History History
      metal-ipi-ovn-ipv6 History History
      metal-ipi-ovn-dualstack History History

      Note that there are 4.19 & 4.20 variants of these, and they will require investigation too.

      This card can be considered complete when:

      • We will likely need TRT's blessing in this process; so we should determine what job metrics they expect before they'll permit a promotion to standard. Per Jerry's convo with TRT, there are no specific numbers, its up to the MCO when we feel ready. Of course, a sub par pass rate will result in TRT asking us to downgrade the tests. 
      • For 4.19-4.21, determine which jobs are performing well, and open bugs for the problematic ones.
      • If certain jobs are more difficult to fix than others, we can selectively exclude them from component readiness, but we should do our best to fixup as much as we can so the MCO's CR is accurate to TRT.
      • Once we're happy with the lists of tests we want to include/exclude, we should update https://github.com/openshift/sippy/blob/main/pkg/variantregistry/ocp.go#L658 and set those to standard.

      Additional history:

      These 14 suites were originally added in https://github.com/openshift/release/pull/64752 for generating signal for CR and feature promotion purposes. We then did a follow-up update https://github.com/openshift/release/pull/66807 to change their frequency.

      Unfortunately this frequency bump for the techpreview does not seem to be enough, as the timing window for the sippy test performance search and the job start window don't exactly match. I'm open to ideas to fix this; but perhaps gangway is fine for the moment if we're just trying to get the extra run or two to get the feature over the promotion threshold. 

              djoshy David Joshy
              djoshy David Joshy
              Isabella Janssen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: