Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-1799

Use Previous Basis for Allowed Regressions

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • None
    • None
    • None
    • None

      When a regression is allowed in release N, release N+1 needs to query N-1 for the basis for those tests that were allowed to regress.

      Temporarily Bypassing a Regression

      Release N = 4.17 has allowed regression
      Release 4.18 queries basis from release 4.16 for the test(s)/variants for which 4.17 has allowed to regress.
      If the tests are allowed to regress for 4.18 as well then 4.19 will query 4.16, etc.

      https://github.com/openshift/sippy/pull/1436/files is an example for adding an allowed regression in the golang code.  This needs to be validated to ensure it works in the new variant / triage framework.

      TRT-1448 discusses creating a new framework outside of the golang code but may be outside the scope of this story based on priority.

       

      Part of the IntentionalRegression struct contains the Previous sample size, and pass percentage. We could ensure we have enough data to override the data for the N query when N is the basis and inject the values from the IntentionalRegression instead of needing to do additional queries.

              rh-ee-fbabcock Forrest Babcock
              rh-ee-fbabcock Forrest Babcock
              None
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: