Uploaded image for project: 'Red Hat Internal Developer Platform'
  1. Red Hat Internal Developer Platform
  2. RHIDP-12294

[QE] Implement flake quarantine system with skip list and CI Medic rotation

    • Icon: Task Task
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • Quality

      Task

      Build a flake quarantine mechanism that auto-skips tests after 3 consecutive flakes. Currently no structured process exists.

      Background

      Implement:

      1. quarantine.json file in e2e-tests/ with metadata (test path, Jira link, date, expiry)
      2. Playwright integration that reads quarantine list and applies test.skip()
      3. Quarantined tests still run in nightly via dedicated showcase-quarantine project (no gating)
      4. CI Medic weekly rotation documented in e2e-tests/README.md

      Process:

      • Detection: automatic via ReportPortal
      • Quarantine: auto-skip after 3 consecutive flakes
      • Tracking: Jira issue created per flake
      • Resolution: CI Medic rotation investigates
      • Reinstatement: test returns after fix verified
      • Auto-expire: quarantine expires after 2 weeks

      Acceptance Criteria

      • quarantine.json schema defined and documented
      • Playwright reads quarantine list and skips listed tests in gating projects
      • Quarantined tests run in nightly showcase-quarantine project
      • CI Medic rotation documented with duties and schedule
      • At least one flaky test quarantined as proof of concept

              Unassigned Unassigned
              gustavolira Gustavo Lira Silva
              RHDH Cope
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: