Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-68870

Improve openshift-virtualization-tests failure management

XMLWordPrintable

    • Icon: Epic Epic
    • Resolution: Unresolved
    • Icon: Critical Critical
    • CNV v4.21.0
    • None
    • None
    • improve-openshift-virtualization-tests-failure-management
    • Quality / Stability / Reliability
    • 77
    • Hide
      1. Document the workflow for Enhanced Failure Management and Quarantine Strategy
      2. An volunteer SIG group to implement the workflow to stablize the Tier2 test, all test lane pass rate to 95%+
      3. Get agreement for the workflow from cnv-leads.
      4. Announcement workflow
      5. Phase 2 EPICs creation to implement the workflow for other SIG group. 
      Show
      Document the workflow for Enhanced Failure Management and Quarantine Strategy An volunteer SIG group to implement the workflow to stablize the Tier2 test, all test lane pass rate to 95%+ Get agreement for the workflow from cnv-leads. Announcement workflow Phase 2 EPICs creation to implement the workflow for other SIG group. 
    • To Do
    • 100% To Do, 0% In Progress, 0% Done
    • Hide

      2025-09-01:
      will try to have a smaller PoC with IUO and storage teams. ...

      Show
      2025-09-01: will try to have a smaller PoC with IUO and storage teams. ...

      Goal

      To ensure a dependable test suite in https://github.com/RedHatQE/openshift-virtualization-tests, we must prioritize detecting flaky tests while keeping the test lanes GREEN.

      Unreliable tests lead to mistrust in test results, forcing the QE to spend extra time determining if failures are actual bugs or just test instability.

      The goal of this proposal is to enhance our testing efficiency and reliability while maintaining full test coverage and ensuring high product quality.

      User Stories

      • As QE, we are going to have the workflow
        • how we make the test lanes GREEN 
        • Daily Review of Failing Tests
        • Initial Assessment and Triage
        • Flake Identification
        • Failure Reporting
        • Quarantine Decision
        • Quarantine Implementation
        • Monitoring Quarantined Tests
        • Dequarantine workflow
      • SIG group to implement the workflow with maintainer, all lanes pass rate to 95%.
      • Announcement workflow

      Non-Requirements

      • List of things not included in this epic, to alleviate any doubt raised during the grooming process.

      Notes

      • Any additional details or decisions made/needed

          1.
          		 upstream roadmap issue Sub-task New Undefined Unassigned
          2.
          		 upstream design Sub-task New Undefined Unassigned
          3.
          		 upstream documentation Sub-task New Undefined Unassigned
          4.
          		 upgrade consideration Sub-task New Undefined Unassigned
          5.
          		 test plans in polarion Sub-task New Undefined Unassigned
          6.
          		 automated tests Sub-task New Undefined Unassigned
          7.
          		 downstream documentation merged Sub-task New Undefined Unassigned
          8.
          		 CNV QE DevOps Requirement/Enablement Sub-task New Undefined Unassigned

              ruth.netser1 Ruth Netser
              ycui@redhat.com Ying Cui
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: