-
Epic
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
cnv-quarantine-hackathon
-
Quality / Stability / Reliability
-
77
-
False
-
-
False
-
None
-
Green
-
To Do
-
83% To Do, 17% In Progress, 0% Done
-
The stability of our test suite is critical for maintaining rapid, reliable development cycles and ensuring product quality. We are currently facing high noise and false positives in CI/CD due to an accumulation of flaky and persistently failing tests. This noise slows down developers, wastes CI resources, and can mask genuine regressions.
This hackathon is focused on proactively addressing this instability by enforcing our test quarantine process. The goal is not just to disable failing tests, but to analyze them and place them under a strict remediation plan to achieve a more signal-rich, reliable testing environment.
🎯 Hackathon Objectives
By the end of this event, participating teams must:
- Reduce CI Noise: Significantly lower the reported failure rate in the main branch by identifying and isolating persistent failures.
- Improve Test Health: Properly document the root cause for every failing test addressed.
- Propose Systemic Fixes: Generate actionable insights and proposals to prevent future test flakiness at the infrastructure or code level.
- Foster Knowledge Sharing: Ensure that the root cause analysis and eventual fix for each failure are documented and shared across the team to build collective resilience against similar issues.
- Streamline the Quarantine Process: Enable immediate merging of quarantine PRs (disabling the test) once the Test Failure Analysis (TFA) is complete, significantly reducing friction for isolating failed tests.
Non-Requirements
- List of things not included in this epic, to alleviate any doubt raised during the grooming process.
Notes
- Any additional details or decisions made/needed
- is related to
-
CNV-73836 Quarantine Hackathon Day
-
- Closed
-
- links to
1.
|
upstream roadmap issue |
|
Closed | |
Unassigned |
2.
|
upstream design |
|
Closed | |
Unassigned |
3.
|
upstream documentation |
|
Closed | |
Unassigned |
4.
|
upgrade consideration |
|
Closed | |
Unassigned |
5.
|
test plans in polarion |
|
Closed | |
Unassigned |
6.
|
automated tests |
|
Closed | |
Unassigned |
7.
|
downstream documentation merged |
|
Closed | |
Unassigned |
8.
|
CNV QE DevOps Requirement/Enablement |
|
Closed | |
Unassigned |