XML

Word

Printable

Epic Name:
virt-no-retries
Story Points:
77
Acceptance Criteria:
Hide

For the 4.18 release, the failure rate (the number of times a PR fails Tier 1 tests before it is merged) must be less than 5%

This is the number of times a CI run failed divided by the total number of CI executions

Only the last code change to a PR should be evaluated. The assumption is that if the code was changed there were bugs that were being addressed.

Gating Tests should always PASS.
Show
For the 4.18 release, the failure rate (the number of times a PR fails Tier 1 tests before it is merged) must be less than 5% This is the number of times a CI run failed divided by the total number of CI executions Only the last code change to a PR should be evaluated. The assumption is that if the code was changed there were bugs that were being addressed. Gating Tests should always PASS.
Epic Status:
To Do
Feature Link:
CNV-38613 - 2024 Code Quality
Parent Link:
CNV-386132024 Code Quality
Hierarchy Progress Bar:

71% To Do, 29% In Progress, 0% Done
Ready-Ready:

dev-ready, doc-ready, po-ready, qe-ready, ux-ready

Goal

sig-compute should always pass when testing good merged code on kubevirtci.
- Identify and Prioritize High Priority Virt T1 Tests for Downstream Builds.
  - Use the "Conformance" label to select the Virt T1 Tests for Gating.
- Remove and/or Rewrite the tests after reviewing the tests.
Ensure the Non Productized Tests from Tier1 Jobs are skipped.
Create Full T1 Scheduled job runs for kubevirt-compute-ocs.
Update existing jobs to pick the tests labeled with Conformance.

Discussion Doc:

As a KubeVirt developer, I want to trust my tests. When they fail, I need to know that I introduced a bug. I must not assume that "this may be a flake".
- As of 2024-06-19, out of 31 PRs that were merged in the recent week, only 13 merged cleanly. The rest (18) required a /retry to pass. As a KubeVirt developer I would accept a failure rate of 5%, but no more.
As an OpenShift Virtualization developer, I would like to see my tests passing in downstream CI without any hiccup.
As an OpenShift Virtualization admin, I would like to receive weekly updates of my operator. I would like it to pass continuous integration smoothly and without human intervention - no failure analysis and no manual wavers.

Non-Requirements

List of things not included in this epic, to alleviate any doubt raised during the grooming process.

blocks

CNV-38704 Quicker test suites

is cloned by

CNV-43408 Network: non-failing T1 lanes

is duplicated by

CNV-46715 Stabilize Virt Tier1 Gating Tests

relates to

CNV-42657 refactor tests/operator/operator.go

1.	upstream roadmap issue	New	Unassigned
2.	upstream design	New	Unassigned
3.	upstream documentation	New	Unassigned
4.	upgrade consideration	New	Unassigned
5.	CEE/PX summary presentation	New	Unassigned
6.	test plans in polarion	New	Unassigned
7.	automated tests	New	Unassigned
8.	downstream documentation merged	New	Unassigned