XML

Word

Printable

Epic Name:
storage-no-retries
Acceptance Criteria:
Hide

for the 4.18 release, the failure rate (the number of times a PR fails Tier 1 tests before it is merged) must be less than 5%

This is the number of times a CI run failed divided by the total number of CI executions

Only the last code change to a PR should be evaluated. The assumption is that if the code was changed there were bugs that were being addressed.

no-ux

no-doc
Show
for the 4.18 release, the failure rate (the number of times a PR fails Tier 1 tests before it is merged) must be less than 5% This is the number of times a CI run failed divided by the total number of CI executions Only the last code change to a PR should be evaluated. The assumption is that if the code was changed there were bugs that were being addressed. no-ux no-doc
Current Status:
Yellow
Feature Link:
CNV-38613 - 2024 Code Quality
Parent Link:
CNV-386132024 Code Quality
Hierarchy Progress Bar:

63% To Do, 13% In Progress, 25% Done
Ready-Ready:

dev-ready, doc-ready, po-ready, qe-ready, ux-ready
Status Summary:

Hide

2024-11-04:
Lanes are still failing > 5%. Team continues to work on this...

Show
2024-11-04: Lanes are still failing > 5%. Team continues to work on this...

Goal

sig-storage should always pass when testing good merged code on kubevirtci

User Stories

As a KubeVirt developer, I want to trust my tests. When they fail, I need to know that I introduced a bug. I must not assume that "this may be a flake".
- As of 2024-06-19, out of 31 PRs that were merged in the recent week, only 13 merged cleanly. The rest (18) required a /retry to pass. As a KubeVirt developer I would accept a failure rate of 5%, but no more.
As an OpenShift Virtualization developer, I would like to see my tests passing in downstream CI without any hiccup.
As an OpenShift Virtualization admin, I would like to receive weekly updates of my operator. I would like it to pass continuous integration smoothly and without human intervention - no failure analysis and no manual wavers.

List of things not included in this epic, to alleviate any doubt raised during the grooming process.

Storage tests have a lot of dependencies on which storage class is being used. We need to audit the test code for runtime/conditional skips and replace them with errors. Then we should label tests according to what storage features they depend on. Test lanes should then select tests to run by specifying the correct set of labels based on the storage configuration being tested.

clones

CNV-43408 Network: non-failing T1 lanes