-
Epic
-
Resolution: Unresolved
-
Normal
-
None
-
Replace "waived" flag with "known issues"
-
False
-
-
False
-
To Do
-
rhel-arr-cki
At the moment kpet and kpet-db support specifying a (conditional) flag called "waived". That flag is exposed via Beaker job XML as an environment variable (e.g. `<param name="CKI_WAIVED" value="True"/>`) for each test task. The upt outputs this as KCIDB's `waived` property for tests. Consequently, both the reporter and the Data Warehouse take it into account when summarizing the testing results, ignoring failures when it is set.
The flag was introduced into kpet-db before there was a Data Warehouse, let alone the concept of "known issues" in it. Now that we have them, we can actually just use them to do the job of the "waived" flag. I.e. masking failing, generally-unreliable (e.g. newly-introduced) tests with precision on par with what kpet-db provides.
For this to work well in practice, the Data Warehouse would need to support the workflow of replacing general-pattern issues with specific-pattern issues. This is needed when going from a recently-introduced test, "waived" with a corresponding wildcard issue, to a generally-working test with only a few exceptions covered by specific issues.
The essential part would be the ability to show a time span of test results matching the more-general issue, and not matching the more-specific issues. The absence of incorrect results in such output would signify that the more-general issue can be safely removed, and that the test can become effective and enter production. This output could potentially become a part of issue removal workflow in general, essentially showing what would happen if the issue is removed.
The overall plan of doing this would be:
- [x] Turn the below into issues
- [ ] Support the general-specific issue transition in Data Warehouse, described above (https://gitlab.com/cki-project/datawarehouse/-/issues/336)
- [ ] Add all the current waived flags from kpet-db as "known issues" in Data Warehouse (https://gitlab.com/redhat/centos-stream/tests/kernel/kpet-db/-/issues/150)
- [ ] Add the kselftest waived flag set in pipeline-definition as a "known issue" in Data Warehouse (https://gitlab.com/cki-project/pipeline-definition/-/issues/216)
- [ ] Remove all the waived flags from cases in kpet-db (https://gitlab.com/redhat/centos-stream/tests/kernel/kpet-db/-/issues/150)
- [ ] Remove the setting of the kselftest waived flag from pipeline-definition (https://gitlab.com/cki-project/pipeline-definition/-/issues/216)
- [ ] Remove all mentions of the waived flag from the documentation website (https://gitlab.com/cki-project/documentation/-/issues/80)
- [ ] Do not use the waived flag exposed by kpet in kpet-db Jinja2 templates, and instead hardcode their output to False (https://gitlab.com/redhat/centos-stream/tests/kernel/kpet-db/-/issues/151)
- [ ] Remove support for the waived flag from kpet (optional, pending on HW enablement) (https://gitlab.com/cki-project/kpet/-/issues/64)
- [ ] Remove processing of the waived flag from Data Warehouse (https://gitlab.com/cki-project/datawarehouse/-/issues/338)
- [ ] Remove processing of the waived flag from the reporter (https://gitlab.com/cki-project/reporter/-/issues/66)
- [ ] Remove processing of the waived flag from upt (https://gitlab.com/cki-project/upt/-/issues/148)
- [ ] Remove processing of the waived flag from cki-tools (https://gitlab.com/cki-project/cki-tools/-/issues/112)
- [ ] Remove the waived flag from cki-lib test data (https://gitlab.com/cki-project/cki-lib/-/issues/44)
- [ ] Remove the (now-hardcoded) output of the waived flag from kpet-db (https://gitlab.com/redhat/centos-stream/tests/kernel/kpet-db/-/issues/152)
- [ ] Remove processing of the waived flag from whatever else we missed
Jira: CKI-7170