Uploaded image for project: 'CKI Project'
  1. CKI Project
  2. CKI-7170

Replace "waived" flag with "known issues"

XMLWordPrintable

      At the moment kpet and kpet-db support specifying a (conditional) flag called "waived". That flag is exposed via Beaker job XML as an environment variable (e.g. `<param name="CKI_WAIVED" value="True"/>`) for each test task. The upt outputs this as KCIDB's `waived` property for tests. Consequently, both the reporter and the Data Warehouse take it into account when summarizing the testing results, ignoring failures when it is set.

      The flag was introduced into kpet-db before there was a Data Warehouse, let alone the concept of "known issues" in it. Now that we have them, we can actually just use them to do the job of the "waived" flag. I.e. masking failing, generally-unreliable (e.g. newly-introduced) tests with precision on par with what kpet-db provides.

      For this to work well in practice, the Data Warehouse would need to support the workflow of replacing general-pattern issues with specific-pattern issues. This is needed when going from a recently-introduced test, "waived" with a corresponding wildcard issue, to a generally-working test with only a few exceptions covered by specific issues.

      The essential part would be the ability to show a time span of test results matching the more-general issue, and not matching the more-specific issues. The absence of incorrect results in such output would signify that the more-general issue can be safely removed, and that the test can become effective and enter production. This output could potentially become a part of issue removal workflow in general, essentially showing what would happen if the issue is removed.

      The overall plan of doing this would be:

      Jira: CKI-7170

              Unassigned Unassigned
              rh-ee-tdaapare Tales Lelo da Aparecida
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: