Uploaded image for project: 'Operator Runtime'
  1. Operator Runtime
  2. OPRUN-2342

[Spike] Investigate mechanisms to track historical CI failures for individual tests

    XMLWordPrintable

Details

    • Task
    • Resolution: Unresolved
    • Undefined
    • None
    • None
    • None
    • ABBA [OLM 209], Buzz Lightyear [OLM 210], Corundum [OLM 211], [OLM 212]Digimon, [OLM 213] Ectoplasm
    • 0

    Description

      Currently the upstream project has no historical knowledge of CI pass rates, both for individual test cases as well as in aggregate. There are a number of flaky tests that today pull request authors simply retry until all tests turn green, but historical trends should allow us to drill down into specific flaky tests much sooner as well as alert us to when a test that normally does not flake starts to in an individual pull request.

      AC:

      • Investigate mechanisms to track historical aggregate CI test run data. To start, we should do this against master to build some confidence in our current state. Then, later, we can add these metrics to open pull requests for comparison data.
      • Record any findings made during the sprint
      • Determine next steps (tasks/stories/etc.)

      In addition to aggregate CI success, also track historical pprof data from test runs.

      Attachments

        Activity

          People

            Unassigned Unassigned
            krizza@redhat.com Kevin Rizza
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: