Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-2323

Summarize a test details report with AI for users of component readiness

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • None
    • Future Sustainability
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • None
    • None
    • None

      With the hateoas PR landing, we now have regression IDs we can give to sippy chat, and it can figure out how to query this regression and get to the test details report for the regression. The AI should then provide a summary to the user by analyzing the details on the report:

      • Pull the test failures for each failed job in the report.
      • Analyze those failures for the commonalities and differences in the failure patterns. Are these mass failure jobs. What tests fail in the jobs, is it just this one or is there a common set?
      • Detect tests that lost the ability to flake and highlight this for the user. (basis has mostly flakes, sample has none and failures) Explain why test retries are being removed and are dangerous. Show the flake rate vs the fail rate so it's easy to see the delta.
      • Is the regression triaged, how many regressions are related, what test names are involved.
      • Highlight if the regression component does not match a triaged jira component, is this actually a bug elsewhere?
      • Consider using this for the actual jira text in the future, or as part of this feature.
      • Highlight the length of time the regression is open, how long ago the jira was updated, when the last failure was, is it looking like it may be fixed and rolling off after a period of full failing and now full green.
      • Inform the user what the status of the traige means, believed fixed vs failed fix vs untriaged.

              stbenjam Stephen Benjamin
              rhn-engineering-dgoodwin Devan Goodwin
              None
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: