Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-2366

Risk Analysis Should Flag New Long Running Tests

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • Future Sustainability
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • None
    • None
    • None

      In addition to flagging new tests in PRs, and noting when there are failures for those tests, we would like to extend that functionality to check how long the tests run and flag tests that run over a determined amount of time.

      This is part of an effort to keep our job run times from continually increasing. We want the users to be aware of the impact of the tests and consider if the tests can be refactored or isolated in different jobs / suites to not impact existing jobs.

      Junit output should include the test timings as well as e2e events.  We want to be able to analyze and comment on the test timings.  If it is longer than X (1 minute?) upgrade the risk to Medium and note it. If longer than Y (5 min?) upgrade the risk to High.

      When completed RA will be able to determine the time new tests take to complete and include an analysis of that time in the update.

              Unassigned Unassigned
              rh-ee-fbabcock Forrest Babcock
              None
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: