Overview

As a ci maintainer adding new tests to Origin or external binaries
I want risk analysis to report all test risks and new tests in an easy to reference location
So that I can clearly understand the potential impact of my changes.

Summary

Currently, risk analysis leaves a comment on PRs in Origin. It comments up to 20 risky and/or new tests respectively. This is useful, but it is pretty chatty and dirties up the PR with many comments (multiple per commit, for some reason?). We should utilize check runs to make this more user friendly. ci-tools has used these before. We can store more information there, and it will not be chatty on the PR itself.

Optionally, we could make the check_run blocking and fail if we find that a newly added test doesn't succeed. Or for any other high risk reason we see fit.

This will open the door to get this functionality into repos other than Origin, which will be very useful with the expansion of OTE.

Acceptance Criteria

• Risk analysis no longer has test list comments on PRs, instead it just links to the check_run
• Use check_runs instead to supply all of the risk details
• Consider marking the check_run as a failure if certain, high risk, criteria are met
• Consider making these checks blocking