-
Story
-
Resolution: Done
-
Major
-
None
-
None
-
None
-
None
-
False
-
-
False
-
None
-
None
-
None
-
None
We have an issue where new tests often don't reliably produce success results, or have other issues like dynamic components in the test name (e.g. a namespace).
Could risk analysis be configured to report on all never-before-seen tests in presubmits? And maybe a link/reminder of our test standards (no dynamic names, must always produce a result, renames should be accounted for)
Benefits of just commenting the new test names:
- Developer might notice dynamic names in the list
- We can search github for the whole test name and find the PR easily that added it (or push it to bigquery?)
This would potentially let developers see the tests they're creating. We might also be able to apply heuristics to know that things aren't working correctly?
e.g. we only saw a test name once across all presubmits, or we saw failures but no successes, etc.
Outcomes:
- Risk analysis comment would include new tests to increase visability and searchability
- Risk analysis would mark the risk High if a new test failed as we don't want new failing tests to enter the system without validation
- is depended on by
-
TRT-1802 Risk analysis should detect new tests and mark failures as high risk
-
- Closed
-
- links to