-
Story
-
Resolution: Done
-
Major
-
None
-
None
-
None
-
Future Sustainability
-
False
-
-
False
-
None
-
None
-
None
-
None
-
None
For the CI cost cutting effort, to effectively study what is killing jobs and what e2e retries within one job might do, we need to know what is killing jobs across e2es (which can be retried) and monitortests (which cannot).
There's a very loose convention that some monitortests include [invariant] in the test name, but only a fraction of them follow this. We could modify the monitortest framework where it collects junit results from each montest, to inject this tag if not already present.
Then we'd have a test rename problem in ci-mapping, unless Stephen B can propose something where it's handled automatically.
[monitortest] or [montest] would be nicer than [invariant], imo.
This effort would also help a little with the chaos from flake or fail feature in CR, which today picks up masses of invariant tests that permaflake violating new test pass rates, etc.