-
Sub-task
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
-
False
-
-
False
-
-
Extract metrics from CI executions to further, and quickly, be analysed to detect anomalies (OPCT-46)
Metrics we expected on the results for each execution:
- Total counters[1] for each result by plugin: Total/Passed/Skipped/Failed[...]
- Failure ratio (counter of the operation failed/total) by plugin
- Failure ratio after applying filters (recently added 'report' command)
- Total time to create the cluster*, run the tool, destroy the cluster*
ENGINEERING DETAILS
[1] Previous work / examples:
[2] WIP automation to automatically create AWS cluster [agnostic], run OPCT, and destroy: https://github.com/redhat-openshift-ecosystem/provider-certification-tool/pull/38