Details
-
Spike
-
Resolution: Won't Do
-
Critical
-
None
-
None
-
False
-
-
False
Description
We need to evaluate whether there is any benefit to the results and validation environment when running the suites in serial mode.
why? We are seeing a slightly high number of failures in 'know platforms' such as AWS, even checking the existing flakes. We need to check if the parallel is impacting the results.
Goal:
- Run OPCT on AWS (Platform=None) with default execution (parallel)
- Run OPCT on AWS (Platform=None) with both suites (kubernetes/conformance and openshift/conformance) in serial mode
Collect the results of both and measure the differences, focusing on:
- failure ratio
- plugin execution time and total execution time