We need to account for flake tests, understand the root cause and, if needed, create action items for each one.

The suite execution is failing due to flaky tests for monitoring:

• status

```
Mon Mar 28 23:54:48 -03 2022> Sonobuoy has finished.
PLUGIN STATUS RESULT COUNT PROGRESS
openshift-provider-cert-level1 complete failed 1 50/50 (0 failures)
openshift-provider-cert-level2 complete failed 1 17/17 (0 failures)
openshift-provider-cert-level3 complete passed 1 0/0 (0 failures)

Sonobuoy plugins have completed. Preparing results for download.

```

• review test name on result artifacts:

```bash
./report.sh

(....)
Getting tests count by status for each Level:
#> openshift-provider-cert-level1 [passed]:
10
#> openshift-provider-cert-level1 [skipped]:
42
#> openshift-provider-cert-level1 [failed]:
1
#> openshift-provider-cert-level2 [passed]:
19

Getting 'failed' test names by Level:

#> openshift-provider-cert-level1 [failed]:
[sig-arch] Monitor cluster while tests execute

#> openshift-provider-cert-level2 [failed]:
[sig-arch] Monitor cluster while tests execute

```

1. 1. Suggestions:

• create a post-processor script to review JUnit files for {}known flaky{} tests
• Some ideas raised in Cloud team meetings: Run tests serially, improve flakiness, parallelize later.
• Need to make sure that the tool with the final set of tests by Level will pass (report certified) in an existing integration (AWS for example).