Details
-
Bug
-
Resolution: Done
-
Critical
-
None
-
None
-
None
-
False
-
OCPSTRAT-343 - Onboarding New Providers/Platforms (Phase 2)
Description
USER STORY:
As a [type of user], I want [an action] so that [a benefit/a value].
DESCRIPTION:
There are some points on the plugin which is not creating the failed JUnit when exiting with failures, then the Sonobuoy will report the job wrongly as passed.
Steps to reproduce:
- Download the old CLI version (<=v0.1.0)
- Run the tool
Current Results:
- post-processor report as passed
$ ../openshift-provider-cert-linux-amd64-v0.1.0-devel status Thu, 24 Nov 2022 15:40:02 -03> Global Status: complete JOB_NAME | STATUS | RESULTS | PROGRESS | MESSAGE 10-openshift-kube-conformance | complete | passed | | Total tests processed: 1 (1 pass / 0 failed) 20-openshift-conformance-validated | complete | passed | | Total tests processed: 1 (1 pass / 0 failed) INFO[2022-11-24T15:40:02-03:00] The execution has completed! Use retrieve command to collect the results and share the archive with your Red Hat partner.
- plugin exited with errors
$ oc logs sonobuoy-05-openshift-cluster-upgrade-job-6c6e300a1f294433 -n openshift-provider-certification -c plugin./runner.sh:17> 20221124-174610> [runner] Setting config for PLUGIN_ID=[05]... #./global_fn.sh:68> [init_config] Unknow value for PLUGIN_ID=[05] #./runner.sh:17> 20221124-174610> [runner] Saving results triggered. Slowing down... (...) #./runner.sh:17> 20221124-174615> [runner] Looking for junit result files... ls: junit*.xml: No such file or directory #./runner.sh:17> 20221124-174615> [runner] Creating failed JUnit result file [/tmp/sonobuoy/results/junit_failed_e2e_20221124-174615.xml]
- Artifacts with default results (fallback junit)
$ cat results-oldplugin/plugins/10-openshift-kube-conformance/sonobuoy_results.yaml
name: 10-openshift-kube-conformance
status: passed
meta:
type: summary
items:
- name: junit_failed_e2e_20221124-175044.xml
status: passed
meta:
file: results/global/junit_failed_e2e_20221124-175044.xml
type: file
items:
- name: openshift-tests
status: passed
items:
- name: '[conformance] fallback error: possible that openshift-tests has crashed'
status: passed
Expected results:
It should follow the error from plugin. It can be done by creating a failed JUnit. The plugin already support to create it, but there are some points which is exiting without it.
The goal is to review the exit without failed JUnit files, and protect the plugin for running outdated CLI.
Errors something like:
$ ../openshift-provider-cert-linux-amd64-v0.1.0-devel run -w --dedicated INFO[2022-11-24T17:03:38-03:00] Waiting for post-processor... Thu, 24 Nov 2022 17:06:38 -03> Global Status: complete JOB_NAME | STATUS | RESULTS | PROGRESS | MESSAGE openshift-conformance-validated | complete | failed | | Total tests processed: 1 (0 pass / 1 failed) openshift-kube-conformance | complete | failed | | Total tests processed: 1 (0 pass / 1 failed) INFO[2022-11-24T17:06:38-03:00] The execution has completed! Use retrieve command to collect the results and share the archive with your Red Hat partner. $ ../openshift-provider-cert-linux-amd64-v0.1.0-devel results 202211242003_sonobuoy_b8ffc3c5-d10a-4295-8731-50c6301cc19b.tar.gz Plugin: openshift-conformance-validated Status: failed Total: 1 Passed: 0 Failed: 1 Skipped: 0Failed tests: [opct] [init_config] Wrong CLI version. Please update the openshift-provider-cert binary and try again [CERT_LEVEL=1] Plugin: openshift-kube-conformance Status: failed Total: 1 Passed: 0 Failed: 1 Skipped: 0Failed tests: [opct] [init_config] Wrong CLI version. Please update the openshift-provider-cert binary and try again [CERT_LEVEL=0]
{}Required:{}
...
{}Nice to have:{}
...
{}ACCEPTANCE CRITERIA:{}
{}ENGINEERING DETAILS:{}