Problem:

Today we only have sunny day scenario test cases defined in our flow. We need to cover some error paths that will make sure we fail gracefully and consistently. We also want to make sure the flow fails on security checks as expected to make sure there is no intrusion. We also do not have any test cases for community path.

Goal:

1. To increase our coverage of test cases in error paths
2. To increase our coverage of test cases for community charts.

Why is it important?

By increasing this coverage we harden our framework and decrease the risk of regression as we continue to enhance the workflows and the chart verifier checks.

Use cases

These test cases will set the scope of this epic to make sure we know when we are done

1. Check developer not listed in owners cannot submit a chart. Make sure the access message is clear in PR.
2. Check the appropriate message is displayed in PR when the chart version and the version in project path do not match.
3. Check that partner charts are not merged if there is chart verifier checks that fail
4. Check that redhat charts are merged even when there are chart verifier checks that fail
5. Check that community charts never merge even when all chart verifier checks pass.
6. Check when a chart tarball is submitted with a report and the sha in the report does not match the chart an appropriate message is output in the PR.
7. Check when a PR includes a file which is not part of a chart an appropriate message is output in the PR.
8. Check when a report includes a invalid open shift version annotation an appropriate message is output in the PR.
9. Check if a report only PR includes a chart link which is not a valid url an appropriate message is output in the PR.
10. Check when a PR includes a report which does not include a full set of checks an appropriate message is output in the PR.
11. Check when a PR includes a report in json format an appropriate message is output in the PR.
12. Check when a PR includes and chart with no kubeVersion, It should fail for partners but pass for redhat and community. Community will need to do a force-publish

Acceptance criteria

1. All scenarios described above are covered for partners, redhat, and community charts.

Dependencies (External/Internal)

BDD Framework already in place

Design Artifacts

Exploration

Note