Establish GitOps product Quality baseline

Story (Required)

As a software quality engineer trying to validate a GitOps operator release I want to be able to determine whether we can ship live a build with confidence and warrantees and whether we should reject it or at least be opposed to it.

Background (Required)

Right now we rely on Kuttl tests for the minimum acceptance process, prioritizing the check of new features, but the rest of the validations (like Webconsole, ArgoCD E2E, different upgrade methods...) are up to each QE, resulting in inconsistent criteria in every one of the releases. Moreover, some other important and critical things are missing (such as checking the container image tags, bundle sanity... etc).

Approach (Required)

As stakeholders of the product quality we need to come up with a document (or confluence page) with concise and concrete criteria for validating a Release Candidate, preferably in a checklist format. Those criteria have to be customer focus, which means we must put ourselves _in the shoes _of our clients, testing the product in the exact same way in which they install, update, and use it in their respective real production scenarios, instead of focusing in the changes of the codebase as we've been doing until now.

Acceptance Criteria

1. The QE team has a document that you can refer to as a reference whenever you need to verify what they need to check in each release candidate to confidently approve the new bundle.
2. The document includes each and every one of the elements that need to be tested, which means that if something is not covered in the document, it does not affect or block a release, so it can be ignored

Done Checklist

• The document has been created and uploaded to some common place where every person in the GitOps team (Developers, QE, PM...) is able to open and read it.
• Acceptance criteria are met