Description

Today we have release tests for ClusterTasks but similar tests when QE ran for ecosystem there are failure

Creating this story to add release tests for ecosystem tasks as well to avoid such issues in future

ref: https://github.com/openshift-pipelines/release-tests/tree/master/specs/clustertasks

Goals of this story are:

• Add tests for various tasks that are already present for clustertasks.
• Add tests to check for resolverTests in addon of tektonconfig.

Currently Added Tests

• #1: Test whether the tektonconfig addons will load ecosystem tasks in openshift-pipelines namespace when "resolverTasks" value is true.
• #2: Test whether on changing "resolverTasks" value does it affect any other tasks in the same namespace. 
• #3: Test to check buildah task.
• #4: Test to check buildah task in disconnected environment.
• #5: Test to check git-cli task.
• #6: Test to check oc task.
• #7: Test to check skopeo-copy task.
• #8: Test to check tkn task by listing pipelines.
• #9: Test to check tkn task with pac subcommand.
• #10: Test to check tkn task by checking its version.

How To Run Tests?

1. Setup the release-tests repo.

2. Setup atleast 1.16 Release of Operator.

3. Run the following command to run tests #1 & #2

     gauge run --log-level=debug --verbose --tags e2e specs/operator/addon.spec

4. Run the following command to run tests #3 to #10 excluding #4

     gauge run --log-level=debug --verbose --tags e2e specs/ecosystem/ecosystem.spec

5. To run #4 you need to setup a disconnected environment & then run the following command 

     gauge run --log-level=debug --verbose --tags disconnected-e2e specs/ecosystem/ecosystem.spec