Overview:

tl,dr: Goal is to

• test exactly the methods a customer would use (not our deployment scripts),
• speed up execution time of the test suite
• make the test suite logs more concise
• tune deployment mechanism to the requirements of the test suite, because that is the heart of what this test suite is executing.

The scanner-v4-install-tests in its current form highly depend on our deployment deployment machinery (tests/e2e/lib.sh, k8sbased.sh, etc.). This has several downsides:

This collection of deployment scripts has grown over the time to an enormous size with every potentially-useful configuration knob having custom support in these scripts, with most of the configuration state being propagated through environment variables with sometimes unclear semantics.

The scanner-v4-install-tests are different from the all other tests in that they don't test functionality of deployed applications (for which they need to be properly configured in terms of clusterresources), but their whole purpose is exactly to test the deployment method itself.

Therefore the requirements for this test suite are quite different:

• Our customers don't use our engineering deployment machinery, they use something like `helm` and `oc`/`kubectl` directly. We should essentially do the same by keeping the scripting layer above the actual installation invocation as thing as possible.
• Our deployment configuration does not need be configured so that the services have the resources for actual work, they can be adjusted to the use-case of testing if installation succeeds (in different scenarios).
• We can even leverage some tricks to save resources and waiting times which are not applicable for our users, for example configuring collector to only run on a single worker node instead of on all nodes of a cluster. This saves resources and waiting time and we still get exactly the results we need from the tests.
• The scanner-v4-install-tests suite is testing different installation scenarios, requiring installations/reinstallations of our components more than a dozen times. For being able to parse the logs of these tests easily it is helpful of to keep the output concise – the deployment scripts are overly verbose, making the output difficult to follow.

Acceptance Criteria:

• As a start, new deployment functions are introduced in the run-scanner-v4-install-tests suite for deploying the central-services and secured-cluster-services Helm charts directly with the `helm` CLI.
• These new functions apply resource configurations directly to make installation as snappy as possible. Only one collector and one admission-control pod is deployed.
• In the implementation any unnecessary waiting time is reduced without impacting test stability.
• The new functions are used in (at least) one test scenario for deploying Helm charts.