Goal

To have reproducible/repeatable/reliable builds/runs of upstream and downstream typescript tests and adhere to best practices in given area (npm packages, CI/CD)

Current situation

Upstream

Typescript tests

Everytime the tests are being run in CI, all the dependencies are downloaded again using `npm install`, thus not respecting the reproducibility of the build in CI.

This npm project is not being published anywhere (in npm registry) for our downstream to properly “consume” it.

Container image with tests

Container image (quay.io/eclipse/che-e2e) is being build&tagged together with all the other Che container images. That means we have image version tagged with every Che release. In addition we have `latest` tag pointing to latest released image & `next` one.

Source code versioning

The whole eclipse/che repository gets tagged&branched as part of Che release process.

Downstream

Typescript tests

Typescript tests in downstream directly depend on upstream tests/package. For this dependency, custom bash scripts were created to target this. These scripts are able to install any specified version (by tag/branch/commit) of upstream tests/package by cloning the upstream repo, installing upstream test dependencies (`npm install`) and then hooking them as dependency for downstream tests.

Container image with tests

• quay.io/crw/e2e-tests:latest is being built on each push into e2e-codeready/latest branch in CI;
• quay.io/crw/e2e-tests:main is being built on each push into e2e-codeready/main branch in CI;
• quay.io/crw/e2e-tests:2.x.y image is built and pushed manually after GA version of CRW is released and it’s tagged by the CRW version. 

Upstream dependencies inside this container are provided during build time by script described in Downstream->Typescript tests section.

Source code versioning

Branching and tagging is being done manually after GA version of CRW is released

Desired state

Upstream

Typescript tests

• We should properly version the upstream test package. The version should ideally correspond to the Che version.
• For any CI/CI purposes (building&publishing, running tests in CI, …) we should not use `npm install`. Instead we should use `npm ci`, as this creates a reproducible dependency tree and solves few more issues.
• We should publish the package under eclipse-che org/user in npmjs.com
• Build next package on each push into main branch by doing something like this: https://github.com/che-incubator/vscode-theia-comparator/blob/main/.travis.yml#L25
• For the next package we could also consider publishing with `npm publish --tag next`
• We need to do "npm audit --fix" while e2e-test image.

• Properly versioned package as part of the Che release process

Container image with tests

• As mentioned in previous paragraph, build of this image should use `npm ci` instead of `npm install`
• Otherwise I think we are good here

Source code versioning

Nothing to do in this regard

Downstream

Typesciript tests

• Dependency to upstream package should be handled the typical way - by specifying the dependency package in package.json 
• Because upstream package will be published on each push into main branch and as well as with Che releases, we should have the same variability/possibilities of selecting which version of upstream tests we want to depend on.
• Any CI (building image, running testing jobs, …) using our downstream tests should use `npm ci` instead of `npm install` for installing dependencies.

Container image with tests

• Build of this container image should use `npm ci` for installing dependencies
• As the upstream and downstream container image shouldn’t be much different, we can also think here a bit about somehow consolidating these two images into one? Or maybe have one base image and then just a lightweight upstream&downstream ones to improve the maintainability (now we have to do all the changes twice, once in upstream, once in downstream)
• Build of this image should happen:
• Next - from main branch and should be tagged by commit & `next` tag
• After CRW is GA’d.
• This job, which will be building it, should be triggered by UMB message about promotion to production
• Link latest branch with "x" bug fix version, not particular bug fix version, e.g. 7.40.x, instead of 7.40.0.
• We need to do "npm audit --fix" while update e2e-test image.

Source code versioning

• Branching and versioning (package.json etc.) should be done automatically and should be triggered by the same message as build of e2e-tests image.

Breakdown into issues:

• [upstream] Have proper versioning of e2e npm package #18771
• [upstream] Publish che-e2e npm package into public registry #18772
• [upstream] Stop using npm-install in CI. Use npm-ci instead #18773
• [downstream] Switch dependency management from custom scripts to npm based. https://issues.redhat.com/browse/CRW-1470
• [downstream] Rewrite all existing scripts/pipelines (including dockerfile build/entrypoint) to use `npm ci` instead of `npm install` in CI https://issues.redhat.com/browse/CRW-1471

Out of scope:

• [upstream] Move e2e tests to new repo CRW-1732
• [downstream] Build&tag quay.io/crw/e2e-tests on CRW release https://issues.redhat.com/browse/CRW-1347