MicroShift runs OpenShift conformance tests

Goal

The goal is to get a pre-submit job so that new tests would fail if the break on MicroShift. That notice authors to either fix or skip the test. MicroShift would benefit from all future test cases of OpenShift. And we need as much test coverage as possible to validate our claim "MicroShift is API-compatible with OCP".

Benefit Hypothesis:

We believe that the result of doing this work will be important to the quality of MicroShift. We need to understand / learn about potential deviations / differences in behaviour between OpenShift and MicroShift as soon as possible.

Resources

Add any resources (docs, slides, etc.) pertinent to the definition of the work. These might not be known until later. Update as necessary.

Responsibilities

Indicate which roles and/or teams will be responsible for contributing to the initiative and generally what they might be expected to do.

Success Criteria

1. go through all of the tests that fail, and add code to them to "skip" if the cluster being tested is a microshift cluster.
2. At that point it will be possible for us to run the suite in the way needed.After that, we need to show the job passes 70% or more of the time for 1 sprint.
3. Then we can add an informing job to the OCP payload, and TRT will start watching it for breakages.
4. Then we have to wait 2 more sprints before that job can become blocking, so that if it fails OCP does not release.
5. At that time, we would be able to add a pre-submit job to the repo with the test suite in it so that authors making changes to the tests would not be able to land breaking changes.

Outcomes

1. a pre-submit job to the repo with the test suite in it so that authors making changes to the tests would not be able to land breaking changes.