Goal:

• Allow incoming workloads to be scheduled in the dedicated node, which has NoSchedule taint, when e2e suite is running, preventing false-positive issues, like OCPBUGS-31407 

Suggested solution:

• 1/ Create a mutation webhook to patch any e2e workload to the dedicated node to have the required toleration
• 2/ Create a mechanism to make the pods failing to schedule caused by 'dedicated node' (tainted worker node required by OPCT architecture) to not fail in the final test results, having minimum impact in the conformance test execution: create a controller watching all failed pods, mutating it to allow scheduling in the tainted node by applying the required tolerations to the pod spec.

Why:

• Almost every execution of OPCT there are false-positive failures due tests trying to schedule workloads in the dedicated node and getting failure
• This is caused by OPCT architecture using dedicated node preventing running the controller (aggregator server) in client side to prevent issues with client connectivity with OCP cluster. So it would not be possible to add exceptions to OCP product for non-OCP utility (OPCT).
• Change the OPCT environment would be more complicated due factor aforementioned.

Acceptance Criteria:

• Solution must be validated in a real job execution
• Jobs which where previously failing to schedule in the dedicated node must be executed normally. Example of jobs:
  • TODO (see the attached issues)
• No regressions should be introduced, no false positives must be seen in different CI executions
• An Alpha v0.6 release must be created to be able to use this solution as "dev preview" by CI jobs