Outcome Overview

This initiative ensures that our E2E tests are written primarily in upstream, unless they are related to specific downstream OpenShift (OCP) features. This approach provides several clear benefits:

• Catch problems earlier: By running tests in upstream, we can find issues before the code reaches downstream. Fixing bugs in downstream (when we try to sync) often requires a huge effort compared to debugging, fixing and troubleshooting in upstream.
• Reuse tests in downstream: Upstream E2E tests also run in the downstream. In upstream, we run them against Kind; in downstream, we use OCP. This setup already works, but we want to improve it by keeping more logic in upstream for better test ownership, lower maintainability and consistency.
• Lower maintenance and faster delivery: Writing and running tests in upstream are simpler and have a lower dev cost workflow. It reduces time spent on setup and debugging, helping us deliver better test coverage with less work.
• Avoid duplicated tests and extra effort: In some cases, the same feature is tested in both upstream and downstream. This leads to duplicated effort and harder maintenance. By focusing on upstream, we can reduce overlap and improve test quality.
• Support early testing of tech-preview features: We can test feature-gated (tech-preview) functionality directly in upstream. This helps catch risky changes earlier and allows us to reuse those tests in OpenShift/Origin where possible.
• Reduce the number of test suites across repos: Having fewer separate test setups across multiple repositories makes our testing setup easier to manage, debug, and maintain over time.  

Success Criteria

• Feature-gated functionality is tested in upstream.
• E2E test scenarios can be fully mocked in upstream and reused in downstream.
• Upstream tests are executed against OCP in the downstream.
• All testable logic is covered in upstream first. ( Unless they are covering specific OCP features like Proxy )
• No E2E tests exist only in the downstream; if they could be done in the upstream.