Summary

Integrate upgrade testing capabilities into the WINC-Regression Jenkins pipeline by adding an optional Stage 4 that performs OCP cluster upgrades and executes post-upgrade validation tests.

Background

The WINC-Regression pipeline currently provisions clusters, runs tests, and destroys clusters. To validate WMCO functionality across OCP upgrades (e.g., 4.20 → 4.21), we need automated upgrade testing integrated into the CI pipeline.

Implementation Details

Changes Made:

1. New Jenkins Parameters:
   • RUN_UPGRADE_TESTS (boolean, default: false) - Opt-in upgrade testing
   • UPGRADE_TARGET_VERSION (string, optional) - Target OCP version (auto-detects if empty)
2. Dynamic Timeout:
   • Automatically extends to 12 hours when RUN_UPGRADE_TESTS=true
   • Uses standard TIME_OUT parameter otherwise
3. Stage 4: Run Upgrade Tests:
   • Downloads upgrade_e2e.sh from winc-qe-docs GitLab
   • Clones openshift-tests-private repository
   • Upgrades OCP cluster to target version
   • Executes post-upgrade validation tests (test 43832)
   • Archives upgrade logs and test results
4. WMCO Upgrade Auto-Detection:
   • If LAUNCHER_VARS contains wmco_upgrade_index_image: Uses specified image
   • If LAUNCHER_VARS contains wmco_index_image but NO upgrade image: Auto-detects latest image
   • Pattern: {{quay.io/.../fbc-release-4- {targetVersion}

     :latest}}

   • Patches WMCO CatalogSource after successful OCP upgrade

Files Modified

• jenkins-jcasc-n/jobs/winc/winc-regression.groovy (+26 lines)
• jenkins-jcasc-n/jobs/winc/winc-regression.jenkinsfile (+136 lines)

Total: 162 insertions, 1 deletion

Git Repository

• Branch: winc-upgrade-stage
• Commits:
  • 85d5923f: Add WMCO upgrade image auto-detection
  • 7818b135: Add optional upgrade testing stage to WINC-Regression
• Worktree: /Users/rrasouli/Documents/GitHub/jenkins-jcasc-n-winc-upgrade

Testing Requirements

Before production deployment:

1. ✓ Backward compatibility (RUN_UPGRADE_TESTS=false)
2. ✓ Safety guard (skip when REMOVE_CLUSTER=true)
3. Test OCP-only upgrade (no WMCO)
4. Test OCP + WMCO upgrade (with auto-detection)
5. Test OCP + WMCO upgrade (with manual image)
6. Test auto-version detection
7. Test error handling and artifact archival

Benefits

• Automated OCP upgrade validation in CI pipeline
• WMCO compatibility testing across OCP versions
• Reduces manual upgrade testing effort
• Catches upgrade regressions early
• Maintains cluster for post-upgrade investigation

Acceptance Criteria

• [x] Parameters added to Jenkins job configuration
• [x] Stage 4 implemented with conditional execution
• [x] WMCO upgrade auto-detection working
• [x] Dynamic timeout configuration
• [x] Graceful error handling (UNSTABLE vs FAILURE)
• [ ] Code review completed
• [ ] Testing plan executed
• [ ] Documentation updated
• [ ] Merged to master and deployed to Jenkins

Related Work

• Script: {{https://gitlab.cee.redhat.com/winc/winc-qe-docs/-/raw/master/scripts/upgrade_e2e.sh}}
• Test: WINC-43832 (post-upgrade validation)
• PR: openshift-tests-private#28806 (test fixes)