Summary

The test [Jira:"Cluster Version Operator"] cluster-version-operator should work with accept risks is failing consistently on techpreview-enabled configurations with a ClusterVersion API validation error.

Component Readiness Regressions

This bug covers the following Component Readiness regressions:

• Regression 36127 (arm64+aws) - 0% pass rate, opened 2026-03-06
  • Test Details: https://sippy-auth.dptools.openshift.org/sippy-ng/component_readiness/test_details?Architecture=arm64&FeatureSet=techpreview&Installer=ipi&LayeredProduct=none&Network=ovn&Platform=aws&Suite=serial&Topology=ha&Upgrade=none&baseEndTime=2026-02-03T23%3A59%3A59Z&baseRelease=4.21&baseStartTime=2026-01-04T00%3A00%3A00Z&capability=Other&columnGroupBy=Network%2CPlatform%2CTopology&component=Cluster+Version+Operator&confidence=95&dbGroupBy=Architecture%2CFeatureSet%2CInstaller%2CLayeredProduct%2CNetwork%2CPlatform%2CSuite%2CTopology%2CUpgrade&environment=Architecture%3Aarm64+FeatureSet%3Atechpreview+Installer%3Aipi+LayeredProduct%3Anone+Network%3Aovn+Platform%3Aaws+Suite%3Aserial+Topology%3Aha+Upgrade%3Anone&flakeAsFailure=false&ignoreDisruption=true&ignoreMissing=false&includeMultiReleaseAnalysis=true&testId=openshift-tests%3A1d86f065b246798036334c87a6c3f335
• Regression 36213 (amd64+metal) - 42.86% pass rate, opened 2026-03-07
  • Test Details: https://sippy-auth.dptools.openshift.org/sippy-ng/component_readiness/test_details?Architecture=amd64&FeatureSet=techpreview&Installer=ipi&LayeredProduct=none&Network=ovn&Platform=metal&Suite=serial&Topology=ha&Upgrade=none&baseEndTime=2026-02-03T23%3A59%3A59Z&baseRelease=4.21&baseStartTime=2026-01-04T00%3A00%3A00Z&capability=Other&columnGroupBy=Network%2CPlatform%2CTopology&component=Cluster+Version+Operator&confidence=95&dbGroupBy=Architecture%2CFeatureSet%2CInstaller%2CLayeredProduct%2CNetwork%2CPlatform%2CSuite%2CTopology%2CUpgrade&environment=Architecture%3Aamd64+FeatureSet%3Atechpreview+Installer%3Aipi+LayeredProduct%3Anone+Network%3Aovn+Platform%3Ametal+Suite%3Aserial+Topology%3Aha+Upgrade%3Anone&flakeAsFailure=false&ignoreDisruption=true&ignoreMissing=false&includeMultiReleaseAnalysis=true&testId=openshift-tests%3A1d86f065b246798036334c87a6c3f335
• Regression 36237 (amd64+aws) - 71.43% pass rate, opened 2026-03-08
  • Test Details: https://sippy-auth.dptools.openshift.org/sippy-ng/component_readiness/test_details?Architecture=amd64&FeatureSet=techpreview&Installer=ipi&LayeredProduct=none&Network=ovn&Platform=aws&Suite=serial&Topology=ha&Upgrade=none&baseEndTime=2026-02-03T23%3A59%3A59Z&baseRelease=4.21&baseStartTime=2026-01-04T00%3A00%3A00Z&capability=Other&columnGroupBy=Network%2CPlatform%2CTopology&component=Cluster+Version+Operator&confidence=95&dbGroupBy=Architecture%2CFeatureSet%2CInstaller%2CLayeredProduct%2CNetwork%2CPlatform%2CSuite%2CTopology%2CUpgrade&environment=Architecture%3Aamd64+FeatureSet%3Atechpreview+Installer%3Aipi+LayeredProduct%3Anone+Network%3Aovn+Platform%3Aaws+Suite%3Aserial+Topology%3Aha+Upgrade%3Anone&flakeAsFailure=false&ignoreDisruption=true&ignoreMissing=false&includeMultiReleaseAnalysis=true&testId=openshift-tests%3A1d86f065b246798036334c87a6c3f335

Test ID: openshift-tests:1d86f065b246798036334c87a6c3f335

Release: 4.22

Affected Variants

All failures occur on techpreview FeatureSet configurations:

• arm64 + aws + ovn (0% pass rate - completely broken)
• amd64 + metal + ovn (42.86% pass rate)
• amd64 + aws + ovn (71.43% pass rate)

Other variants (non-techpreview) are passing at 88-100%.

Failure Pattern

Pattern: Permafail on arm64+aws, recent regression on amd64 variants

• arm64+aws: All 33 runs failed over 28-day period (started failing before 2026-02-16)
• amd64+metal: Mixed failures (some permafail, some recent)
• amd64+aws: Recent regression (recent failures after previous successes)

Error Message (85% of failures)

ClusterVersion.config.openshift.io "version" is invalid: 
spec.desiredUpdate: Invalid value: cannot set both Architecture and Image

File: github.com/openshift/cluster-version-operator/test/cvo/accept_risks.go:119-120
Error Code: HTTP 422 (Unprocessable Entity)
Resource: ClusterVersion.config.openshift.io "version"
Field: spec.desiredUpdate

15% of failures show timeout errors (likely secondary to the validation issue).

Root Cause Analysis

The CVO API is rejecting ClusterVersion updates where both Architecture and Image fields are set in spec.desiredUpdate. This suggests:

1. Either new validation was added to the CVO API that's too strict
2. Or the accept-risks test started setting both fields incorrectly
3. Or a techpreview-specific code path is incorrectly populating both fields

The fact that this only affects techpreview configurations indicates this is likely related to:

• TechPreview-gated CVO features
• Accept-risks functionality in techpreview mode
• Multiarch support in techpreview

Sample Failed Job

https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-multiarch-main-nightly-4.22-ocp-e2e-aws-ovn-techpreview-serial-multi-a-a-1of3/2030855725336498176

Global Test Health

• Overall pass rate: 71.43% (down from 92.59% in previous period)
• Trend: Regressing (-21.16% net working improvement)
• The test is completely broken (0% pass rate) only on arm64+techpreview+aws
• Other techpreview variants show degraded but non-zero pass rates

Investigation Needed

1. Check if new API validation was added to ClusterVersion CRD for techpreview configurations
2. Review accept-risks test code to see if it's incorrectly setting both Architecture and Image
3. Check if multiarch/techpreview code paths in CVO are populating desiredUpdate incorrectly
4. Determine why this started before mid-February (regression detected late)