Details
-
Bug
-
Resolution: Done
-
Normal
-
None
-
4.13, 4.12.z
-
Moderate
-
False
-
Description
During upgrade tests, the MCO will become temporarily degraded with the following events showing up in the event log:
Mar 20 14:17:21.199 E clusteroperator/machine-config condition/Degraded status/True reason/RequiredPoolsFailed changed: Unable to apply 4.12.0-0.okd-scos-2023-03-20-124235: error during syncRequiredMachineConfigPools: [timed out waiting for the condition, pool master has not progressed to latest configuration: osImageURL mismatch for master in rendered-master-c91c4fadbfa28dd8868e2a0976812cfd expected: registry.ci.openshift.org/origin/4.12-okd-scos-2023-03-20-124235@sha256:1fe1ea31e43879b9b1327498ffc7545c2e8f1d29533312d78b961342cf827c0e got: registry.ci.openshift.org/origin/4.12-okd-scos-2023-03-20-052136@sha256:1fe1ea31e43879b9b1327498ffc7545c2e8f1d29533312d78b961342cf827c0e, retrying]
This seems to be occurring with some frequency as indicated by its prevalence in CI search:
$ curl -s 'https://search.ci.openshift.org/search?search=clusteroperator%2Fmachine-config+condition%2FDegraded+status%2FTrue+reason%2F.*osImageURL+mismatch&maxAge=48h&context=1&type=bug%2Bissue%2Bjunit&name=%5E%28periodic%7Crelease%29.*4%5C.1%5B1%2C2%5D.*&excludeName=&maxMatches=1&maxBytes=20971520&groupBy=job' | jq 'keys | length' 89
The MCO should not become degraded during an upgrade unless it cannot proceed with the upgrade. For this particular failure mode, I think it's a temporary failure caused by a race condition since the MCO eventually requeues and clears its degraded status.
Attachments
Issue Links
- is related to
-
-
- Closed
-