Description of problem:
Upgrades are failing as see in this job: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.13-upgrade-from-stable-4.12-e2e-gcp-ovn-rt-upgrade/1685281763175174144 The failures are consistent (have not accepted a payload in 6 days) though they are only seen in Nightly payloads, not CI payloads. The upgrade failure occurs because the worker nodes fail to upgrade with this error: {{"machineconfiguration.openshift.io/reason": "failed to update OS to quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3d16f67befd4c592d22746a234f31d3f6b2601a22f4cc711e8808ec4ebde3c29 : error running rpm-ostree rebase --experimental ostree-unverified-registry:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3d16f67befd4c592d22746a234f31d3f6b2601a22f4cc711e8808ec4ebde3c29: \u001b[0m\u001b[31merror: \u001b[0mCould not depsolve transaction; 1 problem detected:\n Problem: package kernel-modules-core-5.14.0-284.25.1.el9_2.x86_64 requires kernel-uname-r = 5.14.0-284.25.1.el9_2.x86_64, but none of the providers can be installed\n - conflicting requests\n: exit status 1", }}
Version-Release number of selected component (if applicable):
4.13-nightly
How reproducible:
Always
Steps to Reproduce:
1. install 4.12 nightly 2. attempt upgrade to 4.13 nightly payload 3. See worker nodes go degraded and report the packaging conflict error
Actual results:
upgrade fails to complete due to stuck/degraded worker nodes
Expected results:
Upgrade succeeds
Journald log:
Aug 02 13:43:07.760806 ci-op-gwcxts0y-3a480-22fv6-worker-a-8x7wh rpm-ostree[259674]: Txn Rebase on /org/projectatomic/rpmostree1/rhcos failed: Could not depsolve transaction; 1 problem detected:
Problem: package kernel-modules-core-5.14.0-284.25.1.el9_2.x86_64 requires kernel-uname-r = 5.14.0-284.25.1.el9_2.x86_64, but none of the providers can be installed
- conflicting requests
Aug 02 13:43:07.761038 ci-op-gwcxts0y-3a480-22fv6-worker-a-8x7wh rpm-ostree[259674]: Unlocked sysroot
Aug 02 13:43:07.761267 ci-op-gwcxts0y-3a480-22fv6-worker-a-8x7wh rpm-ostree[259674]: Process [pid: 260048 uid: 0 unit: crio-004d337180533a11c91ff50db9fdea3b34069d1cf2df5f42deb2164de409dcac.scope] disconnected from transaction progress
Aug 02 13:43:07.761762 ci-op-gwcxts0y-3a480-22fv6-worker-a-8x7wh rpm-ostree[259674]: client(id:machine-config-operator dbus:1.906 unit:crio-004d337180533a11c91ff50db9fdea3b34069d1cf2df5f42deb2164de409dcac.scope uid:0) vanished; remaining=0
Aug 02 13:43:07.761780 ci-op-gwcxts0y-3a480-22fv6-worker-a-8x7wh rpm-ostree[259674]: In idle state; will auto-exit in 60 seconds
- is blocked by
-
MCO-721 ImpactStatementRequested: OCPBUGS-17163: Fix for dnf-RHEL worker nodes breaks 4.12 -> 4.13
- Closed