Major Description of problem:
test rollback scenario: 4.12.0 > 4.12.z > 4.12.0, rpm-ostreed service is failed to start with error
MCD:
2023-12-14T10:24:52.533578502+00:00 stderr F I1214 10:24:52.533541 233930 start.go:112] Version: v4.12.0-202301070015.p0.g2b3eba7.assembly.stream-dirty (2b3eba74dd9e4371f35ab41dbda02642f60707ec)
2023-12-14T10:24:52.534815817+00:00 stderr F I1214 10:24:52.534795 233930 start.go:125] Calling chroot("/rootfs")
2023-12-14T10:24:52.616874822+00:00 stderr F F1214 10:24:52.616845 233930 start.go:153] Failed to initialize single run daemon: error reading osImageURL from rpm-ostree: exit status 1
RPM-OSTREE:
rpm-ostree[61487]: Failed to find image ostree-unverified-image:docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fac5e78d53f71604afcdfc29c400ecdb221ac8cf891f8cae1f89f91ccfa11890
rpm-ostree[61487]: error: Couldn't start daemon: Error setting up sysroot: Reading deployment 1: Failed to find image ostree-unverified-image:docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fac5e78d53f71604afcdfc29c400ecdb221ac8cf891f8cae1f89f91ccfa11890
Version-Release number of selected component (if applicable):
How reproducible:
4.12.0
Steps to Reproduce:
1. upgrade cluster from 4.12.0 to 4.12.0-0.nightly-2023-12-13-175027
2. upgrade cluster from 4.12.0-0.nightly-2023-12-13-175027 to 4.12.0
Actual results:
machine-config-daemon pod cannot be started due to rpm-ostree failure
Expected results:
cluster can be rolled back to 4.12.0 successfully
Additional info:
sh-4.4# rpm-ostree --version rpm-ostree: Version: '2022.10' Git: 80afeab4b6236e5e74117425ad2d7cc9dd174e03 Features: - rust - compose - container - rhsm sh-4.4# ostree admin status -v OT: Deployment 5ba5c9e0cff001a35de83ec1fa168e84324061d70ec1b6aca05137957fa6841e.0 unlocked=0 OT: Deployment 1fba1a7715af6a3f8452cd7894b02aa5ead7dda75bfe88523da159ed54c3a740.0 unlocked=0 * rhcos 5ba5c9e0cff001a35de83ec1fa168e84324061d70ec1b6aca05137957fa6841e.0 origin: <unknown origin type> rhcos 1fba1a7715af6a3f8452cd7894b02aa5ead7dda75bfe88523da159ed54c3a740.0 (rollback) origin: <unknown origin type> sh-4.4# cat /ostree/deploy/rhcos/deploy/*.origin [origin] container-image-reference=ostree-unverified-registry:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fac5e78d53f71604afcdfc29c400ecdb221ac8cf891f8cae1f89f91ccfa11890 [origin] container-image-reference=ostree-unverified-registry:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6db665511f305ef230a2c752d836fe073e80550dc21cede3c55cf44db01db365 attached mcd and rpm-ostree log
mcd log: https://drive.google.com/file/d/1ouM-BWUfzGRFu-zgVXBis9DcjGdGtjl1/view?usp=drive_link
rpm-ostree log: https://drive.google.com/file/d/1zagn6YURroCWJJTyj8AwElRsm7I4qQCC/view?usp=drive_link