-
Bug
-
Resolution: Done
-
High
-
None
-
None
-
None
-
False
-
None
-
False
-
-
-
Kata Sprint #258
-
0
-
0
-
Urgent
Description
The installation of kata-containers-3.7.0-1.rhaos4.15.el9.x86_64 on OCP 4.15.30 fails on post-install script due the lack of `strip` command on RHCOS 4.15
Steps to reproduce
- Get a OCP 4.15 (up to 4.15.30) cluster. I've used one that QE created for me
Install the OSC operator 1.7-21
Configure for peer pods and apply the kataconfig - Copy kata-containers-3.7.0-1.rhaos4.15.el9 RPM to any worker (e.g. `dd if=kata-containers-3.7.0-1.rhaos4.15.el9.x86_64.rpm| oc debug -T node/$node – dd of=/host/var/local/kata-containers-3.7.0-1.rhaos4.15.el9.x86_64.rpm`)
- Connect at the worker (`oc debug -T node/$node`), mount the host (chroot /host), remount /usr (mount -o remount,rw /usr)
- Finally upgrade the rpm (`rpm -Uvh /var/local/kata-containers-3.7.0-1.rhaos4.15.el9.x86_64.rpm`)
Expected result
It should upgrade from kata-containers-3.2.0-7.rhaos4.15.el9 version
Actual result
The RPM is indeed upgraded but the postinstall scriptlet fails to rebuild the initrd. See the error logs below.
Impact
The failing script (/usr/libexec/kata-containers/osbuilder/rootfs-builder/rootfs.sh) is called every time the kata's initrd is rebuilt, for example, when the kata-containers and/or kernel packages are upgraded. We cannot guarantee the integrity and correctness of the generated initrd.
It doesn't impact peer pods use case though.
Env
- kata-containers-3.7.0-1.rhaos4.15.el9.x86_64
- OCP 4.15.30
- Cluster created with flexy install build [310013|https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/ocp-common/job/Flexy-install/310013/]
Additional helpful info
The error logs:
Starting pod/ip-10-0-50-96us-east-2computeinternal-debug-g9zm9 ... To use host binaries, run `chroot /host` Pod IP: 10.0.50.96 If you don't see a command prompt, try pressing enter. sh: cannot set terminal process group (44265): Inappropriate ioctl for device sh: no job control in this shell sh-5.1# mount -o remount,rw /usr sh-5.1# rpm -Uvh /var/local/kata-containers-3.7.0-1.rhaos4.15.el9.x86_64.rpm Verifying... ######################################## Preparing... ######################################## Updating / installing... kata-containers-3.7.0-1.rhaos4.15.el9 ######################################## Creating kata appliance initrd... warning: %post(kata-containers-3.7.0-1.rhaos4.15.el9.x86_64) scriptlet failed, exit status 1 Building failed. Here is the log details: + Building dracut initrd dracut: Could not find any syslog binary although the syslogmodule is selected to be installed. Please check. + Extracting dracut initrd rootfs 78887 blocks + Copying agent directory tree into place Calling osbuilder rootfs.sh on extracted rootfs INFO: Create symlink to /tmp in /var to create private temporal directories with systemd INFO: Install tmp.mount in ./etc/systemd/system cp: cannot stat './usr/share/systemd/tmp.mount': No such file or directory INFO: Create /tmp/kata-dracut-rootfs-p3IZG1/etc INFO: Configure chrony file /tmp/kata-dracut-rootfs-p3IZG1/etc/chrony.conf [OK] cp /usr/libexec/kata-containers/agent/usr/bin/kata-agent /tmp/kata-dracut-rootfs-p3IZG1/usr/bin/kata-agent ./rootfs-builder/rootfs.sh: line 685: strip: command not found Failed at 685: ${stripping_tool} ${ROOTFS_DIR}/usr/bin/kata-agent Cleaning up / removing... kata-containers-3.2.0-7.rhaos4.15.el9 ######################################## sh-5.1#
The RPM's postinstall scriptlet:
$ rpm -qp --scripts kata-containers-3.7.0-1.rhaos4.15.el9.x86_64.rpm postinstall scriptlet (using /bin/sh): if [ $1 -eq 1 ] && [ -x "/usr/lib/systemd/systemd-update-helper" ]; then # Initial installation /usr/lib/systemd/systemd-update-helper install-system-units kata-osbuilder-generate.service || : fi # Skip running this on Fedora CoreOS / Red Hat CoreOS if test -w /var/cache/kata-containers; then TMPOUT="$(mktemp -t kata-rpm-post-XXXXXX.log)" echo "Creating kata appliance initrd..." /usr/libexec/kata-containers/osbuilder/kata-osbuilder.sh > ${TMPOUT} 2>&1 if test "$?" != "0" ; then echo "Building failed. Here is the log details:" cat ${TMPOUT} exit 1 fi fi
The RPM's dependencies:
$ rpm -qp --requires kata-containers-3.7.0-1.rhaos4.15.el9.x86_64.rpm /bin/sh /bin/sh /bin/sh /bin/sh /usr/bin/bash dracut kernel libc.so.6()(64bit) libc.so.6(GLIBC_2.10)(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.15)(64bit) libc.so.6(GLIBC_2.17)(64bit) libc.so.6(GLIBC_2.18)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.25)(64bit) libc.so.6(GLIBC_2.27)(64bit) libc.so.6(GLIBC_2.28)(64bit) libc.so.6(GLIBC_2.29)(64bit) libc.so.6(GLIBC_2.3)(64bit) libc.so.6(GLIBC_2.3.2)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.32)(64bit) libc.so.6(GLIBC_2.33)(64bit) libc.so.6(GLIBC_2.34)(64bit) libc.so.6(GLIBC_2.4)(64bit) libc.so.6(GLIBC_2.6)(64bit) libc.so.6(GLIBC_2.7)(64bit) libc.so.6(GLIBC_2.8)(64bit) libc.so.6(GLIBC_2.9)(64bit) libgcc_s.so.1()(64bit) libgcc_s.so.1(GCC_3.0)(64bit) libgcc_s.so.1(GCC_3.3)(64bit) libgcc_s.so.1(GCC_4.2.0)(64bit) libm.so.6()(64bit) libm.so.6(GLIBC_2.29)(64bit) libresolv.so.2()(64bit) libseccomp.so.2()(64bit) qemu-kvm-core >= 17:7.2.0 rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(PayloadIsZstd) <= 5.4.18-1 rtld(GNU_HASH) systemd systemd systemd virtiofsd
rhgkurz figured kata > 3.3 depends on the `strip` tool (see https://github.com/kata-containers/kata-containers/commit/3a6510ad61c02)