Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-36001

boot failure when /boot is specified with --mount=UUID=...

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Normal Normal
    • rhel-9.5
    • rhel-9.4
    • boom-boot
    • None
    • boom-boot-1.6.1-2.el9
    • None
    • Moderate
    • EasyFix
    • Hide
      94616c740b2eaec1b3c3b8d23805e054550b1dbb
      664e80e9cff5d40349ba46746e997217de2e6a5b
      Show
      94616c740b2eaec1b3c3b8d23805e054550b1dbb 664e80e9cff5d40349ba46746e997217de2e6a5b
    • rhel-sst-logical-storage
    • ssg_filesystems_storage_and_HA
    • 1
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • All
    • None

      What were you trying to do that didn't work?

      Attempting to create a multi-volume snapshot boot configuration specifying the /boot mount using UUID= syntax.

      Please provide the package NVR for which bug is seen:

      boom-boot-1.6.0-2.el9.noarch

      How reproducible:

      100%

      Steps to reproduce

      [root@localhost ~]# boom create --title "SnapshotBroken" --root-lv rhel/root-snap --no-fstab --mount UUID=750ebf79-4d35-4985-ab97-c1651d6a496e:/boot:xfs:defaults --mount /dev/rhel/var-snap:/var:xfs:defaults --mount /dev/rhel/home-snap:/home:xfs:defaults --add-opts "console=tty0 console=ttyS1"
      

      Expected results

      Boot succeeds, device with UUID 750ebf79-4d35-4985-ab97-c1651d6a496e is mounted at /boot

      Actual results

      Boot fails with root read-only. It's only possible to log in to diagnose the problem via serial console:

      [FAILED] Failed to start D-Bus System Message Bus.
      See 'systemctl status dbus-broker.service' for details.
      [  OK  ] Started Authorization Manager.
      [  OK  ] Stopped Power Profiles daemon.
      [  OK  ] Listening on D-Bus System Message Bus Socket.
               Starting Modem Manager...
               Starting firewalld - dynamic firewall daemon...
               Starting Power Profiles daemon...
      [FAILED] Failed to start D-Bus System Message Bus.
      See 'systemctl status dbus-broker.service' for details.
      [FAILED] Failed to start Power Profiles daemon.
      See 'systemctl status power-profiles-daemon.service' for details.
      [FAILED] Failed to start User Login Management.
      See 'systemctl status systemd-logind.service' for details.
      [  OK  ] Stopped User Login Management.
      [  OK  ] Listening on D-Bus System Message Bus Socket.
               Starting Load Kernel Module drm...
      [  OK  ] Finished Load Kernel Module drm.
               Starting User Login Management...
      [FAILED] Failed to start D-Bus System Message Bus.
      See 'systemctl status dbus-broker.service' for details.
      [  OK  ] Started Modem Manager.
      [FAILED] Failed to start User Login Management.
      See 'systemctl status systemd-logind.service' for details.
      [  OK  ] Stopped User Login Management.
      [  OK  ] Listening on D-Bus System Message Bus Socket.
               Starting Load Kernel Module drm...
      [  OK  ] Finished Load Kernel Module drm.
               Starting User Login Management...
      [  OK  ] Stopped Power Profiles daemon.
               Starting Power Profiles daemon...
      [FAILED] Failed to start D-Bus System Message Bus.
      See 'systemctl status dbus-broker.service' for details.
      [FAILED] Failed to start User Login Management.
      See 'systemctl status systemd-logind.service' for details.
      [  OK  ] Stopped User Login Management.
      [FAILED] Failed to listen on D-Bus System Message Bus Socket.
      See 'systemctl status dbus.socket' for details.
      [DEPEND] Dependency failed for GNOME Display Manager.
      [DEPEND] Dependency failed for Power Profiles daemon.
      [DEPEND] Dependency failed for fire…lld - dynamic firewall daemon.
      [DEPEND] Dependency failed for User Login Management.
      [  OK  ] Reached target Preparation for Network.
      [  OK  ] Reached target Network.
      [  OK  ] Reached target Network is Online.
               Starting CUPS Scheduler...
               Starting Crash recovery kernel arming...
               Starting Load Kernel Module drm...
               Starting Enable periodic u…f entitlement certificates....
               Starting System Logging Service...
               Starting OpenSSH server daemon...
               Starting Permit User Sessions...
      [  OK  ] Finished Load Kernel Module drm.
      [  OK  ] Started Enable periodic up… of entitlement certificates..
      [  OK  ] Finished Permit User Sessions.
      [  OK  ] Started Deferred execution scheduler.
      [  OK  ] Started Command Scheduler.
               Starting Hold until boot process finishes up...
               Starting Terminate Plymouth Boot Screen...
      [  OK  ] Started CUPS Scheduler.
      [  OK  ] Started OpenSSH server daemon.
      [FAILED] Failed to listen on D-Bus System Message Bus Socket.
      [DEPEND] Dependency failed for Power Profiles daemon.
      [FAILED] Failed to start Crash recovery kernel arming.
      Red Hat Enterprise Linux 9.4 (Plow)
      Kernel 5.14.0-427.13.1.el9_4.x86_64 on an x86_64
      
      Activate the web console with: systemctl enable --now cockpit.socket
      
      localhost login: root
      Password: 
      Last login: Thu May  9 19:04:39 on tty2
      [root@localhost ~]# mount | grep rhel
      /dev/mapper/rhel-root--snap on / type xfs (ro,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
      /dev/mapper/rhel-var--snap on /var type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
      /dev/mapper/rhel-home--snap on /home type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
      [root@localhost ~]# 
      

      Workaround

      The problem can be avoided by specifying the /boot mount with a device path:

      [root@localhost ~]# boom create --title "Snapshot" --root-lv rhel/root-snap --no-fstab --mount /dev/vda1:/boot:xfs:defaults --mount /dev/rhel/var-snap:/var:xfs:defaults --mount /dev/rhel/home-snap:/home:xfs:defaults --add-opts "console=tty0 console=ttyS0"
      Created entry with boot_id 22ab0df:
        title Snapshot
        machine-id adf850dcb85441779af286c0479c528f
        version 5.14.0-427.13.1.el9_4.x86_64
        linux /vmlinuz-5.14.0-427.13.1.el9_4.x86_64
        initrd /initramfs-5.14.0-427.13.1.el9_4.x86_64.img
        options root=/dev/rhel/root-snap ro rd.lvm.lv=rhel/root-snap crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/swap rhgb quiet console=tty0 console=ttyS0 fstab=no systemd.mount-extra=/dev/vda1:/boot:xfs:defaults systemd.mount-extra=/dev/rhel/var-snap:/var:xfs:defaults systemd.mount-extra=/dev/rhel/home-snap:/home:xfs:defaults
        grub_users $grub_users
        grub_arg --unrestricted
        grub_class kernel
      

      The system then boots normally with all snapshot file systems mounted as expected:

      [root@localhost ~]# mount | grep rhel
      /dev/mapper/rhel-root--snap on / type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
      /dev/mapper/rhel-var--snap on /var type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
      /dev/mapper/rhel-home--snap on /home type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
      

      Root cause

      This is caused by a bug in the snapshot-remount-fs generator script supplied by the boom-boot package. The script fails to parse kernel command line options that contain more than one '=' character:

      [root@localhost ~]# /usr/lib/systemd/system-generators/snapshot-remount-fs 
      Traceback (most recent call last):
        File "/usr/lib/systemd/system-generators/snapshot-remount-fs", line 125, in <module>
          main(sys.argv)
        File "/usr/lib/systemd/system-generators/snapshot-remount-fs", line 118, in main
          _debug = _has_debug(cmdline)
        File "/usr/lib/systemd/system-generators/snapshot-remount-fs", line 69, in _has_debug
          return _arg_in_cmdline(cmdline, ARG_DEBUG)
        File "/usr/lib/systemd/system-generators/snapshot-remount-fs", line 59, in _arg_in_cmdline
          (word, _) = word.split("=")
      ValueError: too many values to unpack (expected 2)
      

      Upstream status

      This is fixed upstream in the following two commits:

      commit 94616c740b2eaec1b3c3b8d23805e054550b1dbb
      Author: Bryn M. Reeves <bmr@redhat.com>
      Date:   Tue Apr 23 19:22:19 2024 +0100
      
          snapshot-remount-fs: limit maxsplit when splitting kernel arguments
          
          Fixes: #25
          
          Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
      
      commit 664e80e9cff5d40349ba46746e997217de2e6a5b (origin/bmr-snapshot-remount-fs-fixes, bmr-snapshot-remount-fs-fixes)
      Author: Bryn M. Reeves <bmr@redhat.com>
      Date:   Tue Apr 23 19:24:11 2024 +0100
      
          snapshot-remount-fs: use correct systemd unit path
          
          Related: #25
          
          Signed-off-by: Bryn M. Reeves <bmr@redhat.com>
      

              lvm-team lvm-team
              rhn-support-bmr Bryn Reeves
              Bryn Reeves Bryn Reeves
              Filip Suba Filip Suba
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: