What were you trying to do that didn't work?

leapp upgrade from RHEL 8.9 to 9.3

Please provide the package NVR for which bug is seen:

leapp-0.16.0-2.el8.noarch

How reproducible:

Always (on this system)

Steps to reproduce

1. leapp preupgrade
2. leapp upgrade
3. reboot

Expected results

System is upgraded to RHEL9

Actual results

Failure during upgrade, partial output below:

[   13.929206] localhost upgrade[858]:   6 logical volume(s) in volume group "cent" now active
[   13.969671] localhost kernel: xfs filesystem being remounted at /sysroot supports timestamps until 2038 (0x7fffffff)
[   14.813111] localhost kernel: XFS (md126): Mounting V5 Filesystem
[   14.968079] localhost kernel: XFS (md126): Ending clean mount
[   14.968126] localhost kernel: xfs filesystem being mounted at /boot supports timestamps until 2038 (0x7fffffff)
[   15.702378] localhost upgrade[913]: mount: /boot/efi: can't find UUID=299B-560E.
[   15.715334] localhost kernel: XFS (dm-3): Mounting V5 Filesystem
[   15.877427] localhost kernel: XFS (dm-3): Ending clean mount
[   15.877475] localhost kernel: xfs filesystem being mounted at /home supports timestamps until 2038 (0x7fffffff)
[   15.890301] localhost kernel: XFS (dm-2): Mounting V5 Filesystem
[   15.988538] localhost kernel: XFS (dm-2): Ending clean mount
[   15.988588] localhost kernel: xfs filesystem being mounted at /var supports timestamps until 2038 (0x7fffffff)
[   16.001307] localhost kernel: XFS (dm-4): Mounting V5 Filesystem
[   16.177586] localhost kernel: XFS (dm-4): Ending clean mount
[   16.177633] localhost kernel: xfs filesystem being mounted at /srv supports timestamps until 2038 (0x7fffffff)
[   16.214281] localhost kernel: XFS (dm-5): Mounting V5 Filesystem
[   16.320067] localhost rngd[783]: [jitter]: Unable to obtain AES key, disabling JITTER source
[   16.321024] localhost rngd[783]: [jitter]: Initialization Failed
[   16.321024] localhost rngd[783]: Can't open any entropy source
[   16.321024] localhost rngd[783]: Maybe RNG device modules are not loaded
[   16.322406] localhost systemd[1]: rngd.service: Main process exited, code=exited, status=1/FAILURE
[   16.322837] localhost systemd[1]: rngd.service: Failed with result 'exit-code'.
[   16.323226] localhost systemd[1]: rngd.service: Consumed 16.366s CPU time.
[   16.375090] localhost kernel: XFS (dm-5): Ending clean mount
[   16.375137] localhost kernel: xfs filesystem being mounted at /srv/backups/timemachine supports timestamps until 2038 (0x7fffffff)
[   96.348265] localhost upgrade[954]: ==> Processing phase `InitRamStart`
[   96.348265] localhost upgrade[954]: ====> * remove_upgrade_boot_entry
[   96.348265] localhost upgrade[954]:         Remove boot entry for Leapp provided initramfs.
[   96.619104] localhost kernel: xfs filesystem being mounted at /boot supports timestamps until 2038 (0x7fffffff)
[   99.201695] localhost upgrade[1357]: Process Process-176:
[   99.201695] localhost upgrade[1357]: Traceback (most recent call last):
[   99.201695] localhost upgrade[1357]:   File "/usr/lib64/python3.6/multiprocessing/process.py", line 258, in _bootstrap
[   99.201695] localhost upgrade[1357]:     self.run()
[   99.201695] localhost upgrade[1357]:   File "/usr/lib64/python3.6/multiprocessing/process.py", line 93, in run
[   99.201695] localhost upgrade[1357]:     self._target(*self._args, **self._kwargs)
[   99.201695] localhost upgrade[1357]:   File "/usr/lib/python3.6/site-packages/leapp/repository/actor_definition.py", line 74, in _do_run
[   99.201695] localhost upgrade[1357]:     actor_instance.run(*args, **kwargs)
[   99.201695] localhost upgrade[1357]:   File "/usr/lib/python3.6/site-packages/leapp/actors/_init_.py", line 289, in run
[   99.201695] localhost upgrade[1357]:     self.process(*args)
[   99.201695] localhost upgrade[1357]:   File "/etc/leapp/repos.d/system_upgrade/common/actors/removeupgradebootentry/actor.py", line 20, in process
[   99.201695] localhost upgrade[1357]:     remove_boot_entry()
[   99.201695] localhost upgrade[1357]:   File "/etc/leapp/repos.d/system_upgrade/common/actors/removeupgradebootentry/libraries/removeupgradebootentry.py", line 41, in remove_boot_entry
[   99.201695] localhost upgrade[1357]:     '/bin/mount', '-a'
[   99.201695] localhost upgrade[1357]:   File "/usr/lib/python3.6/site-packages/leapp/libraries/stdlib/_init_.py", line 192, in run
[   99.201695] localhost upgrade[1357]:     result=result
[   99.201695] localhost upgrade[1357]: leapp.libraries.stdlib.CalledProcessError: Command ['/bin/mount', '-a'] failed with exit code 32.
[   99.406911] localhost upgrade[954]: 2024-03-13 14:05:52.10  ERROR    PID: 3 leapp.workflow.InitRamStart: Actor remove_upgrade_boot_entry has crashed: Traceback (most recent call last):
[   99.406911] localhost upgrade[954]:   File "/usr/lib/python3.6/site-packages/leapp/repository/actor_definition.py", line 74, in _do_run
[   99.406911] localhost upgrade[954]:     actor_instance.run(*args, **kwargs)
[   99.406911] localhost upgrade[954]:   File "/usr/lib/python3.6/site-packages/leapp/actors/_init_.py", line 289, in run
[   99.406911] localhost upgrade[954]:     self.process(*args)
[   99.406911] localhost upgrade[954]:   File "/etc/leapp/repos.d/system_upgrade/common/actors/removeupgradebootentry/actor.py", line 20, in process
[   99.406911] localhost upgrade[954]:     remove_boot_entry()
[   99.406911] localhost upgrade[954]:   File "/etc/leapp/repos.d/system_upgrade/common/actors/removeupgradebootentry/libraries/removeupgradebootentry.py", line 41, in remove_boot_entry
[   99.406911] localhost upgrade[954]:     '/bin/mount', '-a'
[   99.406911] localhost upgrade[954]:   File "/usr/lib/python3.6/site-packages/leapp/libraries/stdlib/_init_.py", line 192, in run
[   99.406911] localhost upgrade[954]:     result=result
[   99.406911] localhost upgrade[954]: leapp.libraries.stdlib.CalledProcessError: Command ['/bin/mount', '-a'] failed with exit code 32.
[  101.289980] localhost upgrade[954]: ==========================================================================================================
[  101.289980] localhost upgrade[954]: Actor remove_upgrade_boot_entry unexpectedly terminated with exit code: 1 - Please check the above details
[  101.289980] localhost upgrade[954]: ==========================================================================================================
[  101.289980] localhost upgrade[954]: Debug output written to /var/log/leapp/leapp-upgrade.log
[  101.289980] localhost upgrade[954]: ============================================================
[  101.289980] localhost upgrade[954]:                       REPORT OVERVIEW
[  101.289980] localhost upgrade[954]: ============================================================
[  101.289980] localhost upgrade[954]: HIGH and MEDIUM severity reports:
[  101.289980] localhost upgrade[954]:     1. Packages not signed by Red Hat found on the system
[  101.289980] localhost upgrade[954]: Reports summary:
[  101.289980] localhost upgrade[954]:     Errors:                      1
[  101.289980] localhost upgrade[954]:     Inhibitors:                  0
[  101.289980] localhost upgrade[954]:     HIGH severity reports:       1
[  101.289980] localhost upgrade[954]:     MEDIUM severity reports:     0
[  101.289980] localhost upgrade[954]:     LOW severity reports:        1
[  101.289980] localhost upgrade[954]:     INFO severity reports:       4
[  101.289980] localhost upgrade[954]: Before continuing consult the full report:
[  101.289980] localhost upgrade[954]:     A report has been generated at /var/log/leapp/leapp-report.json
[  101.289980] localhost upgrade[954]:     A report has been generated at /var/log/leapp/leapp-report.txt
[  101.289980] localhost upgrade[954]: ============================================================
[  101.289980] localhost upgrade[954]:                    END OF REPORT OVERVIEW
[  101.289980] localhost upgrade[954]: ============================================================
[  101.289980] localhost upgrade[954]: Answerfile has been generated at /var/log/leapp/answerfile

The key appears to be that /dev/md125 is not being assembled, which is made of /dev/sda2 and /dev/sdb2. This is the /boot/efi filesystem and uses md metadata 1.0 instead of 1.2 (since it's FAT), in case that matters.

Can provide the rdsosreport.txt if needed, as well as any other info.