Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-17421

Kernel Does Not Properly Release Storage Devices

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Undefined Undefined
    • None
    • rhel-9.3.0
    • kernel / File Systems
    • Yes
    • Urgent
    • sst_filesystems
    • ssg_filesystems_storage_and_HA
    • 5
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • None
    • None
    • x86_64
    • None

      What were you trying to do that didn't work?

      We have a system storage-layout customization workflow that re-uses the boot disk-device. Under RHEL 7 and RHEL 8, the method we used encountered no issues. However, as we have started to undertake efforts to port our workflows to RHEL 9-based systems, we are discovering that attempts to modify the boot disk-device's geometry error-out.

      Please provide the package NVR for which bug is seen:

      How reproducible:

      Every time try to use the Red Hat published RHEL 9.x AMIs in AWS to create  RHEL 9 EC2s with customized storage layouts.

      Steps to reproduce

      1. Select the latest-available RHEL 9 AMI in the region (ami-0d2fe9aeb48a17188 in the us-gov-west-1 region; ami-023c11a32b0207432 in the us-east-1 region)
      2. Login to instance with default-user account
      3. Escalate privileges to root (using sudo -i)
      4. Execute a routine to move the disk-based OS content to a RAMfs:
      #!/bin/bash
      set -x
      set -e
      
      yum -y install psmisc
      
      echo "Creating /tmproot..."
      install -Ddm 000755 /tmp/tmproot
      
      echo "Mounting tmpfs to /tmp/tmproot..."
      mount none /tmp/tmproot -t tmpfs
      
      echo "Copying / to /tmp/tmproot..."
      cp -ax / /tmp/tmproot
      
      echo "Copying dev-nodes to /tmp/tmproot..."
      cp -a /dev /tmp/tmproot# Switch / to tmpfs
      
      echo "Creating /tmp/tmproot/oldroot..."
      mkdir /tmp/tmproot/oldroot
      
      echo "Prepare for pivot_root action..."
      mount --make-rprivate /
      
      echo "Execute pivot_root..."
      pivot_root /tmp/tmproot /tmp/tmproot/oldroot
      
      echo "Move sub-mounts into /oldroot..."
      mount --move /oldroot/dev /dev
      mount --move /oldroot/proc /proc
      mount --move /oldroot/sys /sys
      mount --move /oldroot/run /run
      mount --move /oldroot/tmp /tmp || true
      
      MOUNTS=$(
          cut -d ' ' -f 2 /proc/mounts | \
          grep '/oldroot/' | \
          sort -ru
      )
      echo "$MOUNTS" | while IFS= read -r MOUNT
      do
          echo "Attempting to dismount ${MOUNT}... "
          umount "$MOUNT" || \
            echo "Dismount failed for ${MOUNT}"
      done
      
      
      if systemctl is-active --quiet firewalld ; then systemctl stop firewalld ; fi
      
      systemctl restart sshd
      
      pkill --signal HUP sshdExpected results
      1. Reconnect to the instance to finish clearing out processes holding open the root disk
      systemctl daemon-reexec
      
      fuser -vmk /oldroot
      
      umount /oldroot 
      1. Clear out existing disk-header on root disk
      # dd if=/dev/zero of=<ROOT_DEV_PATH> bs=512 count=$(( 2 * 1024 * 10 )) 
      1. Attempt to write new partition table to root disk:
      # parted -s /dev/nvme0n1 -- mktable gpt \
        mkpart primary xfs 1049k 2m \
        mkpart primary fat16 4096s 130m \
        mkpart primary xfs 130m 642m \
        mkpart primary xfs 642m 100% \
        set 1 bios_grub on \
        set 2 esp on \
        set 3 bls_boot on \
        set 4 lvm on
      1. Receive error that kernel can't be informed of updated geometry:
      Error: Partition(s) 4 on /dev/nvme0n1 have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  As a result, the old partition(s) will remain in use.  You should reboot now before making further changes.

      Actual results

      All subsequent attempts to leverage the new geometry (e.g., creating LVM objects or other filesystem structures) fail when referencing partition-IDs that are either not present or do not match expected locations.

       

      In googling around, this behavior resembles one that was linked to a regression that was introduced in the 5.8 kernel and fixed in the 5.9 kernel. It looks like the regression returned sometime before Red Hat selected the 5.14 kernel for RHEL 9

        1. DiskFreed.txt
          3 kB
        2. parted.strace
          3.80 MB
        3. pivot-root.sh
          2 kB
        4. UmountLogs.txt
          3 kB

            fs-maint FS Maint
            redhat@xanthia.com Thomas Jones (Inactive)
            FS Maint FS Maint
            bot fs-qe bot fs-qe
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: