Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-89828

[RHEL10] Unable to remove locked or leftover logical volume segment after a pvmove failure

Linking RHIVOS CVEs to...Migration: Automation ...SWIFT: Generate New Ti...SWIFT: POC ConversionSync from "Extern...XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Major Major
    • rhel-10.1
    • rhel-9.2.0
    • lvm2
    • lvm2-2.03.32-1.el10
    • None
    • Important
    • rhel-storage-lvm
    • ssg_platform_storage
    • 12
    • 15
    • 8
    • Dev ack
    • False
    • False
    • Hide

      None

      Show
      None
    • No
    • None
    • Release Note Not Required
    • None
    • 57,005

      Description of problem:

      Unable to complete or abort pvmove migration when it failed with error ' Logical volume vg00/pvmove0_mimage_0 in use.'

      1. pvmove --abort
        Internal error: Wrong params for _copy_poll_operation_id.

      Version-Release number of selected component (if applicable):

      Red Hat Enterprise Linux 9

      1. uname -r ; rpm -q lvm2
        5.14.0-284.11.1.el9_2.x86_64
        lvm2-2.03.17-7.el9.x86_64

      How reproducible:

      Always

      Steps to Reproduce:
      1. Create a Volume Group and logical volume with multiple PVs.
      2. Start migration using pvmove
      3. Try to access the pvmove image (pvmove0_mimage_0/1) and keep it as busy/open while pvmove is in progress.
      4. After moving all PEs, it failed with the error "LVM command executed by lvmpolld failed."

      Actual results:

      pvmove migration failed and was unable to remove the pvmove segments.

      >> Below is my lvm stack and I trying to migrate PEs from sda to sdc.

      1. lvs -ao +devices vg00
        LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
        lv00 vg00 wi-ao--- 3.50g /dev/sda(0)
        lv01 vg00 wi-a---- 1.00g /dev/sda(896)
        lvol0 vg00 wi-a---- 3.00g /dev/sdb(0)

      >> Start migration using pvmove and try to open the device.

      1. pvmove --atomic /dev/sda /dev/sdc
        /dev/sda: Moved: 0.78%
        /dev/sda: Moved: 100.00%
        LVM command executed by lvmpolld failed.
        For more information see lvmpolld messages in syslog or lvmpolld log file.

      ]# lvs -a
      WARNING: Cannot find matching striped segment for vg00/pvmove0_mimage_0.
      LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
      lv00 vg00 wI-ao--- 3.50g
      lv01 vg00 wI-a---- 1.00g
      lvol0 vg00 wi-a---- 3.00g
      [pvmove0] vg00 p-C-aom--- 4.50g
      pvmove0_mimage_0 vg00 wi-XXX 4.50g
      pvmove0_mimage_1 vg00 vwi-a-v--- 4.50g

      >> These images still exist even after reboot.

      1. lvs -ao +devices vg00
        LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
        lv00 vg00 wI-a---- 3.50g pvmove0(0)
        lv01 vg00 wI-a---- 1.00g pvmove0(0)
        lvol0 vg00 wi-a---- 3.00g /dev/sdb(0)
        [pvmove0] vg00 p-C-aom--- 4.50g /dev/sdc(0)
        [pvmove0] vg00 p-C-aom--- 4.50g /dev/sdc(896)
        pvmove0_mimage_0 vg00 wi-a---- 4.50g /dev/sda(0)
        pvmove0_mimage_1 vg00 vwi-a-v--- 4.50g

      >> Logical volumes are still accessible and able to mount, but a few pvmove segments still exist, and unable to clear them from lvm stack. Also, the source disk is in use, and unable to remove it from VG.

      1. dmsetup table |grep vg00
        vg00-lv00: 0 7340032 linear 253:2 0
        vg00-lv01: 0 2097152 linear 253:2 7340032
        vg00-lvol0: 0 6291456 linear 8:16 2048
        vg00-pvmove0: 0 7340032 linear 8:32 2048
        vg00-pvmove0: 7340032 2097152 linear 8:32 7342080
        vg00-pvmove0_mimage_0: 0 9437184 linear 8:0 2048
        vg00-pvmove0_mimage_1: 0 9437184 error <========

      >> pvmove abort is failing with the below error.

      1. pvmove --abort
        Internal error: Wrong params for _copy_poll_operation_id.

      Aug 15 12:17:39 localhost.localdomain systemd[1]: Started LVM2 poll daemon.
      Aug 15 12:17:59 localhost.localdomain lvmpolld[1816]: W: LVPOLL: PID 1819: STDERR: ' Logical volume vg00/pvmove0_mimage_0 in use.'
      Aug 15 12:17:59 localhost.localdomain lvmpolld[1816]: W: LVPOLL: PID 1819: STDERR: ' ABORTING: Removal of temporary mirror failed'
      Aug 15 12:17:59 localhost.localdomain lvmpolld[1816]: W: LVPOLL: PID 1819: STDERR: ' WARNING: This metadata update is NOT backed up.'
      Aug 15 12:17:59 localhost.localdomain lvmpolld[1816]: LVMPOLLD: lvm2 cmd (PID 1819) failed (retcode: 5) <=====
      Aug 15 12:19:08 localhost.localdomain systemd[1]: lvm2-lvmpolld.service: Deactivated successfully.

      Expected results:

      Looking for some options to clear the leftover pvmove segments properly even after failing the migration.

      Additional info:

      pvmove0_mimage_0/1 can be removed using lvremove, but the hidden "pvmove0" device can't change.

              cmarthal@redhat.com Corey Marthaler
              rhn-support-rmadhuso Ranjith ML
              lvm-team lvm-team
              Cluster QE Cluster QE
              Angana Chakraborty Angana Chakraborty
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Created:
                Updated:
                Resolved: