-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
rhel-7.9.z
-
None
-
Important
-
rhel-sst-logical-storage
-
ssg_filesystems_storage_and_HA
-
2
-
False
-
-
None
-
None
-
None
-
None
-
If docs needed, set a value
-
-
All
-
None
Description of problem:
Unable to complete or abort pvmove migration when it failed with error ' Logical volume testvg/pvmove0_mimage_0 in use.'
- pvmove --abort
Device name for LV testvg/pvmove0 not found in metadata. (unfinished pvmove mirror removal?)
Version-Release number of selected component (if applicable):
RHEL7.9 /lvm2-2.02.187-6.el7_9.5.x86_64
issue reproducible with RHEL9.2 as well [lvm2-2.03.17-7.el9.x86_64]
How reproducible:
Always
Steps to Reproduce:
1. Create volume group and logical volume with multiple PVs.
2. Start migration using pvmove
3. Try to access the pvmove image and keep it as busy/open while pvmove is in progress.
4. After moving all PEs, it failed with the error "LVM command executed by lvmpolld failed."
Actual results:
pvmove migration failed and unable to remove the pvmove segments.
>> Below is my lvm stack and I trying to migrate PEs from sda to sdc.
- lvs -ao +devices testvg
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices
lvol0 testvgwi-a---- 6.00g /dev/sda(0)
lvol0 testvgwi-a---- 6.00g /dev/sdb(0)
lvol1 testvgwi-a---- 1.00g /dev/sda(1023)
>> Start migration using pvmove and try to open the device. [ here I used hexdump utility to open the dm-X device].
- pvmove --atomic /dev/sda /dev/sdc
/dev/sda: Moved: 0.00%
/dev/sda: Moved: 24.63%
/dev/sda: Moved: 52.85%
/dev/sda: Moved: 80.30%
/dev/sda: Moved: 100.00%
LVM command executed by lvmpolld failed.
For more information see lvmpolld messages in syslog or lvmpolld log file.
>> Logical volumes are still accessible and able to mount, but a few pvmove segments still exist, and unable to clear them from lvm stack. Also, the source disk is in use, and unable to remove it from VG.
- dmsetup table |grep testvg
testvg-pvmove0_mimage_1: 0 10477568 error
testvg-pvmove0_mimage_0: 0 8380416 linear 8:0 2048
testvg-pvmove0_mimage_0: 8380416 2097152 linear 8:0 8382464
testvg-lvol1: 0 2097152 linear 253:6 8380416
testvg-pvmove0: 0 8380416 linear 8:32 2048
testvg-pvmove0: 8380416 2097152 linear 8:32 8382464
testvg-lvol0: 0 8380416 linear 253:6 0
testvg-lvol0: 8380416 4202496 linear 8:16 2048
>> pvmove abort is failing with below error.
- pvmove --abort
Device name for LV testvg/pvmove0 not found in metadata. (unfinished pvmove mirror removal?)
- journalctl -u lvm2-lvmpolld
- Logs begin at Wed 2023-08-02 12:54:26 EDT, end at Wed 2023-08-02 13:22:17 EDT. –
Aug 02 13:10:55 node1 systemd[1]: Started LVM2 poll daemon.
Aug 02 13:11:40 node1 lvmpolld[2501]: W: LVPOLL: PID 2504: STDERR: ' WARNING: This metadata update is NOT backed up.'
Aug 02 13:14:27 node1 lvmpolld[2501]: W: LVPOLL: PID 2860: STDERR: ' WARNING: This metadata update is NOT backed up.'
Aug 02 13:16:24 node1 systemd[1]: Started LVM2 poll daemon.
Aug 02 13:17:24 node1 lvmpolld[3222]: W: LVPOLL: PID 3225: STDERR: ' WARNING: This metadata update is NOT backed up.'
Aug 02 13:17:29 node1 lvmpolld[3222]: W: LVPOLL: PID 3225: STDERR: ' Logical volume testvg/pvmove0_mimage_0 in use.' <===============
Aug 02 13:17:29 node1 lvmpolld[3222]: LVMPOLLD: lvm2 cmd (PID 3225) failed (retcode: 5)
- Logs begin at Wed 2023-08-02 12:54:26 EDT, end at Wed 2023-08-02 13:22:17 EDT. –
Expected results:
Looking for some options to clear the leftover pvmove segments properly. There are options to manually edit the VG metadata and restore it back to disk, but for a very large volume, this should be very complicated and may cause the lv in the LOCKED state.
Additional info:
This issue is reproducible with RHEL9 standalone systems as well.
After reproducing this issue, I tried to disable ( use_lvmpolld = 0), but still unable to abort or complete pvmove.
- external trackers