Hello Red Hat team,

When executing autotests for Veeam Agent for Linux backup software, we identified a situation of mutual blocking.

The veeamagent process is trying to release a block device.

But at the same time, the udev service is trying to freeze it.

The problem occurs when trying to recover after performing a backup.

[  616.049684] INFO: task systemd-udevd:749 blocked for more than 122 seconds.

[  616.049695]       Tainted: G           OE    --------- ---  5.14.0-70.13.1.el9_0.x86_64 #1

[  616.049697] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

[  616.049699] task:systemd-udevd   state stack:    0 pid:  749 ppid:     1 flags:0x00000000

[  616.049704] Call Trace:

[  616.049713]  __schedule+0x203/0x560

[  616.049723]  schedule+0x43/0xb0

[  616.049726]  blk_mq_freeze_queue_wait+0x62/0x90

[  616.049732]  ? do_wait_intr_irq+0xa0/0xa0

[  616.049737]  blk_queue_clear_zone_settings+0xe/0x90

[  616.049744]  sd_read_block_characteristics+0xa5/0x230 [sd_mod]

[  616.049749]  sd_revalidate_disk.isra.0+0x48f/0x660 [sd_mod]

[  616.049754]  ? scsi_block_when_processing_errors+0x22/0xd0

[  616.049766]  sd_open+0x1b5/0x220 [sd_mod]

[  616.049770]  blkdev_get_whole+0x22/0x90

[  616.049774]  blkdev_get_by_dev.part.0+0x199/0x2b0

[  616.049777]  blkdev_common_ioctl+0x8c1/0x930

[  616.049782]  blkdev_ioctl+0x10e/0x280

[  616.049784]  __x64_sys_ioctl+0x7f/0xb0

[  616.049790]  do_syscall_64+0x38/0x90

[  616.049794]  entry_SYSCALL_64_after_hwframe+0x44/0xae

[  616.049797] RIP: 0033:0x7f37cc252c0b

[  616.049800] RSP: 002b:00007ffc2e5553e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010

[  616.049802] RAX: ffffffffffffffda RBX: 00007ffc2e555440 RCX: 00007f37cc252c0b

[  616.049804] RDX: 0000000000000000 RSI: 000000000000125f RDI: 000000000000000e

[  616.049805] RBP: 00007ffc2e555590 R08: 000000000000003f R09: 105252ef827ab975

[  616.049806] R10: 0000000000000000 R11: 0000000000000246 R12: 0000558c52636e20

[  616.049807] R13: 00007ffc2e555450 R14: 0000000000000000 R15: 00007ffc2e555408

[  616.050072] INFO: task veeamagent:7332 blocked for more than 122 seconds.

[  616.050076]       Tainted: G           OE    --------- ---  5.14.0-70.13.1.el9_0.x86_64 #1

[  616.050078] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

[  616.050080] task:veeamagent      state stack:    0 pid: 7332 ppid:  7223 flags:0x00004000

[  616.050084] Call Trace:

[  616.050086]  __schedule+0x203/0x560

[  616.050090]  schedule+0x43/0xb0

[  616.050093]  schedule_preempt_disabled+0x11/0x20

[  616.050097]  __mutex_lock.constprop.0+0x234/0x430

[  616.050101]  blkdev_put+0x35/0x1f0

[  616.050103]  blkdev_close+0x18/0x20

[  616.050105]  __fput+0x91/0x240

[  616.050109]  task_work_run+0x59/0x90

[  616.050114]  exit_to_user_mode_loop+0x152/0x160

[  616.050119]  exit_to_user_mode_prepare+0x89/0xd0

[  616.050122]  syscall_exit_to_user_mode+0x12/0x30

[  616.050126]  do_syscall_64+0x48/0x90

[  616.050128]  entry_SYSCALL_64_after_hwframe+0x44/0xae

[  616.050130] RIP: 0033:0x7f850c6837fb

[  616.050131] RSP: 002b:00007f84f64f0b10 EFLAGS: 00000293 ORIG_RAX: 0000000000000003

[  616.050133] RAX: 0000000000000000 RBX: 00007f84ec109450 RCX: 00007f850c6837fb

[  616.050134] RDX: 0000000000000000 RSI: 00007f84ec0008d0 RDI: 0000000000000029

[  616.050135] RBP: 00007f84ec10d560 R08: 0000000000000000 R09: 00007f84f64f0269

[  616.050136] R10: 0000000000000000 R11: 0000000000000293 R12: 00000000ffffffff

[  616.050138] R13: 00007f84f64f0d30 R14: 00007f84f64f0dd0 R15: 00007f84f64f0df0