Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-46516

[RHEL9.4][MD]task mdadm:963 blocked for more than 122 seconds.

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Major Major
    • None
    • rhel-9.5
    • mdadm
    • None
    • None
    • None
    • sst_logical_storage
    • ssg_filesystems_storage_and_HA
    • 4
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None

      What were you trying to do that didn't work?

      Please provide the package NVR for which bug is seen:

      5.14.0-472.el9.x86_64
      mdadm-4.3-2.el9.x86_64

      How reproducible:

      Steps to reproduce

      #mdadm -CR -l1 /dev/md0 -n 2 /dev/sdf /dev/sdg
      #echo writemostly > /sys/block/md0/md/dev-sdg/state
      #mdadm -f /dev/md0 /dev/sdf
      #mdadm /dev/md0 -r /dev/sdf
      #reboot
      

      or

      #mdadm -CR -l1 /dev/md0 -n 2 /dev/sdf /dev/sdg
      #writemostly > /sys/block/md0/md/dev-sdg/state
      #mdadm -f /dev/md0 /dev/sdf
      #mdadm /dev/md0 -r /dev/sdf
      #wipefs /dev/sdf
      #reboot
      #mdadm /dev/md126 -a /dev/sdf
      #mdadm /dev/md126 -f /dev/sdf
      #reboot
      

      Expected results

      Actual results

       

      [  246.650712] INFO: task mdadm:963 blocked for more than 122 seconds.
      [  246.650728]       Not tainted 5.14.0-472.el9.x86_64 #1
      [  246.650735] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      [  246.650739] task:mdadm           state:D stack:0     pid:963   tgid:963   ppid:1      flags:0x00004006
      [  246.650751] Call Trace:
      [  246.650755]  <TASK>
      [  246.650761]  __schedule+0x21b/0x540
      [  246.650779]  schedule+0x2e/0xd0
      [  246.650786]  schedule_preempt_disabled+0x11/0x20
      [  246.650793]  __mutex_lock.constprop.0+0x433/0x6a0
      [  246.650800]  ? kernfs_add_one+0xf0/0x130
      [  246.650809]  bd_link_disk_holder+0x81/0x240
      [  246.650820]  bind_rdev_to_array+0x24b/0x350
      [  246.650833]  md_add_new_disk+0x292/0x500
      [  246.650839]  md_ioctl+0x955/0x970
      [  246.650846]  blkdev_ioctl+0x13e/0x270
      [  246.650856]  ? security_file_ioctl+0x32/0x50
      [  246.650864]  __x64_sys_ioctl+0x8a/0xc0
      [  246.650875]  do_syscall_64+0x5f/0xf0
      [  246.650883]  ? rcu_nocb_try_bypass+0x5e/0x460
      [  246.650894]  ? __pfx_inode_free_by_rcu+0x10/0x10
      [  246.650903]  ? fsnotify_grab_connector+0x49/0x80
      [  246.650911]  ? __pfx___d_free+0x10/0x10
      [  246.650916]  ? __call_rcu_common.constprop.0+0x117/0x2b0
      [  246.650921]  ? rcu_nocb_try_bypass+0x5e/0x460
      [  246.650928]  ? __pfx_file_free_rcu+0x10/0x10
      [  246.650936]  ? __call_rcu_common.constprop.0+0x117/0x2b0
      [  246.650942]  ? syscall_exit_to_user_mode+0x19/0x40
      [  246.650948]  ? do_syscall_64+0x6b/0xf0
      [  246.650952]  ? do_user_addr_fault+0x1d6/0x6a0
      [  246.650958]  ? do_syscall_64+0x6b/0xf0
      [  246.650963]  ? exc_page_fault+0x62/0x150
      [  246.650966]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
      [  246.650975] RIP: 0033:0x7f5f9530357b
      [  246.651009] RSP: 002b:00007ffd35e9ced8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
      [  246.651014] RAX: ffffffffffffffda RBX: 0000559e375c4e90 RCX: 00007f5f9530357b
      [  246.651017] RDX: 00007ffd35e9d408 RSI: 0000000040140921 RDI: 0000000000000004
      [  246.651019] RBP: 0000559e375c5430 R08: 000000000000001e R09: 00007ffd35e9cc37
      [  246.651021] R10: 0000000000000002 R11: 0000000000000246 R12: 00007ffd35e9d0f0
      [  246.651024] R13: 00007ffd35e9db40 R14: 0000559e375c3e50 R15: 0000000000000004
      [  246.651029]  </TASK>
      [  246.651031] INFO: task mdadm:965 blocked for more than 122 seconds.
      [  246.651034]       Not tainted 5.14.0-472.el9.x86_64 #1
      [  246.651036] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      [  246.651038] task:mdadm           state:D stack:0     pid:965   tgid:965   ppid:1      flags:0x00000006
      [  246.651044] Call Trace:
      [  246.651046]  <TASK>
      [  246.651048]  __schedule+0x21b/0x540
      [  246.651056]  schedule+0x2e/0xd0
      [  246.651063]  md_handle_request+0xfb/0x270
      [  246.651070]  ? __pfx_autoremove_wake_function+0x10/0x10
      [  246.651079]  __submit_bio+0x97/0x130
      [  246.651087]  __submit_bio_noacct+0x7e/0x1e0
      [  246.651092]  submit_bh_wbc+0x115/0x140
      [  246.651101]  block_read_full_folio+0x1ec/0x2a0
      [  246.651108]  ? __pfx_blkdev_get_block+0x10/0x10
      [  246.651118]  ? __mod_memcg_lruvec_state+0x76/0xc0
      [  246.651125]  ? __mod_lruvec_page_state+0x97/0x160
      [  246.651130]  ? __pfx_blkdev_read_folio+0x10/0x10
      [  246.651136]  ? __pfx_blkdev_read_folio+0x10/0x10
      [  246.651142]  filemap_read_folio+0x43/0x2c0
      [  246.651151]  ? __pfx_workingset_update_node+0x10/0x10
      [  246.651160]  ? __pfx_blkdev_read_folio+0x10/0x10
      [  246.651166]  ? __pfx_blkdev_read_folio+0x10/0x10
      [  246.651173]  do_read_cache_folio+0x115/0x3c0
      [  246.651179]  ? find_busiest_group+0x43/0x240
      [  246.651188]  ? __rmqueue_pcplist+0x53/0x250
      [  246.651195]  ? vmap_pages_pud_range+0x285/0x470
      [  246.651202]  read_cache_page+0x13/0x60
      [  246.651207]  read_part_sector+0x3a/0x160
      [  246.651214]  read_lba+0xe5/0x250
      [  246.651219]  find_valid_gpt.constprop.0+0xd5/0x540
      [  246.651226]  efi_partition+0x80/0x390
      [  246.651230]  ? vsnprintf+0x3ac/0x570
      [  246.651239]  ? snprintf+0x49/0x70
      [  246.651245]  ? __pfx_efi_partition+0x10/0x10
      [  246.651249]  check_partition+0x103/0x1d0
      [  246.651253]  bdev_disk_changed.part.0+0x182/0x300
      [  246.651258]  blkdev_get_whole+0x7a/0x90
      [  246.651264]  bdev_open+0x278/0x3c0
      [  246.651270]  ? __pfx_blkdev_open+0x10/0x10
      [  246.651274]  blkdev_open+0x87/0xb0
      [  246.651278]  do_dentry_open+0x14f/0x440
      [  246.651286]  do_open+0x21a/0x450
      [  246.651295]  path_openat+0x111/0x280
      [  246.651303]  do_filp_open+0xb2/0x160
      [  246.651310]  ? __check_object_size.part.0+0x47/0xd0
      [  246.651318]  do_sys_openat2+0x96/0xd0
      [  246.651326]  __x64_sys_openat+0x53/0xa0
      [  246.651333]  do_syscall_64+0x5f/0xf0
      [  246.651338]  ? __x64_sys_mknodat+0x33/0x40
      [  246.651341]  ? syscall_exit_to_user_mode+0x19/0x40
      [  246.651345]  ? do_syscall_64+0x6b/0xf0
      [  246.651350]  ? __seccomp_filter+0x45/0x480
      [  246.651360]  ? syscall_trace_enter.constprop.0+0x9a/0x1a0
      [  246.651369]  ? __task_pid_nr_ns+0x97/0xb0
      [  246.651378]  ? syscall_exit_to_user_mode+0x19/0x40
      [  246.651381]  ? do_syscall_64+0x6b/0xf0
      [  246.651385]  ? __x64_sys_openat+0x53/0xa0
      [  246.651391]  ? syscall_exit_to_user_mode+0x19/0x40
      [  246.651394]  ? do_syscall_64+0x6b/0xf0
      [  246.651398]  ? syscall_exit_to_user_mode+0x19/0x40
      [  246.651402]  ? do_syscall_64+0x6b/0xf0
      [  246.651405]  ? syscall_exit_to_user_mode+0x19/0x40
      [  246.651409]  ? do_syscall_64+0x6b/0xf0
      [  246.651414]  ? exc_page_fault+0x62/0x150
      [  246.651417]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
      [  246.651423] RIP: 0033:0x7fdef4cfd70b
      [  246.651430] RSP: 002b:00007ffcd60862a0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
      [  246.651434] RAX: ffffffffffffffda RBX: 000000000000007e RCX: 00007fdef4cfd70b
      [  246.651436] RDX: 0000000000004000 RSI: 00007ffcd6086330 RDI: 00000000ffffff9c
      [  246.651439] RBP: 00007ffcd6086330 R08: 0000000000000000 R09: 00007ffcd60861a0
      [  246.651442] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000004000
      [  246.651444] R13: 0000000000000009 R14: 000000000000097e R15: 0000000000000004
      [  246.651448]  </TASK>
      [  246.651451] INFO: task mdadm:1014 blocked for more than 122 seconds.
      [  246.651454]       Not tainted 5.14.0-472.el9.x86_64 #1
      [  246.651456] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      [  246.651457] task:mdadm           state:D stack:0     pid:1014  tgid:1014  ppid:1      flags:0x00000006
      [  246.651461] Call Trace:
      [  246.651463]  <TASK>
      [  246.651465]  __schedule+0x21b/0x540
      [  246.651473]  schedule+0x2e/0xd0
      [  246.651479]  schedule_preempt_disabled+0x11/0x20
      [  246.651486]  __mutex_lock.constprop.0+0x433/0x6a0
      [  246.651490]  ? __cgroup_bpf_check_dev_permission+0xf0/0x160
      [  246.651497]  bdev_open+0x218/0x3c0
      [  246.651504]  ? __pfx_blkdev_open+0x10/0x10
      [  246.651507]  blkdev_open+0x87/0xb0
      [  246.651511]  do_dentry_open+0x14f/0x440
      [  246.651516]  do_open+0x21a/0x450
      [  246.651523]  path_openat+0x111/0x280
      [  246.651530]  ? __mod_memcg_lruvec_state+0x76/0xc0
      [  246.651535]  do_filp_open+0xb2/0x160
      [  246.651540]  ? __check_object_size.part.0+0x47/0xd0
      [  246.651547]  do_sys_openat2+0x96/0xd0
      [  246.651553]  __x64_sys_openat+0x53/0xa0
      [  246.651559]  do_syscall_64+0x5f/0xf0
      [  246.651564]  ? post_alloc_hook+0xb6/0xd0
      [  246.651569]  ? get_page_from_freelist+0x441/0x650
      [  246.651576]  ? __mod_memcg_lruvec_state+0x76/0xc0
      [  246.651580]  ? __mod_lruvec_page_state+0x97/0x160
      [  246.651585]  ? folio_add_new_anon_rmap+0x44/0xe0
      [  246.651604]  ? do_anonymous_page+0x25a/0x410
      [  246.651611]  ? __handle_mm_fault+0x2fb/0x690
      [  246.651620]  ? __count_memcg_events+0x4f/0xb0
      [  246.651624]  ? mm_account_fault+0x6c/0x100
      [  246.651629]  ? handle_mm_fault+0x116/0x270
      [  246.651635]  ? do_user_addr_fault+0x1d6/0x6a0
      [  246.651639]  ? exit_to_user_mode_prepare+0xb9/0x100
      [  246.651646]  ? exc_page_fault+0x62/0x150
      [  246.651650]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
      [  246.651655] RIP: 0033:0x7fd7108fd70b
      [  246.651662] RSP: 002b:00007ffd873324c0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
      [  246.651665] RAX: ffffffffffffffda RBX: 00007ffd87334bc0 RCX: 00007fd7108fd70b
      [  246.651668] RDX: 0000000000000000 RSI: 00007ffd87336efc RDI: 00000000ffffff9c
      [  246.651671] RBP: 00007ffd87336efc R08: 00007ffd87334bc0 R09: 00007ffd87334930
      [  246.651673] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
      [  246.651675] R13: 00007ffd87334bc0 R14: 0000555868d222a0 R15: 0000000000000000
      [  246.651680]  </TASK>
      [  369.529499] INFO: task mdadm:963 blocked for more than 245 seconds.
      [  369.529514]       Not tainted 5.14.0-472.el9.x86_64 #1
      [  369.529521] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      [  369.529524] task:mdadm           state:D stack:0     pid:963   tgid:963   ppid:1      flags:0x00004006
      [  369.529536] Call Trace:
      [  369.529540]  <TASK>
      [  369.529546]  __schedule+0x21b/0x540
      [  369.529564]  schedule+0x2e/0xd0
      [  369.529571]  schedule_preempt_disabled+0x11/0x20
      [  369.529578]  __mutex_lock.constprop.0+0x433/0x6a0
      [  369.529585]  ? kernfs_add_one+0xf0/0x130
      [  369.529594]  bd_link_disk_holder+0x81/0x240
      [  369.529605]  bind_rdev_to_array+0x24b/0x350
      [  369.529617]  md_add_new_disk+0x292/0x500
      [  369.529622]  md_ioctl+0x955/0x970
      [  369.529629]  blkdev_ioctl+0x13e/0x270
      [  369.529638]  ? security_file_ioctl+0x32/0x50
      [  369.529646]  __x64_sys_ioctl+0x8a/0xc0
      [  369.529657]  do_syscall_64+0x5f/0xf0
      [  369.529665]  ? rcu_nocb_try_bypass+0x5e/0x460
      [  369.529675]  ? __pfx_inode_free_by_rcu+0x10/0x10
      [  369.529684]  ? fsnotify_grab_connector+0x49/0x80
      [  369.529692]  ? __pfx___d_free+0x10/0x10
      [  369.529697]  ? __call_rcu_common.constprop.0+0x117/0x2b0
      [  369.529703]  ? rcu_nocb_try_bypass+0x5e/0x460
      [  369.529709]  ? __pfx_file_free_rcu+0x10/0x10
      [  369.529717]  ? __call_rcu_common.constprop.0+0x117/0x2b0
      [  369.529723]  ? syscall_exit_to_user_mode+0x19/0x40
      [  369.529729]  ? do_syscall_64+0x6b/0xf0
      [  369.529733]  ? do_user_addr_fault+0x1d6/0x6a0
      [  369.529739]  ? do_syscall_64+0x6b/0xf0
      [  369.529744]  ? exc_page_fault+0x62/0x150
      [  369.529747]  entry_SYSCALL_64_after_hwframe+0x73/0x7b
      [  369.529756] RIP: 0033:0x7f5f9530357b
      [  369.529786] RSP: 002b:00007ffd35e9ced8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
      [  369.529791] RAX: ffffffffffffffda RBX: 0000559e375c4e90 RCX: 00007f5f9530357b
      [  369.529794] RDX: 00007ffd35e9d408 RSI: 0000000040140921 RDI: 0000000000000004
      [  369.529797] RBP: 0000559e375c5430 R08: 000000000000001e R09: 00007ffd35e9cc37
      [  369.529799] R10: 0000000000000002 R11: 0000000000000246 R12: 00007ffd35e9d0f0
      [  369.529802] R13: 00007ffd35e9db40 R14: 0000559e375c3e50 R15: 0000000000000004
      [  369.529807]  </TASK> 

            ncroxon@redhat.com Nigel Croxon
            yizhan@redhat.com Yi Zhang
            Nigel Croxon Nigel Croxon
            Fan Fan Fan Fan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: