Loading...

Type: Bug
Resolution: Done-Errata
Priority: Undefined
Fix Version/s: rhel-9.4
Affects Version/s: rhel-9.3.0
Component/s: kernel / Virtualization / KVM
Labels:
None

Fixed in Build:
kernel-5.14.0-415.el9
Keywords:

ZStream, Regression

Pool Team:

sst_virtualization_hwe
Sub-System Group:

ssg_virtualization

Dev Target Milestone:
19
Internal Target Milestone:
23
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Products:

Red Hat Enterprise Linux
Release Blocker:
Approved Blocker
Target Backport Versions:

rhel-9.2.0.z, rhel-9.3.0.z

Preliminary Testing:
Pass
Testable Builds:

Hide
The following Merge Request has pipeline job artifacts available:

Title: kernel/panic.c: Move the location of bust_spinlocks to prevent hanging.
MR: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/3471
MR Pipeline: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/pipelines/1102903162

The Repo URLs are *not* accessible from a web browser! They only function as a dnf or yum baseurl.

Artifacts expire six weeks after creation. If artifacts are needed after that time please rerun the pipeline by visiting the MR's 'Pipelines' tab and clicking the 'Run pipeline' button.

----------------------------------------

Downstream Pipeline Name: RHEL_COMPAT (c9s_rhel9_compat_merge_request)
Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-internal-contributors/-/pipelines/1102903336
Repo URL: https://s3.upshift.redhat.com/DH-PROD-CKI/internal/1102903336/$basearch/5.14.0-397.3471_1102903162.el9.$basearch
Debug Repo URL: https://s3.upshift.redhat.com/DH-PROD-CKI/internal/1102903336/$basearch-debug/5.14.0-397.3471_1102903162.el9.$basearch

5.14.0-397.3471_1102903162.el9.s390x:
Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_s390x/5726662227/artifacts/repo/5.14.0-397.3471_1102903162.el9.s390x/

5.14.0-397.3471_1102903162.el9.aarch64:
Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_aarch64/5726662217/artifacts/repo/5.14.0-397.3471_1102903162.el9.aarch64/

5.14.0-397.3471_1102903162.el9.ppc64le:
Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_ppc64le/5726662211/artifacts/repo/5.14.0-397.3471_1102903162.el9.ppc64le/

5.14.0-397.3471_1102903162.el9.x86_64-debug:
Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_x86_64_debug/5726662203/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/

5.14.0-397.3471_1102903162.el9.x86_64:
Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_x86_64/5726662195/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/

----------------------------------------

Downstream Pipeline Name: REALTIME (c9s_rt_merge_request)
Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-trusted-contributors/-/pipelines/1102903274

5.14.0-397.3471_1102903162.el9.x86_64-debug:
Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903274/publish_x86_64_debug/5726661690/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/
Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903274/publish_x86_64_debug/5726661690/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/

5.14.0-397.3471_1102903162.el9.x86_64:
Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903274/publish_x86_64/5726661684/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/
Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903274/publish_x86_64/5726661684/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/

----------------------------------------

Downstream Pipeline Name: AUTOMOTIVE (c9s_automotive_check_merge_request)
Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-trusted-contributors/-/pipelines/1102903318

5.14.0-397.351.3471_1102903162.el9iv.aarch64:
Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903318/publish_aarch64/5726662112/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.aarch64/
Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903318/publish_aarch64/5726662112/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.aarch64/

5.14.0-397.351.3471_1102903162.el9iv.x86_64-debug:
Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903318/publish_x86_64_debug/5726662015/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/
Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903318/publish_x86_64_debug/5726662015/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/

5.14.0-397.351.3471_1102903162.el9iv.x86_64:
Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903318/publish_x86_64/5726662002/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/
Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903318/publish_x86_64/5726662002/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/

----------------------------------------

Downstream Pipeline Name: 64K (c9s_64k_merge_request)
Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-trusted-contributors/-/pipelines/1102903541

5.14.0-397.3471_1102903162.el9.aarch64:
Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903541/publish_aarch64/5726663851/artifacts/repo/5.14.0-397.3471_1102903162.el9.aarch64/
Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903541/publish_aarch64/5726663851/artifacts/repo/5.14.0-397.3471_1102903162.el9.aarch64/

----------------------------------------

Updated 2023-12-11 18:14 UTC by buglinker:
- KWF FAQ: https://red.ht/kernel_workflow_doc
- Slack #team-kernel-workflow: https://redhat-internal.slack.com/archives/C04LRUPMJQ5
- Source: https://gitlab.com/cki-project/kernel-workflow/-/blob/main/webhook/buglinker.py
- Documentation: https://gitlab.com/cki-project/kernel-workflow/-/blob/main/docs/README.buglinker.md
- Report an issue: https://gitlab.com/cki-project/kernel-workflow/-/issues/new?issue%5Btitle%5D=buglinker%20webhook%20issue

Show
The following Merge Request has pipeline job artifacts available: Title: kernel/panic.c: Move the location of bust_spinlocks to prevent hanging. MR: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/3471 MR Pipeline: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/pipelines/1102903162 The Repo URLs are *not* accessible from a web browser! They only function as a dnf or yum baseurl. Artifacts expire six weeks after creation. If artifacts are needed after that time please rerun the pipeline by visiting the MR's 'Pipelines' tab and clicking the 'Run pipeline' button. ---------------------------------------- Downstream Pipeline Name: RHEL_COMPAT (c9s_rhel9_compat_merge_request) Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-internal-contributors/-/pipelines/1102903336 Repo URL: https://s3.upshift.redhat.com/DH-PROD-CKI/internal/1102903336/$basearch/5.14.0-397.3471_1102903162.el9.$basearch Debug Repo URL: https://s3.upshift.redhat.com/DH-PROD-CKI/internal/1102903336/$basearch-debug/5.14.0-397.3471_1102903162.el9.$basearch 5.14.0-397.3471_1102903162.el9.s390x: Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_s390x/5726662227/artifacts/repo/5.14.0-397.3471_1102903162.el9.s390x/ 5.14.0-397.3471_1102903162.el9.aarch64: Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_aarch64/5726662217/artifacts/repo/5.14.0-397.3471_1102903162.el9.aarch64/ 5.14.0-397.3471_1102903162.el9.ppc64le: Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_ppc64le/5726662211/artifacts/repo/5.14.0-397.3471_1102903162.el9.ppc64le/ 5.14.0-397.3471_1102903162.el9.x86_64-debug: Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_x86_64_debug/5726662203/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/ 5.14.0-397.3471_1102903162.el9.x86_64: Artifacts (RPMs): https://s3.upshift.redhat.com/DH-PROD-CKI/index.html?prefix=internal-artifacts/1102903336/publish_x86_64/5726662195/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/ ---------------------------------------- Downstream Pipeline Name: REALTIME (c9s_rt_merge_request) Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-trusted-contributors/-/pipelines/1102903274 5.14.0-397.3471_1102903162.el9.x86_64-debug: Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903274/publish_x86_64_debug/5726661690/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/ Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903274/publish_x86_64_debug/5726661690/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/ 5.14.0-397.3471_1102903162.el9.x86_64: Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903274/publish_x86_64/5726661684/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/ Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903274/publish_x86_64/5726661684/artifacts/repo/5.14.0-397.3471_1102903162.el9.x86_64/ ---------------------------------------- Downstream Pipeline Name: AUTOMOTIVE (c9s_automotive_check_merge_request) Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-trusted-contributors/-/pipelines/1102903318 5.14.0-397.351.3471_1102903162.el9iv.aarch64: Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903318/publish_aarch64/5726662112/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.aarch64/ Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903318/publish_aarch64/5726662112/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.aarch64/ 5.14.0-397.351.3471_1102903162.el9iv.x86_64-debug: Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903318/publish_x86_64_debug/5726662015/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/ Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903318/publish_x86_64_debug/5726662015/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/ 5.14.0-397.351.3471_1102903162.el9iv.x86_64: Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903318/publish_x86_64/5726662002/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/ Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903318/publish_x86_64/5726662002/artifacts/repo/5.14.0-397.351.3471_1102903162.el9iv.x86_64/ ---------------------------------------- Downstream Pipeline Name: 64K (c9s_64k_merge_request) Downstream Pipeline: https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki-trusted-contributors/-/pipelines/1102903541 5.14.0-397.3471_1102903162.el9.aarch64: Artifacts (RPMs): https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/index.html?prefix=trusted-artifacts/1102903541/publish_aarch64/5726663851/artifacts/repo/5.14.0-397.3471_1102903162.el9.aarch64/ Repo URL: https://s3.amazonaws.com/arr-cki-prod-trusted-artifacts/trusted-artifacts/1102903541/publish_aarch64/5726663851/artifacts/repo/5.14.0-397.3471_1102903162.el9.aarch64/ ---------------------------------------- Updated 2023-12-11 18:14 UTC by buglinker: - KWF FAQ: https://red.ht/kernel_workflow_doc - Slack #team-kernel-workflow: https://redhat-internal.slack.com/archives/C04LRUPMJQ5 - Source: https://gitlab.com/cki-project/kernel-workflow/-/blob/main/webhook/buglinker.py - Documentation: https://gitlab.com/cki-project/kernel-workflow/-/blob/main/docs/README.buglinker.md - Report an issue: https://gitlab.com/cki-project/kernel-workflow/-/issues/new?issue%5Btitle%5D=buglinker%20webhook%20issue

Experience:
Architecture:

x86_64

SFDC Cases Counter:
SFDC Cases Links:

Run this command. You will need to adjust the path to the kernel:

/usr/libexec/qemu-kvm -no-user-config -nodefaults -display none \
    -machine accel=kvm:tcg,graphics=off -cpu max,la57=off -m 1024 \
    -no-reboot -kernel /boot/vmlinuz-5.14.0-349.el9.x86_64 \
    -serial file:/dev/stdout \
    -append 'panic=1 console=ttyS0 printk.time=1'

It will hang at:

[    0.604230] Call Trace:
[    0.604235]  <TASK>
[    0.604237]  dump_stack_lvl+0x34/0x48
[    0.604242]  panic+0xea/0x2e4
[    0.604246]  mount_block_root+0x28c/0x29f
[    0.604251]  prepare_namespace+0x13b/0x16a
[    0.604254]  kernel_init_freeable+0x17d/0x1a2
[    0.604257]  ? __pfx_kernel_init+0x10/0x10
[    0.604261]  kernel_init+0x16/0x130
[    0.604263]  ret_from_fork+0x29/0x50
[    0.604268]  </TASK>

What's supposed to happen is the VM reboots (causing qemu to exit because of the -no-reboot option):

[    2.668471] tsc: Refined TSC clocksource calibration: 2207.933 MHz
[    2.670026] Kernel Offset: 0x5800000 from 0xffffffff81000000 (relocation ran)
[    2.670891] Rebooting in 1 seconds..

Note this bug is somewhat random ...