-
Bug
-
Resolution: Done-Errata
-
Major
-
rhos-18.0 FR 2 (Mar 2025)
-
None
-
2
-
False
-
-
False
-
?
-
kernel-5.14.0-427.60.1.el9_4
-
rhos-dfg-nfv
-
Yes
-
-
Release Note Not Required
-
Done
-
-
-
Rejected
-
NFV 007
-
1
-
Important
To Reproduce Steps to reproduce the behavior:
- Deploy RHOSO 18 using NFV job. I used tigon28 and a job based on uni08theta-rhel9-rhoso18.0-nfv-ovs-dpdk-sriov-trunk-patches (downstream) though the issue is reproduced also when deploying environment with upsream containers. The issue happened also on other environments
- edpm deployment fails on installing bootstrap-edpm-deployment-openstack-edpm-XXXXX pod[1]
- There is kernel trace on compute nodes, see [2]below
- sudo systemctl status tuned reports that service is being deactivating
Expected behavior
No tuned related failures when installing bootstrap-edpm-deployment-openstack-edpm-XXXXX pod while deploying edpm.
Device Info (please complete the following information):
-
- Environment used tigon28.lab.eng.tlv2.redhat.com
- OS on compute nodes rhel-9.4
- openstack version 18.0.4-trunk-20250305.1
- tuned version tuned-2.25.1-1.1.20250203git889387b0.el9fdp.noarch
Bug impact
- Deploying edpm is not possible without changing tuned profile
Known workaround
- changing edpm_tuned_profile: cpu-partitioning-powersave to edpm_tuned_profile: cpu-partitioning seems to work
Additional context
- [1]
zuul@controller-0 ~]$ oc logs -f bootstrap-edpm-deployment-openstack-edpm-65n6l -n openstack | tail -n 10 fatal: [compute-1]: FAILED! => {"changed": false, "cmd": ["/usr/sbin/tuned-adm", "profile", "cpu-partitioning-powersave"], "delta": "0:10:00.933740", "end": "2025-03-11 11:47:48.084486", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2025-03-11 11:37:47.150746", "stderr": "", "stderr_lines": [], "stdout": "Operation timed out after waiting 600 seconds(s), you may try to increase timeout by using --timeout command line option or using --async.", "stdout_lines": ["Operation timed out after waiting 600 seconds(s), you may try to increase timeout by using --timeout command line option or using --async."]} fatal: [compute-0]: FAILED! => {"changed": false, "cmd": ["/usr/sbin/tuned-adm", "profile", "cpu-partitioning-powersave"], "delta": "0:10:00.929367", "end": "2025-03-11 11:47:48.077962", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2025-03-11 11:37:47.148595", "stderr": "", "stderr_lines": [], "stdout": "Operation timed out after waiting 600 seconds(s), you may try to increase timeout by using --timeout command line option or using --async.", "stdout_lines": ["Operation timed out after waiting 600 seconds(s), you may try to increase timeout by using --timeout command line option or using --async."]}
- [2]There is kernel trace on compute nodes:
[ 1475.696162] INFO: task tuned:27202 blocked for more than 122 seconds. [ 1475.696481] Not tainted 5.14.0-427.59.1.el9_4.x86_64 #1 [ 1475.696803] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1475.697148] task:tuned state:D stack:0 pid:27202 ppid:1 flags:0x00000006 [ 1475.697496] Call Trace: [ 1475.697834] <TASK> [ 1475.698171] __schedule+0x21b/0x550 [ 1475.698498] schedule+0x2d/0x70 [ 1475.698823] schedule_preempt_disabled+0x11/0x20 [ 1475.699155] rwsem_down_read_slowpath+0x37f/0x4f0 [ 1475.699486] down_read+0x45/0xa0 [ 1475.699814] show+0x32/0x90 [ 1475.700137] sysfs_kf_seq_show+0x98/0x100 [ 1475.700455] seq_read_iter+0x11d/0x4b0 [ 1475.700767] ? selinux_file_permission+0x108/0x150 [ 1475.701073] vfs_read+0x1e6/0x330 [ 1475.701390] ksys_read+0x5f/0xe0 [ 1475.701684] do_syscall_64+0x59/0x90 [ 1475.701981] ? clear_bhb_loop+0x35/0x90 [ 1475.702291] ? clear_bhb_loop+0x35/0x90 [ 1475.702583] ? clear_bhb_loop+0x35/0x90 [ 1475.702873] ? clear_bhb_loop+0x35/0x90 [ 1475.703161] entry_SYSCALL_64_after_hwframe+0x77/0xe1 [ 1475.703442] RIP: 0033:0x7f36bbafd9ec [ 1475.703730] RSP: 002b:00007f36ba3d3b30 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 [ 1475.704023] RAX: ffffffffffffffda RBX: 00007f36ba3d65c0 RCX: 00007f36bbafd9ec [ 1475.704332] RDX: 0000000000001001 RSI: 00007f36b4046bc0 RDI: 000000000000000b [ 1475.704633] RBP: 0000000000001001 R08: 0000000000000000 R09: 0000000000000000 [ 1475.704932] R10: 0000000008000000 R11: 0000000000000246 R12: 00007f36b9b5d100 [ 1475.705236] R13: 00007f36b4046bc0 R14: 000000000000000b R15: 000055e8635feb70 [ 1475.705535] </TASK>
- relates to
-
OSPRH-14899 Temporarily WA powersave tuned profile issue for CI gate
-
- Closed
-
- links to
-
RHBA-2025:147956 Release of containers for RHOSO 18.0