-
Bug
-
Resolution: Duplicate
-
Undefined
-
None
-
None
-
None
-
True
-
-
False
-
?
-
None
-
-
The tests revolving around dedicated cpus are failing in whitebox are failing post update . Nova-scheduler indicates it cannot find candidates but the resource providers is showing available PCPUs and none are in use. But the scheduler is saying it would over subscribe if it tried to place the guest.
[zuul@controller ~]$ oc rsh openstackclientsh-5.1$ openstack resource provider list+--------------------------------------+--------------------------------+------------+--------------------------------------+----------------------+| uuid | name | generation | root_provider_uuid | parent_provider_uuid |+--------------------------------------+--------------------------------+------------+--------------------------------------+----------------------+| 611c310b-37ba-4e5f-87fe-916e753bf843 | compute-0.ctlplane.example.com | 485 | 611c310b-37ba-4e5f-87fe-916e753bf843 | None || 6eec71ed-299f-46d2-a4f8-4d6d34fa8220 | compute-1.ctlplane.example.com | 400 | 6eec71ed-299f-46d2-a4f8-4d6d34fa8220 | None |+--------------------------------------+--------------------------------+------------+--------------------------------------+----------------------+sh-5.1$ openstack resource provider inventory list 6eec71ed-299f-46d2-a4f8-4d6d34fa8220+----------------+------------------+----------+----------+----------+-----------+-------+------+| resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | used |+----------------+------------------+----------+----------+----------+-----------+-------+------+| MEMORY_MB | 1.0 | 1 | 15757 | 512 | 1 | 15757 | 0 || DISK_GB | 0.9 | 1 | 13 | 1 | 1 | 13 | 0 || PCPU | 1.0 | 1 | 4 | 0 | 1 | 4 | 0 || VCPU | 4.0 | 1 | 2 | 0 | 1 | 2 | 0 |+----------------+------------------+----------+----------+----------+-----------+-------+------+sh-5.1$ openstack resource provider inventory list 611c310b-37ba-4e5f-87fe-916e753bf843+----------------+------------------+----------+----------+----------+-----------+-------+------+| resource_class | allocation_ratio | min_unit | max_unit | reserved | step_size | total | used |+----------------+------------------+----------+----------+----------+-----------+-------+------+| MEMORY_MB | 1.0 | 1 | 15757 | 512 | 1 | 15757 | 0 || DISK_GB | 0.9 | 1 | 13 | 2 | 1 | 13 | 0 || PCPU | 1.0 | 1 | 4 | 0 | 1 | 4 | 0 || VCPU | 4.0 | 1 | 2 | 0 | 1 | 2 | 0 |+----------------+------------------+----------+----------+----------+-----------+-------+------+ nova-scheduler:2024-10-29 19:55:29.643 1 DEBUG nova.scheduler.filters [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] NUMATopologyFilter tries allocation candidate: {'allocations': {'611c310b-37ba-4e5f-87fe-916e753bf843': {'resources': {'DISK_GB': 1, 'MEMORY_MB': 256, 'PCPU': 1}}}, 'mappings': {'': ['611c310b-37ba-4e5f-87fe-916e753bf843']}} filter_candidates /usr/lib/python3.9/site-packages/nova/scheduler/filters/__init__.py:772024-10-29 19:55:29.643 1 DEBUG nova.virt.hardware [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Attempting to fit instance cell InstanceNUMACell(cpu_pinning_raw=None,cpu_policy='dedicated',cpu_thread_policy=None,cpu_topology=<?>,cpuset=set([]),cpuset_reserved=None,id=0,memory=256,pagesize=None,pcpuset=set([0])) on host_cell NUMACell(cpu_usage=0,cpuset=set([6]),id=1,memory=8016,memory_usage=0,mempages=[NUMAPagesTopology,NUMAPagesTopology,NUMAPagesTopology],network_metadata=NetworkMetadata,pcpuset=set([]),pinned_cpus=set([]),siblings=[set([6])],socket=None) _numa_fit_instance_cell /usr/lib/python3.9/site-packages/nova/virt/hardware.py:9342024-10-29 19:55:29.643 1 DEBUG nova.virt.hardware [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] No specific pagesize requested for instance, selected pagesize: 4 _numa_fit_instance_cell /usr/lib/python3.9/site-packages/nova/virt/hardware.py:9632024-10-29 19:55:29.644 1 DEBUG nova.virt.hardware [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Not enough host cell CPUs to fit instance cell; required: 1 + 0 as overhead, actual: 0 _numa_fit_instance_cell /usr/lib/python3.9/site-packages/nova/virt/hardware.py:10012024-10-29 19:55:29.644 1 DEBUG nova.virt.hardware [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Attempting to fit instance cell InstanceNUMACell(cpu_pinning_raw=None,cpu_policy='dedicated',cpu_thread_policy=None,cpu_topology=<?>,cpuset=set([]),cpuset_reserved=None,id=0,memory=256,pagesize=None,pcpuset=set([0])) on host_cell NUMACell(cpu_usage=0,cpuset=set([2]),id=0,memory=7741,memory_usage=256,mempages=[NUMAPagesTopology,NUMAPagesTopology,NUMAPagesTopology],network_metadata=NetworkMetadata,pcpuset=set([0]),pinned_cpus=set([0]),siblings=[set([0]),set([2])],socket=None) _numa_fit_instance_cell /usr/lib/python3.9/site-packages/nova/virt/hardware.py:9342024-10-29 19:55:29.644 1 DEBUG nova.virt.hardware [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] No specific pagesize requested for instance, selected pagesize: 4 _numa_fit_instance_cell /usr/lib/python3.9/site-packages/nova/virt/hardware.py:9632024-10-29 19:55:29.644 1 DEBUG nova.virt.hardware [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Instance has requested pinned CPUs _numa_fit_instance_cell /usr/lib/python3.9/site-packages/nova/virt/hardware.py:10282024-10-29 19:55:29.644 1 DEBUG nova.virt.hardware [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Not enough available CPUs to schedule instance. Oversubscription is not possible with pinned instances. Required: 1 (1 + 0), actual: 0 _numa_fit_instance_cell /usr/lib/python3.9/site-packages/nova/virt/hardware.py:10312024-10-29 19:55:29.644 1 DEBUG nova.scheduler.filters [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] NUMATopologyFilter rejected allocation candidate: {'allocations': {'611c310b-37ba-4e5f-87fe-916e753bf843': {'resources': {'DISK_GB': 1, 'MEMORY_MB': 256, 'PCPU': 1}}}, 'mappings': {'': ['611c310b-37ba-4e5f-87fe-916e753bf843']}} filter_candidates /usr/lib/python3.9/site-packages/nova/scheduler/filters/__init__.py:882024-10-29 19:55:29.645 1 DEBUG nova.scheduler.filters.numa_topology_filter [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] [instance: b4385854-40c8-43dc-8b9b-24ffc47c8c9a] compute-0.ctlplane.example.com, compute-0.ctlplane.example.com fails NUMA topology requirements. The instance does not fit on this host. host_passes /usr/lib/python3.9/site-packages/nova/scheduler/filters/numa_topology_filter.py:1162024-10-29 19:55:29.645 1 INFO nova.filters [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Filter NUMATopologyFilter returned 0 hosts2024-10-29 19:55:29.645 1 DEBUG nova.filters [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Filtering removed all hosts for the request with instance ID 'b4385854-40c8-43dc-8b9b-24ffc47c8c9a'. Filter results: [('AggregateInstanceExtraSpecsFilter', [('compute-1.ctlplane.example.com', 'compute-1.ctlplane.example.com'), ('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('ComputeFilter', [('compute-1.ctlplane.example.com', 'compute-1.ctlplane.example.com'), ('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('ComputeCapabilitiesFilter', [('compute-1.ctlplane.example.com', 'compute-1.ctlplane.example.com'), ('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('ImagePropertiesFilter', [('compute-1.ctlplane.example.com', 'compute-1.ctlplane.example.com'), ('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('ServerGroupAntiAffinityFilter', [('compute-1.ctlplane.example.com', 'compute-1.ctlplane.example.com'), ('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('ServerGroupAffinityFilter', [('compute-1.ctlplane.example.com', 'compute-1.ctlplane.example.com'), ('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('SameHostFilter', [('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('DifferentHostFilter', [('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('PciPassthroughFilter', [('compute-0.ctlplane.example.com', 'compute-0.ctlplane.example.com')]), ('NUMATopologyFilter', None)] get_filtered_objects /usr/lib/python3.9/site-packages/nova/filters.py:1142024-10-29 19:55:29.645 1 INFO nova.filters [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Filtering removed all hosts for the request with instance ID 'b4385854-40c8-43dc-8b9b-24ffc47c8c9a'. Filter results: ['AggregateInstanceExtraSpecsFilter: (start: 2, end: 2)', 'ComputeFilter: (start: 2, end: 2)', 'ComputeCapabilitiesFilter: (start: 2, end: 2)', 'ImagePropertiesFilter: (start: 2, end: 2)', 'ServerGroupAntiAffinityFilter: (start: 2, end: 2)', 'ServerGroupAffinityFilter: (start: 2, end: 2)', 'SameHostFilter: (start: 2, end: 1)', 'DifferentHostFilter: (start: 1, end: 1)', 'PciPassthroughFilter: (start: 1, end: 1)', 'NUMATopologyFilter: (start: 1, end: 0)']2024-10-29 19:55:29.645 1 DEBUG nova.scheduler.manager [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] Filtered [] _get_sorted_hosts /usr/lib/python3.9/site-packages/nova/scheduler/manager.py:6752024-10-29 19:55:29.645 1 DEBUG nova.scheduler.manager [None req-30eb1d4c-60d0-4b4e-a04c-26cbc2a278d3 cdb5cb13a0db4b6ea15ccc3b14d9d3a1 ab5ddd8288a64d93aee53c342bdeb484 - - default default] There are 0 hosts available but 1 instances requested to build. _ensure_sufficient_hosts /usr/lib/python3.9/site-packages/nova/scheduler/manager.py:494
Job link : https://gitlab.cee.redhat.com/ci-framework/ci-framework-testproject/-/merge_requests/794
Job logs : https://sf.apps.int.gpc.ocp-hub.prod.psi.redhat.com/zuul/t/components-integration/build/45bc3c31487846c680960a94663d1298
Whitebox Tests results pre update :
Whitebox Test results post-update :
- depends on
-
OSPRH-10772 nova-compute restart unpins pinned VMs if [libvirt]power_management=true.
-
- Backlog
-