Feature Request Overview
This RFP covers the implementation of PCIe device grouping logic within Cyborg. Nova today cannot currently guarantee that a set of PCIe devices (e.g., a GPU and its corresponding NIC) with ensured avoidance of overlapping of reporting the device group and contained separate devices during schedule request. This is a critical requirement for high-performance workloads (AI/ML, NFV). Implementing this intelligence in Cyborg provides a clean abstraction, preventing vendor-specific topology logic from polluting Nova.

Business justification
Allocating a PCI group to a VM instance without potential overlapping scheduling request of a PCI device contained in the group will increase VM scheduling determinism. 

Functional requirements 

• Develop a device profiling system within Cyborg that can discover and define groups of PCIe devices based on host topology (NUMA affinity, common upstream PCIe switch).
• Expose these device groups as allocatable resources via the Cyborg API 

Scope

• IN SCOPE:
  • Development within the Cyborg service using existing Nova/Cyborg API integration.
  • Topology discovery via sysfs.
  • Group creation logic.
  • Exposing groups via Cyborg's internal models and API.

• OUT OF SCOPE:
  • Nova scheduler modifications to understand groups (this is for a later phase).
  • Management of non-PCIe devices.
  • NUMA awareness for the groups on Cyborg side.

Acceptance Criteria

• Cyborg is able to discover, manage and provide PCI groups to Nova Placement for VM allocation.
• Management of the PCI groups is provided the standard way compatible with other management capabilities within OpenStack services.

Open Questions

• Devices provided by Cyborg are currently lacking NUMA awareness. With implementing PCI groups on Cyborg side we are loosing NUMA awareness of such devices
  • Without significant refactoring on Nova side (NUMA in Placement) this shortcoming can be overridden by implementation of weigher adding the NUMA awareness to the PCI group scheduling process, though minor change to Nova scheduler is required in this case. Question is if this acceptable solution on Nova side.