Description and Summary:

This Jira ticket outlines the necessity for initial smoke screen testing of the rteval container in preparation for further extensive testing in the 9.6 development cycle. The immediate aim is to establish a base test case for the initial release of the rteval container, determine the benchmarking procedures, and analyze the initial benchmark results within rhel-9.5. Future considerations will involve automating this test to facilitate ease of repetition, but this will be handled by a separate ticket.

Base Case Definition:
The core scenario for testing involves deploying a privileged container on a bare-metal host configured to access all available host resources fully. This setup is designed to closely mimic running rteval directly on the host, with the container runtime being the only differing factor. This method is intended to accurately measure any latency that the containerization process may introduce.

Testing Procedure:
The testing will be bifurcated into two distinct series based on CPU partitioning configurations to adequately assess performance under varied conditions:

1. Without CPU Partitioning on a Single NUMA Node Machine: This series will benchmark rteval's performance in the absence of CPU partitioning, establishing a control for performance comparison and understanding the impact of partitioning on performance. **
2. With CPU Partitioning on a 2 NUMA Node Machine: This will explore rteval's performance when CPU partitioning is implemented. We will use the tuned realtime profile with with the first cpu per NUMA node used for housekeeping. Given observed potential performance declines in non-partitioned setups on two node systems, we will need to use two seperate machines so that the results are not skewed. One machine with only one NUMA node. And one machine with two NUMA nodes.

In both configurations, tests will be conducted in both bare-metal environments and within Podman containers. The initial cpu partitioning benchmarks will utilize isolcpus and Podman cpusets, with a possible shift towards employing cgroupsv2 cpuset exclusive flag in the future.

Each testing scenario will consist of four three-hour runs of rteval, with the outcomes averaged to assess and compare performance. The expectation is to observe negligible differences in performance between the bare-metal and containerized executions, particularly in the partitioned setups, thereby indicating minimal latency introduced by containerization.

Feature:
Initial smoke screen testing of the rteval container for benchmarking and performance analysis.

Reason:
To define a standardized base test case for the rteval container's initial release, facilitating the measurement of any potential latency introduced by containerization. This is crucial for ensuring that the rteval container remains efficient and reliable across various deployment scenarios.

Result:
Completion of this testing will result in a comprehensive benchmark report detailing the performance of the rteval container in rhel-9.5, under both partitioned and non-partitioned CPU configurations on bare-metal hosts and within containers. This will guide future optimizations and provide a baseline for the automation of subsequent testing processes, aligning with the broader goal of enhancing the usability and reliability of real-time tools in containerized environments.