Epic Goal

The goal of this epic is to provide the user with the ability of specifying a sidecar container to their plugin scheduler deployed with the Secondary Scheduler Operator.

Why is this important?

As of today, the Secondary Scheduler Operator creates the pod for the scheduler plugin and the user can customize the command to be executed by the main container. Some schedulers might have a sidecar container, or even an init container, and the Operator does not allow this level of customization. Changes to the container and container list in a pod cannot be done when the pod is running (e.g., cannot use edit pod-name).

Scenarios

1. User wants to a scheduler plugin that has sidecar container. An example is KubeFlux. The YAML file is attached
2. The sidecar container needs to configure image and command, although command can be optional. Possibly, other values might needed to be passed to the container spec (i.e., volumeMounts)
3. The sidecar container may interact with the secondary scheduler in two main ways
   1. by direct API calls (e.g., gRPC), either as a client or a server, without requiring any connection with other pods or services outside of the cluster
   2. as a service to the secondary scheduler, and requiring connection to other pods or services. An example can be a sidecar container running an informer to some API object
4. If the sidecar needs access to API objects, then the relevant RBAC rules might needed if they are not already fulfilled by the rules implemented for the secondary scheduler (i.e., Nodes, Pods). For example, the KubeFlux's sidecar container connects to the cluster and uses a Node lister. Another possible example is getting access to CRDs or configmaps 

Acceptance Criteria

• CI - MUST be running successfully with tests automated
• Release Technical Enablement - Provide necessary release enablement details and documents.

Dependencies (internal and external)

1. ...

Previous Work (Optional):

1. …

Open questions::

1. …

Done Checklist

• CI - CI is running, tests are automated and merged.
• Release Enablement <link to Feature Enablement Presentation>
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub Issue>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub Issue>
• DEV - Downstream build attached to advisory: <link to errata>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Downstream documentation merged: <link to meaningful PR>