An API to install and distribute cloud native bundle content (RukPak)

Customer Problem: Operator Management UX

Distributing, installing and updating arbitrary content onto a cluster is a core problem of problem of any package manager. OLM has a number of these generic problems (how to be aware of specific install content, how to pivot between updates, etc) as well as a set of problems specific to managing content installs and updates on a Kubernetes cluster (declarative, asynchronous, managing state in a distributed way). 

Goal: Provide a set of underlying APIs as building blocks to create a first class component that is concerned with installing and updating bundles onto a cluster – Tentatively referred to upstream as RukPak, which will comprise low level APIs for Bundles, Instances, and ProvisionerClasses

Problem: Historically, OLM was designed with a very opinionated way to define a cloud native application (Operator) that has a number of limitations about how to package and bundle your application. Originally, this meant that an "Operator" was defined as a Cluster Service Version(CSV) and a set of CRDs. The packaging format of applications was expanded to be defined as a versioned Bundle object, but the on cluster OLM experience was only trivially expanded to bolt on install functionality for bundles. Now, we need to deliver a first class solution for defining bundle installation on cluster as a first class object that independent controllers can manage and reason about outside the scope of the InstallPlan API that munges many of these ideas together. Providing these first class APIs will allow us to build a number of core features and create user stories that will allow our higher order controllers and APIs to actually resolve some of the cluster admin and operator author requirements that are stated in https://issues.redhat.com/browse/OLM-1579

Dependencies (internal and external):

• n/a

Prioritized deliverables (in scope / not in scope):

1. Define a Bundle API, which represents the content of a bundle that can be applied to a cluster. This content is not installed, but defines the manifests and content that is used to install or pivot to an active installation. (in scope)
2. Define an Instance API, which represents an actively installed bundle on a cluster. (in scope)
3. Define a ProvisionerClass API, which represents a configuration for a particular Provisioner – a controller that understands Bundles and Instances of a particular type. The ProvisionerClass allows us to define arbitrary controllers that react to specific bundle types. While today we only have a single bundle type (registry+v1), this abstraction aims at defining a future where other components can use multiple types (registry+v2, helm, raw kube manifests, etc) (in scope)
4. Enable defining approval workflows for Provisioners via rbac. (in scope)
5. Bundle content in this form in a trivial implementation would be restricted to a single object in etcd. We already have cases where bundles are not able to fit in a single configmap, so we should ensure that bundles of a larger size can be unpacked.
6. Unpacked content is continuously reconciled, i.e. if a resource extracted from a bundle instance is modified or delete, those changes are rolled back autonomously as part of OLM reconciliation

Note:

• bullet 1 to 5 above are completed (hence crossed out).
• bullet 6 above is partially completed and need further discussions.

High-level acceptance criteria:

• must support bundles of arbitrary sizes
• must not prevent non-cluster admin persons to utilize this API
• must support a fully declarative approach (GitOps-friendliness), i.e. continuous reconciliation even in failure states
• must support max bundle sizes larger than OLM's current registry+v1 limit

Note:

• these 4 bullets are completed.

Cases where bundles > 1MB:

• CNV
• Redis
• Prometheus

Need to make sure we surface any API server warnings. 
 

Estimate (XS, S, M, L, XL, XXL): L