Epic Goal

In https://issues.redhat.com/browse/OSDK-2588 two designs were proposed for the new OLM v2 CatalogSource arch. A third option, one that uses Open Container Initiative (OCI) registries to store and release fbc content, is being looked at as another viable option. This epic will track the spike+design+Poc of the third option.  

WHY is this important? 

Using OCI artifacts to store and share catalogs, packages, channels, and bundles sounds like a natural fit. We think using cloud-native storage for cloud-native content such like bundle manifests etc makes a lot of sense. There could be many benefits, but here are a few:

1. Different catalogs can reference the same packages, channels can reference the same bundles, bundles can reference the same container images (e.g. sidecar images, operand images, etc.). OCI registries are very good at de-duplicating these shared artifacts, which makes them optimal for this use case where lots of sharing occurs
2. Actions like channel promotion don't involve a complete rebuild of the catalog. You would push a single channel artifact update with the new bundle reference, a new package artifact update with the updated channel reference, and a new catalog artifact with the updated package reference. Only a very small sliver of the overall catalog would change, which means less bandwidth require to push the change and less bandwidth required for users to pull or mirror the change.
3. There's no concern about a mismatch between what's in the catalog and what's in the bundle. The catalog is simply the root node of a directed acyclic graph of artifacts, where any change must be reflected back to the root to have an affect. Immutability is built-in.
4. Identity is easy. Each artifact has a digest-based OCI artifact reference.
5. Clients don't have to fetch the entire catalog, packages, channels, bundles, and container images. They can instead query the registry for just what they need.
6. Using OCI artifact and nesting constructs make mirroring a simple, straightforward process using off the shelf OCI-compliant tools.
7. We no longer have to ship binaries around in catalog images that are currently require to serve the content from those images. In many cases, those binaries are larger than the catalog itself.

Done Checklist

• Design doc the outlines workflow of using OCI registries to store and release fbc catalogs, and how they're used on cluster
• PoC that demos the design.