Uploaded image for project: 'Operator Runtime'
  1. Operator Runtime
  2. OPRUN-2198

Differential Operator catalog updates for continuous mirroring

XMLWordPrintable

    • 0

      Story: As a cluster-admin attempting to do a disconnected install of OpenShift I would like to set up a cronjob with my mirroring configuration so that I keep my mirror updated by continuously downloading whatever Operators may have been published since the last sync while keeping the amount of data that needs downloading low.

      Side notes:

      • While we want this initially in opm it should be exposed in shareable code libraries so that oc can vendor in the logic easily.
      • Filter logic is covered by OLM-2282.

      Background: It's not only important to cut down the amount of data downloaded from Operator catalogs at "Day 1" but also mirror incrementally the difference to keep the download length/volume low at "Day 2". Here is the rough sequence of events:

      1. User initially mirrors from the upstream catalog, potentially with filters defined that are defined in this epic, e.g. only channel-heads
      2. Users re-runs the same mirroring job with the same or updated filters, either interactively or periodically with a system cronjob, this time also references the mirrored catalog created in the previous step
      3. filtering logic figures out the difference between the catalog contents resulting from the initial mirroring run and the potentially updated upstream catalog, e.g. newer channel-heads
      4. Bundles already mirrored are not synced again
      5. Bundles that fit the filter criteria are downloaded plus all necessary bundles to preserve the update path, e.g. current channel-heads and all operators in between this one and the channel heads in the mirrored catalog
      6. The result is an updated mirror catalog that contains the content of the previous mirror catalog and the incremental difference

      Acceptance criteria:

      • a shareable library that is implemented / used internally in opm and can later on be shared with oc
      • this behavior is only available when either no filter is applied at all, or when filtering by package name, channel names or channel-heads is employed
      • input is an existing published "upstream" catalog with a floating image tag that will be updated over time with references to newer Operator versions
      • a second input is "mirror" catalog that was created in an initial/previous mirroring operation at some point in the past
      • the filtering logic should take into account which bundles are available in the existing mirror catalog vs. the potentially updated upstream catalog while preserving the Operator upgrade path, this requires determining the differential set of Operator that need to be present for installed Operators in disconnected clusters to "walk" this update path
      • if filtering for channels-heads (heads-only mode) the differential is the channel head in each channel of the Operator in the upstream catalog and all the bundles along the update path to reach all the channel heads in the mirror catalog or, if they don't exist, just the channel head(s) in the upstream catalog
      • the library is built into the opm CLI as opm alpha diff

      Open Questions:

      • pruning the catalog this way will likely conflict with catalog validation logic in opm - how do we deal with that? do we relax the particular criteria about missing replaces? do we disable validation?
         

            estroczy@redhat.com Eric Stroczynski (Inactive)
            DanielMesser Daniel Messer
            Xia Zhao Xia Zhao
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: