Uploaded image for project: 'OpenShift Cloud'
  1. OpenShift Cloud
  2. OCPCLOUD-3342

Implement disconnected support for provider image fetching

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • None
    • None
    • None

       Description:

        The pkg/providerimages/ package fetches container images at startup to extract CAPI provider manifests. It currently pulls directly from the
        canonical registry in the image reference (e.g. registry.ci.openshift.org/...). On disconnected clusters, these registries are unreachable — images
        are served from local mirrors configured via cluster resources. This must be fixed before GA.

        Solution:

        Two changes: resolve image references through registry mirrors, and trust the cluster's additional CA certificates.

        Mirror resolution:

        Two OpenShift APIs configure mirrors:
        - ImageDigestMirrorSet (IDMS) — config.openshift.io/v1, stable API. Already registered in the operator's scheme.
        - ImageContentSourcePolicy (ICSP) — operator.openshift.io/v1alpha1, deprecated but still present on older clusters. Requires registering
        operatorv1alpha1 in the scheme.

        Resolution is string prefix replacement: given a source quay.io/openshift mapped to mirror mirror.local/openshift, the image ref
        quay.io/openshift/capi-aws@sha256:abc becomes mirror.local/openshift/capi-aws@sha256:abc. When multiple sources match, the longest prefix (most
        specific) wins. The first mirror in a source's mirror list is used.

        Custom CA trust:

        remote.Image() makes HTTPS calls from the operator pod, bypassing the node-level trust store that CRI-O uses. Mirror registries on disconnected
        clusters typically use certs signed by an internal CA not in the pod's system cert pool. CAs are read from two sources:
        - openshift-config-managed/trusted-ca-bundle — the CNO-managed merged bundle containing system CAs plus install-time CAs from additionalTrustBundle
        (via proxy.config). This is the primary source; AWS and agent-based disconnected installs configure CAs here, not in image.config.
        - image.config.openshift.io/cluster additionalTrustedCA — a Day 2 field for registry-specific CAs that may not be in the proxy trust chain.

        Both are merged into a single cert pool. The transport is cloned from go-containerregistry's DefaultTransport with the custom TLS config overlaid.
        Non-NotFound API errors (RBAC, timeouts) are propagated rather than silently degraded.

        Changes:
        - New pkg/providerimages/mirrors.go: list IDMS/ICSP resources, build map[string][]string (source→mirrors), resolve refs via longest-prefix match.
        Uses meta.IsNoMatchError to gracefully skip CRDs that aren't installed.
        - New pkg/providerimages/trustedca.go: read CA bundles from both sources, build cert pool, clone DefaultTransport with custom TLS config.
        - Modify ReadProviderImages: after pull secret setup, resolve all image refs through mirrors, build custom transport with additional CAs, pass both
        to remoteImageFetcher. Short-circuits when no mirrors or CAs are configured (common case on connected clusters — no behavior change).
        - Register operatorv1alpha1 scheme in cmd/capi-operator/main.go for ICSP support.

        Scope notes:
        - Only digest-based refs are supported (we control these). ImageTagMirrorSet (ITMS) is out of scope.
        - No public API changes — ReadProviderImages signature is unchanged.
        - Mirror auth is handled by the existing pull secret keychain (resolves per-registry).
        - A future enhancement (decorator with multi-mirror fallback) is documented in the research but not needed for initial implementation.

        Acceptance criteria:
        - On disconnected clusters with IDMS/ICSP configured, provider images are pulled from mirror registries
        - On disconnected clusters with internal CA, registry TLS handshake succeeds using CAs from trusted-ca-bundle and/or additionalTrustedCA
        - On connected clusters with no mirrors or additional CAs, behavior is unchanged
        - Unit tests cover: prefix matching, longest-prefix wins, IDMS+ICSP merge, graceful handling of missing CRDs, CA loading from both sources, mixed
        valid/invalid PEM, error propagation for non-NotFound API errors

              rh-ee-tbarberb Theo Barber-Bany
              rhn-gps-mbooth Matthew Booth
              None
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: