Uploaded image for project: 'OpenShift Pipelines'
  1. OpenShift Pipelines
  2. SRVKP-8880

Implement resolver caching per TEP 0161

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Hide
      ## Resolver Caching for Bundle, Git, and Cluster Resolvers
      Support caching for bundle, git, and cluster resolvers, reducing redundant fetches and improving pipeline performance.

      **Key Features:**
      - **Automatic caching** for immutable references (digest-based bundles, git SHAs)
      - **Three cache modes**: `always` (cache everything), `never` (disable caching),`auto` (cache only immutable references - default)
      - **Configurable via ConfigMap**: Set cache size and TTL without restarting controllers
      - **Per-task override**: Tasks can override global cache settings using the `cache` parameter
      - **Observability**: Cache hits/misses and timestamps added to resource annotations

      **Configuration:**
      Control cache behavior via the `resolver-cache-config` ConfigMap with `max-size` (default: 1000) and `ttl` (default: 5m). Cache mode can be set globally in resolver ConfigMaps or per-task.

      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: resolver-cache-config
      data:
        max-size: "1000" # Maximum cache entries (default: 1000)
        ttl: "5m" # Cache entry lifetime (default: 5m)

      Set cache mode in resolver ConfigMaps (bundleresolver-config, git-resolver-config, cluster-resolver-config):

      data:
        cache: "auto" # Options: always, never, auto (default: auto)

      Override cache behavior per-task using the cache parameter:

      params:
        - name: cache
          value: "always" # Force caching for this task

      Cache annotations are added to resolved resources for observability.

      This helps reduce external API calls, improves pipeline exec speed, and provides better resilience during remote resource resolution.
      Show
      ## Resolver Caching for Bundle, Git, and Cluster Resolvers Support caching for bundle, git, and cluster resolvers, reducing redundant fetches and improving pipeline performance. **Key Features:** - **Automatic caching** for immutable references (digest-based bundles, git SHAs) - **Three cache modes**: `always` (cache everything), `never` (disable caching),`auto` (cache only immutable references - default) - **Configurable via ConfigMap**: Set cache size and TTL without restarting controllers - **Per-task override**: Tasks can override global cache settings using the `cache` parameter - **Observability**: Cache hits/misses and timestamps added to resource annotations **Configuration:** Control cache behavior via the `resolver-cache-config` ConfigMap with `max-size` (default: 1000) and `ttl` (default: 5m). Cache mode can be set globally in resolver ConfigMaps or per-task. apiVersion: v1 kind: ConfigMap metadata:   name: resolver-cache-config data:   max-size: "1000" # Maximum cache entries (default: 1000)   ttl: "5m" # Cache entry lifetime (default: 5m) Set cache mode in resolver ConfigMaps (bundleresolver-config, git-resolver-config, cluster-resolver-config): data:   cache: "auto" # Options: always, never, auto (default: auto) Override cache behavior per-task using the cache parameter: params:   - name: cache     value: "always" # Force caching for this task Cache annotations are added to resolved resources for observability. This helps reduce external API calls, improves pipeline exec speed, and provides better resilience during remote resource resolution.
    • Feature
    • Proposed

      < High-Level description of the feature ie: Executive Summary >

      Summary
      This TEP proposes adding caching capabilities to Tekton's remote resolvers to improve performance and reduce occurences of failed pipelines due to rate limiting by registries and git forges. The caching mechanism will be configurable per resolver type and will support different caching strategies based on the nature of the resolved resource.

      Motivation
      Avoid failed resolutions due to external services imposing rate limiting
      Reduce latency for frequently accessed resources
      Minimize external API calls and network traffic
      Improve reliability by serving cached content when external services are briefly unavailable
      Reduce load on external services (GitHub, OCI registries, etc.)

      see the TEP for more info

      Requirements

      Requirements Notes IS MVP
           

        • (Optional) Use Cases

      < What are we making, for who, and why/what problem are we solving?>

      Out of scope

      <Defines what is not included in this story>

      Dependencies

      < Link or at least explain any known dependencies. >

      Background, and strategic fit

      < What does the person writing code, testing, documenting need to know? >

      Assumptions

      < Are there assumptions being made regarding prerequisites and dependencies?>

      < Are there assumptions about hardware, software or people resources?>

      Customer Considerations

      < Are there specific customer environments that need to be considered (such as working with existing h/w and software)?>

      Documentation Considerations

      < What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)? >

      What does success look like?

      < Does this feature have doc impact? Possible values are: New Content, Updates to existing content, Release Note, or No Doc Impact?>

      QE Contact

      < Are there assumptions being made regarding prerequisites and dependencies?>

      < Are there assumptions about hardware, software or people resources?>

      Impact

      < If the feature is ordered with other work, state the impact of this feature on the other work>

      Related Architecture/Technical Documents

      <links>

      Done Checklist

      • Acceptance criteria are met
      • Non-functional properties of the Feature have been validated (such as performance, resource, UX, security or privacy aspects)
      • User Journey automation is delivered
      • Support and SRE teams are provided with enough skills to support the feature in production environment

              rh-ee-vbobade Vibhav Bobade
              brianwcook Brian Cook
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: