Uploaded image for project: 'OpenShift Pipelines'
  1. OpenShift Pipelines
  2. SRVKP-10944

Add GetFiles() caching for Gitea/Forgejo provider to prevent linear API call scaling

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Minor Minor
    • Pipelines 1.23.0
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Implemented caching for Gitea provider changed files to reduce redundant API calls during reconciliation, improving performance and aligning with existing provider patterns.
    • Release Note Not Required

      Story (Required)

      As a Pipelines-as-Code user on Gitea/Forgejo trying to use file-based CEL triggers across multiple PipelineRuns I want GetFiles() results to be cached per webhook event so that pipeline matching completes in seconds rather than scaling linearly with PipelineRun count.

      The Gitea/Forgejo provider is missing the GetFiles() caching that already exists for GitHub, GitLab, and Bitbucket Data Center. This causes repeated identical API calls for every PipelineRun evaluation, leading to severe performance degradation in repositories with many PipelineRuns using files.* patterns.

      Background (Required)

      When a webhook event is received, PaC evaluates each PipelineRun's CEL expression to determine if it should be triggered. If the expression references files.*, PaC calls GetFiles() to fetch changed files from the Git provider API.

      GitHub, GitLab, and Bitbucket Data Center providers all implement a cachedChangedFiles field that ensures the file list is fetched only once per event and reused for subsequent evaluations. The Gitea/Forgejo provider does not have this cache, so it makes a fresh API call for every PipelineRun evaluation.

      A customer reported a production case where 132 PipelineRuns with file-based triggers caused a 3 hour 36 minute delay between webhook receipt and first PipelineRun creation, due to serial API calls and HTTP/2 stream errors triggering retry cascades.

      GitHub Issue: https://github.com/openshift-pipelines/pipelines-as-code/issues/2324

      Out of scope

      • Higher-level architectural changes to how CEL evaluation calls GetFiles() (e.g., pre-fetching at the webhook event level)
      • Bitbucket Cloud GetFiles() implementation (currently returns empty results — separate concern)
      • Performance improvements unrelated to GetFiles() caching

      Approach (Required)

      Apply the same caching pattern already used by GitHub, GitLab, and Bitbucket Data Center providers to the Gitea/Forgejo provider. The existing implementations in the other providers serve as the reference pattern.

      Dependencies

      None — the caching pattern is already established in three other providers and can be applied independently.

      Acceptance Criteria (Mandatory)

      1. The Gitea/Forgejo provider caches GetFiles() results so that multiple calls within the same event processing return cached data without additional API calls.
      2. The caching follows the same pattern used by the GitHub, GitLab, and Bitbucket Data Center providers.
      3. Existing Gitea/Forgejo unit tests continue to pass.
      4. A repository with N PipelineRuns using files.* CEL expressions triggers only 1 GetFiles() API call per webhook event, not N.

              cboudjna@redhat.com Chmouel Boudjnah
              cboudjna@redhat.com Chmouel Boudjnah
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: