Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-10879

Quay 3.17 Org Mirror: Org-level mirror lacks tag filtering capability, syncing all tags unconditionally

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • quay-v3.17.0
    • quay, quay-ui
    • False
    • Hide

      None

      Show
      None
    • False

      Summary

      Organization-level mirroring does not support tag filtering. Every tag present in a source repository is unconditionally synced to Quay, with no way for the user to configure tag glob patterns at the org level. By contrast, repository-level mirroring supports per-repo tag filter rules. This omission causes org-mirror to perform vastly more sync work than necessary, particularly for repositories that contain large numbers of transient or non-production tags (CI builds, dev snapshots, etc.).

      Affected File

      • workers/repomirrorworker/_init_.py:1288_get_all_tags_for_org_mirror() returns all tags with no filtering applied

      Bug Details

      For repository-level mirroring, tags are filtered before sync using glob patterns configured on the mirror rule:

      # workers/repomirrorworker/__init__.py
      def tags_to_mirror(skopeo, mirror):
          all_tags = get_all_tags(skopeo, mirror)
          matching_tags = []
          for pattern in mirror.root_rule.rule_value:  # user-configured glob patterns
              matching_tags += [tag for tag in all_tags if fnmatch.fnmatch(tag, pattern)]
          return list(set(matching_tags))
      

      For org-level mirroring, all tags are returned without any filtering:

      # workers/repomirrorworker/__init__.py:1288
      def _get_all_tags_for_org_mirror(skopeo, config, external_reference):
          # ... fetches tags via skopeo ...
          return result.tags  # all tags, no filtering
      

      The caller perform_org_mirror_repo() then syncs every tag in the returned list with no opportunity to apply patterns:

      for tag in tags:  # every tag, no filter
          result = skopeo.copy(src_image, dest_image, ...)
      

      There is no field on OrgMirrorConfig or OrgMirrorRepository to store tag filter patterns, and no UI or API surface to configure them.

      Impact

      Operational:

      • A Harbor repo with 1,000 tags (e.g. CI build artifacts tagged by commit SHA, branch, date) forces 1,000 skopeo copy operations per sync cycle for that single repo
      • Across 10,000 org-mirrored repos at 100 tags each = 1,000,000 skopeo copy operations per sync cycle
      • Sync time, network bandwidth, storage consumption, and destination registry write load all scale with the total unfiltered tag count
      • Workers may never finish a sync cycle before the next one is scheduled

      User experience:

      • Users migrating from repo-level to org-level mirroring lose the tag filtering capability they previously had
      • There is no workaround: org-level mirror configuration has no tag filter field
      • Operators cannot exclude transient tags (e.g. dev-, ci-, sha-*) from being mirrored

      Functional regression:

      • Repo-level mirroring supports tag glob patterns (mirror.root_rule.rule_value)
      • Org-level mirroring has no equivalent, creating a capability gap between the two mirroring modes

      Reproduction Steps

      1. Configure an org-level mirror against a Harbor project
      2. Ensure at least one source repository contains tags matching multiple patterns (e.g. latest, v1.0, dev-abc123, ci-20260101, etc.)
      3. Trigger a sync
      4. Observe that all tags — including transient build tags — are synced to Quay with no filtering

      Expected Behavior

      Org-level mirroring should support configurable tag filter patterns (e.g. glob expressions) at the org mirror config level, consistent with the tag filtering capability available in repo-level mirroring.

      Actual Behavior

      All tags in every discovered repository are synced unconditionally. There is no tag filtering capability at the org mirror level.

      Additional Context

      • Repository filter (glob patterns on repo names) is already supported at the org mirror config level via OrgMirrorConfig.repository_filters. Tag filtering is the analogous capability for tag names within each repo, and is currently absent.
      • Related: PROJQUAY-10878 (discovery phase expiry for large projects) — the absence of tag filtering compounds the total sync volume, making the expiry problem worse.

              Unassigned Unassigned
              lzha1981 luffy zhang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: