Feature Overview (aka. goal summary)

Enable users to configure a single mirroring task to replicate all repositories from a source namespace (e.g., a "Project" in Harbor, an "Organization" in Quay) into a target Quay organization. 

This feature will also support optionally filtering the set of repositories to be mirrored from the source namespace using a list or regular expression.

This ticket reduces the scope of PROJQUAY-9603 to deliver the most critical functionality first.

Goals (aka. expected user outcomes)

The primary goal is to drastically reduce the operational overhead and potential for human error when managing large-scale repository mirroring.  

This feature directly addresses major customer pain points and empowers users to:

• Automate bulk mirroring: Configure a single task to discover and sync all current and future repositories from an entire upstream organization/project.

• Simplify configuration: Avoid the "time consuming and tedious work" of creating and managing individual mirroring rules for hundreds or thousands of repositories.

• Increase reliability: Reduce the risk of manual configuration errors that lead to missed repositories.

• Selectively mirror organizations: Define a subset of repositories (via list or regex) within a source organization to mirror, providing more granular control.

Background

Quay's current repository mirroring feature operates on a per-repository basis.  This limitation presents a significant operational bottleneck for customers and partners.  These users often manage applications composed of hundreds or even thousands of repositories, grouped under a single "Project" (in Harbor) or "Organization" (in Quay).

The current state requires creating and managing a separate mirroring rule for each repository, a process that is "time consuming and tedious" and "prone to manual errors," as highlighted repeatedly in customer feedback.  This feature gap has been cited as a key blocker, a competitive disadvantage, and has contributed to lost business opportunities.

This ticket scopes down the original feature to deliver the most critical functionality first:

1. Scenario 1: Mirroring an entire organization (highest priority).
2. Scenario 2: Filtering repositories within that organization mirror (secondary priority).

Key customer-requested paths include Harbor-to-Quay and Quay-to-Quay (including podman-to-operator and operator-to-operator) mirroring.

Requirements (aka. acceptance criteria):

• Users can create a single mirroring configuration that targets an entire source "Project" (in Harbor) or "Organization" (in a source Quay instance).

• The mirroring task automatically discovers all repositories within the specified source namespace.

• The mirroring task automatically creates corresponding organizations (if they don't exist) and repositories in the target Quay instance.  

• The mirroring configuration supports defining a repository filter (via a list of names or a regular expression) to optionally mirror only a subset of repositories from the source namespace (Scenario 2).

• The creation of organizations and repositories is subject to the permissions of the user or robot account owning the mirroring task.

• Mirroring tasks/jobs are configurable to run on a user-specified interval.

• Quay’s audit events are created and shown in the action logs when organizations and repositories are created as part of this mirroring task, distinguishable from regular creation events.

• The organization-mirroring subsystem exposes Prometheus metrics for monitoring job success, failure, duration, and number of repos synced.

• The entire organization-mirroring feature can be disabled via a configuration toggle (default: enabled).

Use cases (optional):

• Scenario 1: Mirroring an entire "Harbor Project" to a "Quay Organization"
  • As a: Quay admin at a company, 
  • I want to: configure a single mirroring rule that points to a specific "Project" in our upstream Harbor registry.
  • So that: all 1,000+ repositories within that Harbor project are automatically discovered, created, and kept in sync within a specified organization in our Quay deployment, without me needing to create 1,000+ individual rules.

• Scenario 2: Mirroring between "Quay instances" (Quay-to-Quay)
  • As a: Quay admin,
  • I want to: set up a mirroring job on a "downstream" Quay instance (managed by the Quay Operator) that targets an organization in our "central" Quay instance (Podman Quay).
  • So that: all repositories from that central organization are automatically replicated to the downstream instance, ensuring environments stay in sync.

• Scenario 3: Selectively mirroring repositories from a source organization
  • As a: Quay admin,
  • I want to: configure an organization-level mirror from a source (Harbor or Quay) but specify a regex filter (e.g., ^app-prod-.*).
  • So that: only the repositories matching my production criteria are mirrored, while test/dev repositories in the same source organization are ignored.

Out of scope

This phase-1 ticket excludes the following functionality from the original feature ticket (PROJQUAY-9603) to ensure a focused and timely initial delivery:

• Filtering mirrored content based on tags (using lists or regular expressions).
• Mirroring of referrers (e.g., signatures, SBoMs).
• Restricting manifest/image types or os/platform.
• Configuration options for deleting local repositories, tags, or organizations that are removed from the upstream source.
• Advanced error handling configuration (e.g., "stop on error," "re-attempt count").
• Mirroring of registry-specific metadata (e.g., Quay-to-Quay mirroring of organization settings, repository permissions, user/team access).

Documentation considerations

• Documentation explains how to configure the new organization-level mirroring.

• Provide distinct and step-by-step examples for:
  • Mirroring from a Harbor Project to a Quay Organization.
  • Mirroring from a source Quay Organization to a target Quay Organization.

• Document the syntax for the repository filtering (list and regular expression).

• Document the new configuration field for admins to enable/disable the feature.

• Document the new Prometheus metrics and audit/action log events.

Questions to Answer (Optional):

• What are the specific API endpoints required to list all repositories within a Harbor "Project"? (No standard OCI endpoint for this; hence, it needs to be implemented for each registry).

• What is the expected behavior when a repository is deleted from the source organization? (For this scope, we assume the mirrored copy in Quay is left untouched).

• How are credentials for the source registry (Harbor or remote Quay) managed for this organization-level task?

• What default permissions are applied to the newly created repositories in the target Quay organization?

Interoperability considerations

• Harbor: The implementation will depend on a stable Harbor API for enumerating all repositories within a "Project".  This API endpoint and its authentication mechanism must be verified.

• Quay-to-Quay: The feature must support mirroring between different Quay instances and deployment types (e.g., from a podman-based Quay to an Operator-deployed Quay, and vice-versa).  This implies using the standard Quay API for repository enumeration.